CN118196006A - Circuit board element detection method, device, equipment, chip and readable storage medium - Google Patents

Abstract

The invention discloses a circuit board element detection method, a device, equipment, a chip and a readable storage medium, wherein the method comprises the following steps: acquiring a position image to be detected and a template image of an element to be detected; determining main body matching data of the position image to be detected and the template image in a main body area and background matching data of the template image in a background area; and determining a detection result of the element to be detected based on the main body matching data and the background matching data. Therefore, the detection accuracy can be effectively improved by utilizing the matching data of the background area to assist the detection result of the judging element.

Description

Circuit board element detection method, device, equipment, chip and readable storage medium

Technical Field

The present invention relates to the field of circuit boards, and in particular, to a method, an apparatus, a device, a chip, and a readable storage medium for detecting a circuit board element.

Background

In the manufacturing process of circuit boards, it is necessary to place corresponding prescribed elements at different positions on the circuit board. Because the components are prone to various mounting defects such as missing, misloading, skew and the like, the reliability of the electronic product is affected, and therefore the components at corresponding positions need to be detected.

In the related art, an automatic optical inspection method is generally used to automatically inspect a component defect. However, the element detection method in the related art is prone to misjudgment and missed judgment, and the detection accuracy is to be improved.

Disclosure of Invention

The present invention aims to solve at least one of the technical problems in the related art to some extent. Therefore, the invention provides a method, a device, equipment, a chip and a readable storage medium for detecting a circuit board element, which are used for improving the detection accuracy of detecting the circuit board element and improving the recognition accuracy of the conditions of element error, missing part, deflection and the like on the basis of less sample data.

To achieve the above object, an embodiment of a first aspect of the present invention provides a method for inspecting a circuit board element, the method including: acquiring a position image to be detected and a template image of an element to be detected; the to-be-detected position image is an area image, in which preliminary matching data between the to-be-detected initial image and the template image meet preset matching conditions, and the to-be-detected position image and the template image are respectively divided into a main area and a background area according to a preset dividing mode; determining main body matching data of the position image to be detected and the template image in the main body area and background matching data of the template image in the background area; and determining a detection result of the element to be detected based on the main body matching data and the background matching data.

According to one embodiment of the present invention, the position image to be detected is acquired by: determining a window image to be detected in the initial image to be detected according to a preset sliding window; wherein the preset sliding window is determined according to the size of the template image; performing template matching on the template image and the window image to be detected to obtain preliminary matching data between the window image to be detected and the template image; and if the preliminary matching data corresponding to the window image to be detected meets the preset matching condition, determining the window image to be detected as the position image to be detected.

According to one embodiment of the present invention, the performing template matching on the template image and the window image to be detected to obtain preliminary matching data between the window image to be detected and the template image includes: performing pixel correlation matching on the pixel data of the template image and the pixel data of the window image to be detected to obtain pixel matching data of the window image to be detected; performing correlation matching by using edge data obtained by edge detection in the template image and edge data obtained by edge detection of the window image to be detected to obtain edge matching data of the window image to be detected; and determining preliminary matching data between the window image to be detected and the template image according to the pixel matching data and the edge matching data.

According to one embodiment of the present invention, the pixel matching data corresponds to pixel matching weight data, and the edge matching data corresponds to edge matching weight data; the determining preliminary matching data between the window image to be detected and the template image according to the pixel matching data and the edge matching data comprises the following steps: and carrying out weighted summation calculation according to the pixel matching data, the pixel matching weight data, the edge matching data and the edge matching weight data to obtain preliminary matching data of the window image to be detected.

According to one embodiment of the invention, the subject match data and the background match data are determined by: performing template matching in the main body area according to the position image to be detected and the template image to obtain main body matching data; and carrying out template matching in the background area according to the position image to be detected and the template image to obtain the background matching data.

According to one embodiment of the present invention, the preset dividing manner is used for indicating that the background area is set around the main area, and the background area is divided into N sub-areas, where the value of N is greater than or equal to 4.

According to one embodiment of the present invention, if the value of N is equal to 4, the background area includes a first sub-area, a second sub-area, a third sub-area, and a fourth sub-area, where the first sub-area and the second sub-area are located on the left and right sides of the main area; the third sub-area and the fourth sub-area are positioned on the upper side and the lower side of the main body area; or if the value of N is equal to two of 8,8 sub-areas located at the left and right sides of the main body area, two of the 8 sub-areas are located at the upper and lower sides of the main body area, and four of the 8 sub-areas are located at four vertexes of the main body area.

According to an embodiment of the present invention, the determining the detection result of the element to be detected based on the subject matching data and the background matching data includes: determining a first comparison result between the main body matching data and a first preset data threshold value and a second comparison result between the background matching data and a second preset data threshold value; and if the first comparison result indicates that the main body matching data is larger than the first preset data threshold value, and the second comparison result indicates that the background matching data is larger than the second preset data threshold value, obtaining the detection result of the qualified element to be detected.

According to one embodiment of the invention, the background area is divided into N sub-areas, and the value of N is greater than or equal to 4; the background matching data comprise sub-region matching data of the position image to be detected and the template image in any sub-region; the second preset data threshold value comprises the preset area data threshold value of each of the N sub-areas; if the first comparison result indicates that the main body matching data is greater than the first preset data threshold, and the second comparison result indicates that the background matching data is greater than the second preset data threshold, the detection result of the element to be detected is obtained, including: and if the first comparison result indicates that the main body matching data is larger than the first preset data threshold value, and the second comparison result indicates that the sub-region matching data of each sub-region is larger than the preset region data threshold value of each sub-region, obtaining the detection result of the qualified element to be detected.

According to one embodiment of the present invention, the first preset data threshold is obtained from the body matching reference data of the detected element, and the second preset data threshold is obtained from the background matching reference data of the detected element; the detected elements include elements whose detection results are acceptable, and/or elements whose detection results are unacceptable.

According to an embodiment of the present invention, the determining the detection result of the element to be detected based on the subject matching data and the background matching data further includes: performing difference calculation according to the main body matching data and the preliminary matching data corresponding to the position image to be detected to obtain first difference data; performing difference calculation according to the background matching data and the preliminary matching data corresponding to the position image to be detected to obtain second difference data; and if the first difference data is larger than a third preset data threshold value and the second difference data is larger than a fourth preset data threshold value, obtaining the detection result of the element to be detected.

To achieve the above object, according to a second aspect of the present invention, there is provided a circuit board element inspection apparatus including: the image acquisition module is used for acquiring a position image to be detected and a template image of the element to be detected; the to-be-detected position image is an area image, in which preliminary matching data between the to-be-detected initial image and the template image meet preset matching conditions, and the to-be-detected position image and the template image are respectively divided into a main area and a background area according to a preset dividing mode; the matching data determining module is used for determining main body matching data of the position image to be detected and the template image in the main body area and background matching data of the background area; and the detection result determining module is used for determining the detection result of the element to be detected based on the main body matching data and the background matching data.

To achieve the above object, an embodiment of a third aspect of the present invention provides a computer device, including a memory and a processor, where the memory stores a first computer program, and the processor executes the first computer program to implement the steps of the circuit board element detection method according to any one of the foregoing embodiments.

To achieve the above object, a fourth aspect of the present invention provides a chip, including a storage unit and a processing unit, where the storage unit stores a second computer program, and the processing unit implements the steps of the method for detecting a circuit board element according to any one of the foregoing embodiments when executing the second computer program.

To achieve the above object, a fifth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the circuit board element detection method according to any one of the foregoing embodiments.

According to the embodiments provided by the invention, the initial image to be detected and the template image are subjected to preliminary matching based on the preset matching condition so as to obtain the position image to be detected, and the optimal matching position can be quickly positioned in the initial image to be detected and used as the position of the element for detection. Then dividing a main body area and a background area by the position image to be detected and the template image, and further carrying out accurate matching based on the main body area and the background area, so as to assist in judging the detection result of the element by utilizing the matching data of the background area, improve the detection accuracy and precision, ensure the judgment accuracy of the element error, missing element, deflection and other conditions, and reduce the error of manual judgment.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

Fig. 1 is a schematic flow chart of a circuit board element detection method according to an embodiment of the present disclosure.

Fig. 2 is a schematic flow chart of acquiring a position image to be detected according to an embodiment of the present disclosure.

Fig. 3 is a schematic flow chart of obtaining preliminary matching data according to an embodiment of the present disclosure.

Fig. 4a is a schematic flow chart of determining subject matching data and background matching data according to one embodiment of the present disclosure.

Fig. 4b is a schematic diagram showing the division of a main area and a background area according to an embodiment of the present disclosure.

Fig. 4c is a schematic diagram of a division of a body region and a background region according to an embodiment of the present disclosure.

Fig. 4d is a schematic diagram of a division of a main area and a background area according to an embodiment of the present disclosure.

Fig. 5 is a schematic flow chart of determining a detection result according to an embodiment of the present disclosure.

Fig. 6a is a schematic flow chart of determining a detection result according to another embodiment of the present disclosure.

Fig. 6b is a schematic flow chart of a circuit board element detection method according to another embodiment of the present disclosure.

Fig. 7 is a block diagram of a circuit board element detecting device according to an embodiment of the present disclosure.

Fig. 8 is a block diagram of a computer device provided according to one embodiment of the present description.

Fig. 9 is a block diagram of a chip provided according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.

The quality of the printed circuit board (Printed Circuit Board, PCB) as an important component of the electronic product has a relatively decisive role in the quality and performance of the whole electronic product, so that the production process of component mounting (PCBA, printed Circuit Board Assembly, PCB blank on surface mount technology or whole process of double-sided card technology card) on the PCB has higher standard requirements. In the production and manufacturing process of the PCB, corresponding specified elements are required to be placed at different positions on the PCB, and due to the fact that the elements are prone to defects of various types such as missing, misloading and skew, the defects can affect the reliability of electronic products, the elements at corresponding positions are required to be detected to judge whether the elements are wrong, leaked and deflected.

At present, the detection method of the circuit board element mainly adopts a manual detection method and an AOI (Automated Optical Inspection, automatic optical detection) detection method. The manual detection has the defects of low efficiency, higher cost, high omission rate and high false detection rate. In the related art, the AOI detection method mainly comprises a positioning algorithm based on traditional image processing and a positioning algorithm based on deep learning.

Localization algorithms based on conventional image processing typically design related component inspection features and inspection processes by human means to form solutions for automatic detection of component defects. However, the method has the problems of good detection capability in a specific environment scene and poor generalization capability, and when the environment illumination, the element shooting angle and the like are changed, the situation of poor detection capability possibly occurs, so that the positioning result of the element is inaccurate. Therefore, when the conditions of wrong parts, missing parts, deflection and the like of the elements are judged through the traditional image matching, color distinguishing and other modes, the conditions of misjudgment and missing judgment are easy to occur, and therefore, after the conditions of wrong parts, missing parts, deflection and the like of the elements are judged by the AOI detection equipment, the elements are usually confirmed through manpower.

Deep learning based positioning algorithms typically require extensive positive and negative sample data for correlated deep learning network training to develop good detection capabilities. The method has strong characteristic extraction capability and strong fitting capability, so that the method has strong generalization capability. However, since deep learning networks rely primarily on statistical rules in the learning dataset rather than well-defined logic rules, for some situations where decisions need to be made following specific manual decision rules, it may be difficult for deep learning networks to directly encode these rules into the network structure. Meanwhile, in order to ensure that the deep learning network can stably and accurately execute the defect positioning task of the element under different scenes so as to obtain a stable detection effect, a large amount of training data samples under various conditions are generally required to be collected and expanded.

In order to improve the detection accuracy of the detection of the circuit board element, and improve the recognition accuracy of the element error, missing element, deflection and other conditions on the basis of less sample data, it is necessary to provide a circuit board element detection method, device, equipment, chip and readable storage medium. The method comprises the steps of firstly obtaining a template image of an element to be detected, and extracting an image of a region to be detected from an initial image to be detected. The region is obtained by preliminary matching of the initial image to be detected and the template image, and when preliminary matching data for indicating similarity or characteristic correspondence between the image of the region and the template image and the like meets a preset matching condition, the region can be regarded as a region containing the element to be detected, and the image of the region is taken as the position image to be detected. Secondly, dividing the position image to be detected and the template image into a main area and a background area according to a preset dividing mode. The main area is an area reflecting the position of the key structure of the element body in the image, and the background area is other area parts except the main area in the image. Then, for the subject region, subject matching data of the position image to be detected and the template image within the subject region is calculated, and for the background region, background matching data of the position image to be detected and the template image within the background region is calculated. Finally, based on the obtained main body matching data and background matching data, comprehensively judging whether the to-be-detected position image contains the to-be-detected element or not, and whether the state of the to-be-detected element reaches a certain degree of similarity with the standard state or the reference state of the to-be-detected element represented by the template image or not. If the to-be-detected position image is judged to contain the to-be-detected element and the installation state of the to-be-detected element and the standard state represented by the template image reach a certain degree of similarity, the to-be-detected image can be considered to have no conditions of wrong parts, missing parts, deflection and the like of the to-be-detected element, otherwise, the to-be-detected position image can be considered to have the problem of installation defects of wrong parts, missing parts, deflection and the like of the to-be-detected element.

Therefore, the initial image to be detected and the template image are subjected to preliminary matching based on preset matching conditions, so that a position image to be detected is obtained, and the optimal matching position can be quickly positioned in the initial image to be detected and used as the position of the element for further accurate detection. The main area and the background area are divided by the position image to be detected and the template image, and further accurate matching is carried out based on the main area and the background area, so that matching data are calculated respectively, the detection result of the judging element is assisted by using the matching data of the background area, key characteristics of the element can be compared more accurately, the detection precision is improved, the judgment accuracy of the conditions of element missing, missing element, deflection and the like is ensured, and meanwhile, the error of artificial judgment is reduced. The circuit board element detection method provided by the specification can detect missing parts, wrong parts, deflection and the like of the element under the condition of less data, can rapidly detect the element under the condition that only one positive sample is needed, can be suitable for identifying the missing deflection of the element or the object under a fixed scene, and provides better robustness.

In the circuit board element detection method provided by the specification, the position image to be detected is obtained by performing template matching on the initial image to be detected and the template image. The template matching comprises pixel correlation matching based on pixel data in the image and edge correlation matching based on edge data obtained by edge detection of the image. By adding edge detection and edge correlation matching, the recognition capability of the contour edge and the structural edge of the element in a complex environment scene can be effectively improved, so that the accuracy of preliminary matching data of the position image to be detected is improved, and the accuracy of a final element detection result is improved.

Further, the main body matching data is obtained by performing template matching in the main body area according to the position image to be detected and the template image, the background matching data is obtained by performing template matching in the background area according to the position image to be detected and the template image, and the template matching mode is similar to the above. By respectively carrying out template matching on the main body area and the background area, the method can further focus on key features of the element body to judge the integrity, the correctness and the consistency of the element body, and meanwhile, whether the background of the position of the element accords with the expectation can be evaluated, so that the method is favorable for reducing the conditions of false detection and omission and improving the accuracy and the reliability of detection of the circuit board element.

According to the circuit board element detection method provided by the specification, the background area can be divided into N sub-areas according to a preset division mode, wherein the value of N is greater than or equal to 4. For each sub-region, a respective preset region data threshold value of each sub-region can be set, and template matching can be performed on each sub-region according to the position image to be detected and the template image to obtain sub-region matching data. When the element to be detected is detected, if the main body matching data is larger than a first preset data threshold value and the sub-region matching data of each sub-region is larger than a preset region data threshold value of each sub-region, the detection result can be determined to be qualified for the element to be detected. Therefore, the background area is divided into N sub-areas, each sub-area has clear characteristics and detection standards, detail change of the background area can be better captured, and erroneous judgment is reduced. And setting a respective preset region data threshold value for each sub-region, so that the circuit board element detection method provided by the specification has better robustness to different types of background interference. And setting multiple condition constraints through the first preset data threshold and the preset area data threshold of each sub-area, so that the accuracy and the reliability of detection are further improved, and the decision reliability of a detection result is further improved.

Still further, the circuit board element detection method provided in the present specification determines a detection result of the element to be detected by calculating first difference data between the main body matching data and preliminary matching data of the position image to be detected, and calculating second difference data between the background matching data and the preliminary matching data, and according to a comparison result between the first difference data and a third preset data threshold, and a comparison result between the second difference data and a fourth preset data threshold. The second difference data may include difference data between sub-region matching data and preliminary matching data of each sub-region, and the fourth preset data threshold may include a respective sub-region difference threshold of each sub-region. Thus, by calculating the first difference data of the subject matching data and the preliminary matching data, it is possible to more finely analyze the degree of correlation between the structural features of the position image to be detected in the subject region and the key structural features of the element to be detected contained in the subject region by the template image. Likewise, by calculating the second difference data between the background matching data and the preliminary matching data, it is possible to more finely analyze the degree of correlation between the background feature information contained in the background area of the position image to be detected and the background feature information contained in the background area of the template image. By using the difference data as the detection result judgment standard of the element to be detected, the similarity between the position image to be detected and the template image can be more accurately evaluated, and the possibility of misjudgment is reduced, so that the accuracy and the reliability of detection are improved.

The embodiment of the present disclosure provides a circuit board element detection method, which may include the following steps with reference to fig. 1.

S110, acquiring a position image to be detected and a template image of an element to be detected; the to-be-detected position image is an area image of which preliminary matching data between the to-be-detected initial image and the template image meets a preset matching condition, and the to-be-detected position image and the template image are respectively divided into a main area and a background area according to a preset division mode.

And S120, determining main body matching data of the position image to be detected and the template image in a main body area and background matching data of the template image in a background area.

S130, determining a detection result of the element to be detected based on the main body matching data and the background matching data.

The initial image to be detected may be an image of the circuit board obtained by an optical imaging technology, or an image of a region to be detected of the circuit board (i.e., a region where the element to be detected should be correctly mounted in an ideal state), including information of the element that may exist and other background information.

The template image is obtained according to a standard picture of an ideal state or a reference state of the element to be detected, wherein the standard picture can be obtained according to a design drawing or a specification, a correct installation example of the element to be detected, and the like.

The position image to be detected is extracted from the initial image to be detected. The preliminary matching data of the position image to be detected is used for indicating the matching degree of the whole image between the position image to be detected and the template image.

The preset division manner can be determined according to the body distribution area (the distribution area of the key structure of the element body) of the element to be detected in the template image. In this specification, the preset division manner may also be referred to as a preset structure division manner.

The subject region is a region of the image that is considered to contain critical structures of the element's body, and the background region may include other regions of the image than the subject region, and is considered to contain background information and/or non-critical structures of the element (i.e., the flared portion of the element).

It will be appreciated that the size of the initial image to be detected is larger than the size of the template image, and the size of the position image to be detected is the same as the size of the template image.

Specifically, a template image of the element to be detected is prepared in advance based on a standard picture of the element to be detected. The image of the area to be detected of the circuit board (i.e., the image of the area which is considered to contain the element to be detected according to the correct mounting example of the element on the circuit board) is obtained as an initial image to be detected, and the initial image to be detected and the template image can be subjected to preliminary matching by an image processing method (e.g., an image matching method) to obtain preliminary matching data between each area image in the initial image to be detected and the template image. If the preliminary matching data between a certain area image in the initial image to be detected and the template image is maximum, the preset matching condition is met, and the area image is used as the position image to be detected.

And setting a preset dividing mode according to the body distribution area of the element to be detected in the template image, so as to divide the position image to be detected and the template image into a main area and a background area respectively according to the preset dividing mode. For the main body area, the similarity of the position image to be detected and the template image in the main body area can be analyzed to obtain main body matching data. For the background area, the background similarity of the position image to be detected and the template image in the background area can be analyzed to obtain background matching data. By comprehensively analyzing the main body matching data and the background matching data, whether the element to be detected is mounted in the position image to be detected or not can be judged, namely, whether the element to be detected is mounted in the position image to be detected (or the initial image to be detected) or not can be judged, and indexes such as whether the mounting mode of the element to be detected is correct, whether the quality of welding spots is qualified, whether the model of the element is consistent with the design or not and the like can be judged, so that quality problems such as wrong piece, missing piece, deflection and the like of the element to be detected in the position image to be detected can be judged, and the detection result of the element to be detected can be obtained.

In some embodiments, a preset subject matching data threshold may be set for the subject area, and a preset background matching data threshold may be set for the background area, so as to determine a detection result of the element to be detected in the position image to be detected according to a comparison result between the subject matching data and the preset subject matching data threshold, and a comparison result between the background matching data and the preset background matching data threshold.

Illustratively, a corresponding match data threshold is set for the subject region, denoted as threshold 1, and a corresponding match data threshold is set for the background region, denoted as threshold 2. And marking the main body matching data of the obtained position image to be detected and the template image in the main body area as D1, and marking the background matching data of the position image to be detected and the template image in the background area as D2. Comparing the main body matching data D1 with a threshold value 1, and comparing the background matching data D2 with a threshold value 2, if the main body matching data D1 is larger than the threshold value 1 and the background matching data D2 is larger than the threshold value 2, the element to be detected in the position image to be detected can be considered to be qualified. If the subject matching data D1 is equal to or less than the threshold value 1, or if the background matching data D2 is equal to or less than the threshold value 2, the element to be detected in the position image to be detected may be considered as failed.

In other embodiments, a preset matching data threshold may be set, and target matching data is obtained according to the main matching data and the background matching data, so as to determine a detection result of the element to be detected according to a comparison result between the target matching data and the preset matching data threshold.

Illustratively, a preset match data threshold is set, noted as threshold Th. And marking the main body matching data of the obtained position image to be detected and the template image in the main body area as D1, and marking the background matching data of the position image to be detected and the template image in the background area as D2. The subject matching data D1 and the background matching data D2 may be subjected to addition calculation, average calculation, weighted summation calculation, or the like to obtain target matching data, which is denoted as FD. Comparing the target matching data FD with a threshold Th, and if the target matching data FD is larger than the threshold Th, considering that the element to be detected in the position image to be detected is qualified; if the target matching data FD is equal to or smaller than the threshold Th, the element to be detected in the position image to be detected may be considered as failed.

Illustratively, the above-described threshold Th, subject matching data D1, and background matching data D2 are taken as examples. For the subject matching data D1 and the background matching data D2, the smallest matching data among them may also be regarded as target matching data. Assuming that the subject matching data D1 is larger than the background matching data D2, the background matching data D2 may be regarded as target matching data. And comparing the background matching data D2 with a threshold Th, and if the background matching data D2 is larger than the threshold Th, considering that the element to be detected is qualified.

It will be appreciated that in the process of making a template image from a standard picture of an element to be detected, a process of noise reduction, smoothing, etc. may be included, as well as a process of dividing the processed image into a body portion and a background portion according to a body distribution area of the element to be detected (i.e., a process of dividing into a body area and a background area in a preset division manner).

After the position image to be detected is obtained, the position image to be detected is divided into a corresponding main body area and a background area according to the same structural division mode as that of the template image, the size of the main body area of the position image to be detected is the same as that of the background area of the template image, and the size of the background area is the same as that of the main body area of the template image.

The circuit board may be a printed circuit board, or may be another circuit board with a requirement for detecting an installation condition of a component, and the initial matching data, the main matching data, and the background matching data may be obtained by any one of image matching modes such as feature matching, template matching, and matching based on deep learning, which is not particularly limited in this specification.

In the above embodiment, the initial image to be detected and the template image are initially matched based on the preset matching condition to obtain the position image to be detected, so that the best matching position possibly containing the element to be detected can be quickly positioned in the initial image to be detected as the region position image for subsequent detection. The main area and the background area are divided by the position image to be detected and the template image, matching data are calculated for the main area and the background area respectively, so that the matching data of the background area are utilized to assist in determining the element matching result of the position image to be detected and the template image, the detection result of the element to be detected is obtained, more accurate comparison between key features of the element to be detected and the background features of the element to be detected can be realized, the detection precision is improved, the judgment accuracy of the element error, missing piece, deflection and other conditions is ensured, the possibility of misjudgment and missing judgment is reduced, meanwhile, the artificial participation degree is reduced, and the error of manual judgment is reduced.

In some embodiments, referring to fig. 2, the position image to be detected may be acquired by:

S210, determining a window image to be detected in an initial image to be detected according to a preset sliding window; the preset sliding window is determined according to the size of the template image.

And S220, performing template matching on the template image and the window image to be detected to obtain preliminary matching data between the window image to be detected and the template image.

And S230, determining the window image to be detected as a position image to be detected if the preliminary matching data corresponding to the window image to be detected meets the preset matching condition.

The preset sliding window may also be referred to as a search window or a scan window. The window image to be detected is an area image in a window at the position where the preset sliding window is located.

Specifically, a preset sliding window matching the size of the template image may be set according to the size of the template image. The preset sliding window can be moved in a certain step length in the initial image to be detected, and after each movement, the area image of the current position of the preset sliding window can be extracted to be used as the image of the window to be detected. And carrying out correlation matching on the template image and each window image to be detected by using a template matching mode aiming at each window image to be detected, so that correlation data between each window image to be detected and the template image can be obtained and used as preliminary matching data of each window image to be detected. If the preliminary matching data of a certain window image to be detected meets the preset matching condition, that is, the preliminary matching data of the certain window image to be detected is the maximum value in the preliminary matching data of all the window images to be detected, the window image to be detected can be determined as the position image to be detected.

In some embodiments, template matching may be performed on the template image and the window image to be detected by comparing pixel correlations.

Illustratively, it is assumed that the window image to be detected includes window image 1, window images 2, … …, window image n. For the window image 1, a pixel similarity value between the pixel value in the template image and the pixel value in the window image 1 may be calculated, denoted as PD1, as preliminary matching data between the window image 1 and the template image. For the window image 2, a pixel similarity value between the pixel value in the template image and the pixel value in the window image 2 may be calculated, denoted as PD2, as preliminary matching data between the window image 2 and the template image. And so on, preliminary matching data PDn between the window image n and the template image can be calculated. Assuming that the value of preliminary matching data PD5 between the window image 5 and the template image is the largest among the n window images, the preliminary matching data of the window image 5 satisfies a preset matching condition, and the window image 5 is determined as the position image to be detected.

It will be appreciated that the pixel similarity value between the template image and the window image to be detected may be calculated by any one of the ways of euclidean distance, hamming distance, or normalized cross-correlation.

In other embodiments, template matching may also be performed on the template image and the window image to be detected by comparing the correlation of the features such as color, shape, texture, and the like.

Illustratively, taking the above n window images as an example, for the window image 1, the preliminary matching data between the window image 1 and the template image may be obtained by extracting image features such as colors, shapes, textures, and the like in the template image and the window image 1, respectively, and calculating a similarity value between the image features in the template image and the image features in the window image 1. For the window image 2, the similarity value between the image features in the template image 2 and the image features in the window image 2 can be calculated by extracting the image features in the template image and the window image respectively, as preliminary matching data between the window image 2 and the template image. And so on, preliminary matching data between the window image n and the template image can be calculated. The description of determining the position image to be detected is similar to the above description, and detailed description thereof will be omitted.

In this specification, the process of determining the position image to be detected in the initial image to be detected may be regarded as a process of locating the region possibly including the element to be detected in the initial image to be detected.

In some embodiments, referring to fig. 3, performing template matching on the template image and the window image to be detected to obtain preliminary matching data between the window image to be detected and the template image may include the following steps.

And S310, performing pixel correlation matching on pixel data of the template image and pixel data of the window image to be detected to obtain pixel matching data of the window image to be detected.

And S320, performing correlation matching by using edge data obtained by edge detection in the template image and edge data obtained by edge detection of the window image to be detected, so as to obtain edge matching data of the window image to be detected.

S330, preliminary matching data between the window image to be detected and the template image is determined according to the pixel matching data and the edge matching data.

The pixel data, namely pixel values, can be used for indicating the matching degree or the correlation degree between the template image and the window image to be detected on the pixel level of the whole image, and can be used for evaluating the consistency of the template image and the window image to be detected in the aspects of whole tone, texture and the like.

The edge data, namely pixel values of details, contours and the like extracted from the image, can be used for indicating the matching degree or the correlation degree between the template image and the window image to be detected on the edge characteristic layer of the whole image, and can be used for evaluating the similarity of the template image and the window image to be detected in the aspects of element structure, shape, orientation and the like.

Specifically, for the window image to be detected, by performing pixel correlation matching on pixel data of pixels or pixel blocks at corresponding positions in the template image and the window image to be detected, the similarity between the pixels or pixel blocks at the corresponding positions can be calculated, and a similarity value for representing the matching degree of the template image and the window image to be detected at the pixel level is obtained and is used as pixel matching data of the window image to be detected. And simultaneously, edge detection is carried out on the template image to obtain edge data of edge characteristics in the template image, and edge detection is carried out on the window image to be detected to obtain edge data of the edge characteristics in the window image to be detected. And performing correlation matching on the edge data of the template image and the edge data of the window image to be detected, and calculating the similarity between the edge characteristics of the template image and the edge characteristics of the window image to be detected to obtain an edge characteristic similarity value which is used as the edge matching data of the window image to be detected. By combining the pixel matching data and the edge matching data, preliminary matching data of the window image to be detected can be obtained.

In some embodiments, an edge detection method based on a Sobel operator (or a Sobel edge detection method) may be used to perform edge detection on the template image and the window image to be detected, respectively.

Illustratively, pixel correlation matching is performed on pixel data of the template image and pixel data of the window image to be detected, so as to obtain pixel matching data of the window image to be detected, which is denoted as PM. And (3) assuming that a filter with the size of 3 multiplied by 3 is adopted to respectively perform edge detection based on a Sobel operator on the template image and the window image to be detected, and performing correlation matching on edge data in the two images after edge detection to obtain edge matching data of the window image to be detected, and marking the edge matching data as EM. According to the pixel matching data PM and the edge matching data EM, summation operation can be carried out on the pixel matching data PM and the edge matching data EM, and preliminary matching data between the window image to be detected and the template image can be obtained.

For example, average operation may be performed on the pixel matching data PM and the edge matching data EM to obtain preliminary matching data of the window image to be detected.

For example, if the pixel matching data PM is larger than the edge matching data EM, the edge matching data EM may be used as preliminary matching data of the window image to be detected. Or if the pixel matching data PM is smaller than the edge matching data EM, the pixel matching data PM may be used as preliminary matching data of the window image to be detected.

For example, the pixel matching data PM and the edge matching data EM may be weighted and summed to obtain preliminary matching data of the window image to be detected.

The size of the filter used in the edge detection method based on the Sobel operator may be determined according to an actual application scenario or an application requirement, and the edge detection method may also be any one of Canny edge detection, prewitt edge detection, and the like, which is not particularly limited in this specification.

In this specification, pixel correlation matching is performed on pixel data of a template image and pixel data of a window image to be detected, which may also be understood as standard correlation matching of pixels of an overall image, and pixel matching data may also be referred to as an overall position matching score.

The correlation matching is performed by using edge data obtained by edge detection in a template image and edge data obtained by edge detection in a window image to be detected, and can also be understood as standard correlation matching (or edge post-detection matching) based on edge characteristics, and the edge matching data can also be called edge position matching score or edge post-detection matching score.

In the embodiment, by adding the correlation matching based on edge detection in the template matching, the position matching precision of the obtained position image to be detected can be improved to a certain extent, so that the matching effect of the template image and the position image to be detected is improved. Through double matching of the pixel level and the edge level, the similarity or the matching degree between the window image to be detected and the template image can be more comprehensively evaluated, so that a more reliable result is provided, and errors caused by adopting a single evaluation mode are reduced. Meanwhile, under the complex conditions of illumination change, shadow or partial shielding and the like, the matching method combining the pixel and the edge information has better robustness, can improve the adaptability and stability of template matching, and is beneficial to reducing false detection rate and omission rate.

In some embodiments, the pixel match data corresponds to pixel match weight data and the edge match data corresponds to edge match weight data. Determining preliminary matching data between the window image to be detected and the template image according to the pixel matching data and the edge matching data may include: and carrying out weighted summation calculation according to the pixel matching data, the pixel matching weight data, the edge matching data and the edge matching weight data to obtain preliminary matching data of the window image to be detected.

Specifically, according to the influence degree of the pixel information and the edge information of the image on the final matching result of the template matching, corresponding weight data may be set for the pixel information and the edge information, respectively, that is, the pixel matching weight data corresponding to the pixel matching data and the edge matching weight data corresponding to the edge matching data may be set. According to the obtained pixel matching data and edge matching data of the window image to be detected, weighting summation calculation can be carried out according to the pixel matching data, the pixel matching weight data, the edge matching data and the edge matching weight data so as to obtain preliminary matching data of the window image to be detected.

Illustratively, the pixel matching weight data is set to λ1 and the edge matching weight data is set to λ2. According to the obtained pixel matching data PM and edge matching data EM of the window image to be detected, the pixel matching data PM and the edge matching data EM are combined through weighted summation calculation, preliminary matching data of the window image to be detected can be obtained through calculation according to PM×λ1+EM×λ2, so that the duty ratio of the pixel matching data PM is λ1, and the duty ratio of the edge matching data EM is λ2. Wherein λ1+λ2=1.

In some embodiments, the values of the pixel matching weight data λ1 and the edge matching weight data λ2 may both take 0.5.

Note that, the values of the pixel matching weight data and the edge matching weight data may be set according to an actual application scenario or an application requirement, and the present disclosure is not limited specifically.

In the above embodiment, by allocating different weights for different features (pixel information and edge information), template matching between the template image and the window image to be detected can adapt to different application scenes and requirements, and can meet matching requirements under different environments or conditions, so as to improve robustness of template matching.

In some embodiments, referring to fig. 4a, the subject match data and the background match data may be determined by:

And S410, performing template matching in a main body area according to the position image to be detected and the template image to obtain main body matching data.

And S420, performing template matching in a background area according to the position image to be detected and the template image to obtain background matching data.

Specifically, the correlation matching can be performed on the portion of each of the position image to be detected and the template image in the main body region by using a template matching mode to obtain main body matching data, and the correlation matching can be performed on the portion of each of the position image to be detected and the template image in the background region by using a template matching mode to obtain background matching data.

In some embodiments, the template matching for the subject region and the template matching for the background region may be achieved by combining the pixel correlation matching from the pixel data and the correlation matching from the edge data as described above.

For example, pixel correlation matching may be performed on pixel data of the position image to be detected in the main area and pixel data of the template image in the main area, so as to obtain a pixel matching score of the main area, which is denoted as PM1. It is assumed that Sobel edge detection is performed on a main area of a template image and a main area of a position image to be detected, respectively, using a 3×3-size filter, and correlation matching is performed on edge data in the main areas of the two images after edge detection, so as to obtain an edge matching score of the main area, which is denoted as EM1. And carrying out weighted summation calculation according to the pixel matching score PM1, the pixel matching weight data lambda 1, the edge matching score EM1 and the edge matching weight data lambda 2 of the main body area, so as to obtain main body matching data.

And carrying out pixel correlation matching on pixel data of the position image to be detected in the background area and pixel data of the template image in the background area to obtain a pixel matching score of the background area, and marking the pixel matching score as PM2. And (3) respectively carrying out Sobel edge detection on a main area of the template image and a background area of the position image to be detected by adopting a filter with the size of 3 multiplied by 3, carrying out correlation matching on edge data in the background areas of the two images after edge detection, and obtaining an edge matching score of the background area, which is marked as EM2. And carrying out weighted summation calculation according to the pixel matching score PM2, the pixel matching weight data lambda 1, the edge matching score EM2 and the edge matching weight data lambda 2 of the background area, so as to obtain background matching data.

In other embodiments, the template matching may be performed on the main area and the background area by comparing the correlation of pixels or comparing the correlation of features such as color, shape, texture, etc., which will not be described in detail.

It should be noted that, the subject matching data may also be referred to as a subject matching score or a subject fusion matching score, and the background matching data may also be referred to as a background matching score or a background fusion matching score. Since the subject region is determined according to the body distribution region of the element to be detected in the template image, the subject matching data may also be referred to as body matching data or body matching score.

In the present specification, a process of performing template matching between a template image and a window image to be detected is referred to as a rough matching process, and a process of performing template matching in a main area according to a position image to be detected and a template image and performing template matching in a background area according to a position image to be detected and a template image is referred to as a fine matching process.

In the above embodiment, considering that the template matching method in the related art mainly focuses on the similarity between the element body part in the template image and the corresponding region part in the position image to be detected, the influence of the background information is usually ignored or simplified, and mismatching or low matching precision is easily caused, so the circuit board element detection method provided in the present specification adds the relevance judgment of the background related information in the template matching. By considering the correlation of the body portion features of the element to be detected while fully considering the features and correlation of the surrounding environment (i.e., background information) of the element to be detected, a more comprehensive matching metric can be provided to improve the accuracy of the matching result. Meanwhile, in a fixed scene, the background of the element to be detected is often relatively fixed or has specific rules to be circulated, so that the background information is similar in nature, and by incorporating the background information into template matching calculation, jump information between an element body and the background can be enhanced, so that the influence of background interference is reduced, the matching precision is improved, and the robustness and accuracy of element detection are effectively improved.

In addition, on the basis of rough matching, the position image to be detected obtained by rough matching is decomposed into a background and a body, the installation condition of the element is assisted and judged by utilizing the correlation between the position image to be detected and the background of the template image, particularly when the element is inclined to a certain extent, obvious differences exist in the background area, and the matching result of the background area is obviously changed, so that the accuracy of the detection result can be effectively improved by splitting the body and the background. In the process of fine matching, in order to ensure the operation efficiency, the whole matching position is not judged any more, the corresponding matching score calculation is only carried out on the region position image obtained after the rough matching, and compared with the method for detecting the element based on the template matching in the related technology, the detection method combining the rough matching and the fine matching in the specification does not increase too much calculated amount, so that the calculation and the detection speed are not influenced too much.

By adopting the template matching mode, the whole demand of the data sample can be reduced, the relevant detection can be carried out only by using the standard normal sample of the element to be detected, and the method has better application scene under the scene with less data quantity.

In some embodiments, the preset dividing manner is used for indicating that the background area is set around the main area, and the background area is divided into N sub-areas, where the value of N is greater than or equal to 4.

Wherein, main part region and background region are rectangle.

Specifically, in order to facilitate region division and correlation matching for each region, it is possible to set the main body region as a central region, set the background region around the main body region, and further divide the background region into not less than 4 sub-regions, as a preset division manner.

For example, for a background area other than the main body area, the background area may be divided by areas located at positions of upper, lower, left, and right sides of the main body area to obtain 4 sub-areas.

For the background region, the background region may be divided into upper, lower, left, and right regions in positions above, below, left, and right the main body region. Further, the upper region may be divided into m1 parts and the lower region into m2 parts (m 1> 1, m2 > 1) to divide into the final m1+m2+2 sub-regions. Alternatively, the upper region may be divided into m1 parts, the lower region into m2 parts, the left region into m3 parts, and the right region into m4 parts (m 1> 1, m2 > 1, m3 > 1, m4 > 1) to obtain the final m1+m2+m3+m4 sub-regions.

It should be noted that, the dividing manner of the background area may be determined according to an actual application scenario or an application requirement, and the description is not limited specifically.

In some embodiments, if the value of N is equal to 4, the background area includes a first sub-area, a second sub-area, a third sub-area, and a fourth sub-area, where the first sub-area and the second sub-area are located on the left and right sides of the main area; the third sub-area and the fourth sub-area are positioned on the upper side and the lower side of the main body area. Or if the value of N is equal to 8, two of 8 sub-areas are positioned on the left side and the right side of the main body area, two of 8 sub-areas are positioned on the upper side and the lower side of the main body area, and four of 8 sub-areas are positioned at four vertexes of the main body area.

Specifically, if the value of N is equal to 4, that is, if the background area is divided into 4 sub-areas, the first sub-area and the second sub-area may be obtained by dividing the background area into areas located on the left and right sides of the main area, and the third sub-area and the fourth sub-area may be obtained by dividing the background area into areas located on the upper and lower sides of the main area.

If the value of N is equal to 8, that is, if the background area is divided into 8 sub-areas, 4 sub-areas can be obtained by dividing the background area into areas at four vertexes of the main area; in other regions than the 4 sub-regions, 2 sub-regions may be divided according to regions located at both left and right sides of the main body region, and 2 sub-regions may be divided according to regions located at both upper and lower sides of the main body region.

Illustratively, referring to fig. 4b, the region corresponding to reference numeral 5 in the figure is the subject region. The background area may be divided into 4 sub-areas, including sub-area 1 corresponding to mark 1, sub-area 2 corresponding to mark 2, sub-area 3 corresponding to mark 3, and sub-area 4 corresponding to mark 4 in the figure. Wherein, subregion 2 is located the left side of main body region, and subregion 3 is located the right side of main body region, and subregion 1 is located the upside of main body region, and subregion 4 is located the downside of main body region.

For example, referring to fig. 4c, the region corresponding to the mark 5 in the figure is a main region, and the background region may be further divided into the sub-region 1 corresponding to the mark 1, the sub-region 2 corresponding to the mark 2, the sub-region 3 corresponding to the mark 3, and the sub-region 4 corresponding to the mark 4 in the figure.

Illustratively, referring to fig. 4d, the region corresponding to reference numeral 5 in the figure is the subject region. The background area may be divided into 8 sub-areas, including sub-area 1 corresponding to mark 1, sub-area 2 corresponding to mark 2, sub-area 3 corresponding to mark 3, sub-area 4 corresponding to mark 4, sub-area 6 corresponding to mark 6, sub-area 7 corresponding to mark 7, sub-area 8 corresponding to mark 8, and sub-area 9 corresponding to mark 9 in the figure. Wherein, subregion 1, subregion 3, subregion 7, subregion 9 are located four summits of main body region respectively, and subregion 2 is located the upside of main body region, and subregion 8 is located the downside of main body region, and subregion 4 is located the left side of main body region, and subregion 6 is located the right side of main body region.

It can be understood that if the value of N is equal to 8, the preset division manner can be understood as a division manner similar to a nine-square grid.

Note that, if the value of N is equal to 8, since the blocks of the background portion are 8 blocks, the number of the four corner partial regions (refer to the sub-regions 1, 3, 7, and 9 in fig. 4 d) is relatively large, so that the blocks of the background portion can be converted into 4 sub-regions (refer to the sub-regions 1,2, 3, and 4 in fig. 4b or 4 c) to achieve partial simplification. Therefore, if the value of N is equal to 4, the 4 sub-regions can be understood as being obtained by fusing the 4 sub-regions located at the four corner portions into other sub-regions to perform sub-region simplification after dividing into 8 sub-regions of the main region and the background region in a nine-grid-like manner.

Further, the background matching data includes matching data of the position image to be detected and the template image in each sub-area.

In some embodiments, corresponding subject matching weight data may be set for the subject region, and corresponding sub-region weight data may be set for each sub-region separately. And carrying out weighted summation calculation according to the main body matching data, the main body matching weight data, the matching data of each sub-region and the sub-region weight data of each sub-region, so as to obtain weighted matching data, and determining the detection result of the element to be detected according to the comparison result between the weighted matching data and the preset matching threshold.

Illustratively, taking the main body region and the 4 sub-regions shown in fig. 4b as an example, main body matching weight data corresponding to the main body region 5 may be set to α, and sub-region weight data corresponding to each of the 4 sub-regions may be set to β. Where α+4β=1. The main body matching data of the position image to be detected and the template image in the main body area 5 are denoted as D1, the matching data of the position image to be detected and the template image in the sub-area 1 are denoted as D21, the matching data in the sub-area 2 are denoted as D22, the matching data in the sub-area 3 are denoted as D23, and the matching data in the sub-area 4 are denoted as D24.

The weighted matching data of the main body matching data and the matching data of each sub-region may be calculated in the form of d1×α+d21×β+d22×β+d23×β+d24×β, or the weighted matching data may be calculated in the form of d1×α+ (d21+d22+d23+d24) ×β. The preset matching data threshold is marked as Th, the weighted matching data is compared with the preset matching data threshold Th, and if the weighted matching data is larger than the preset matching data threshold Th, the element to be detected can be considered to be qualified; if the weighted matching data is smaller than or equal to the preset matching data threshold Th, the element to be detected is considered to be unqualified.

Illustratively, α may range in value from 0.6 to 0.9. For example, α may be 0.6 or 0.7 or 0.8, etc.

It can be understood that the preset matching data threshold, the main body matching weight data of the main body area, and the sub-area weight data of each sub-area may be set according to an actual application scenario or an application requirement, which is not specifically limited in this specification.

In some embodiments, referring to fig. 5, determining the detection result of the element to be detected based on the subject matching data and the background matching data may include the following steps.

S510, determining a first comparison result between the main body matching data and a first preset data threshold value and a second comparison result between the background matching data and a second preset data threshold value.

And S520, if the first comparison result shows that the main body matching data is larger than a first preset data threshold value, and the second comparison result shows that the background matching data is larger than a second preset data threshold value, obtaining a qualified detection result of the element to be detected.

The first preset data threshold may be used to determine whether the main body matching data reaches a preset standard value, so as to determine whether the to-be-detected position image includes the to-be-detected element, and whether the main feature of the to-be-detected element and the main feature of the to-be-detected element in the template image reach sufficient similarity.

The second preset data threshold may be used to determine a degree of similarity of a background region in the position image to be detected and a background region in the template image.

Specifically, a matching score threshold of the body may be set for the main body area as a first preset data threshold, and a matching score threshold of the background may be set for the background area as a second preset data threshold, so as to implement fusion determination of the detection result of the element to be detected by adopting the matching score threshold of the body and the matching score threshold of the background. And comparing the main body matching data with a first preset data threshold value to obtain a first comparison result, and comparing the background matching data with a second preset data threshold value to obtain a second comparison result. If the first comparison result indicates that the main body matching data is greater than the first preset data threshold value, and the second comparison result indicates that the background matching data is greater than the second preset data threshold value, the similarity between the main body region in the position image to be detected and the main body region in the template image can be indicated to exceed a preset standard value, and the similarity between the background region in the position image to be detected and the background region in the template image exceeds the preset standard value. Therefore, the main body area in the position image to be detected can be further indicated to contain the element to be detected, the installation condition of the main body area is close to the standard installation condition of the element to be detected in the template image, the installation position of the element to be detected in the position image to be detected is correct, the conditions of dislocation, pollution, skew and the like do not occur, the element to be detected in the position image to be detected can be considered to meet the expectations, and accordingly the qualified detection result of the element to be detected is obtained.

If the first comparison result indicates that the main body matching data is smaller than or equal to a first preset data threshold value, or if the second comparison result indicates that the background matching data is smaller than or equal to a second preset data threshold value, the main body area in the position image to be detected does not contain elements to be detected (including the cases of wrong parts and missing parts), or the position image to be detected contains the elements to be detected but has serious deflection, and the like, the elements to be detected in the position image to be detected can be considered to be not in line with expectations, so that the unqualified detection result of the elements to be detected is obtained.

In some embodiments, the background area is divided into N sub-areas, where N has a value equal to or greater than 4. The background matching data may be obtained from matching data of the position image to be detected and the template image in each sub-area.

Illustratively, as shown in fig. 4b, the main body matching data of the position image to be detected and the template image in the main body area 5 is denoted as D1, the matching data of the position image to be detected and the template image in the sub-area 1 is denoted as D21, the matching data in the sub-area 2 is denoted as D22, the matching data in the sub-area 3 is denoted as D23, and the matching data in the sub-area 4 is denoted as D24. The background matching data may be obtained by summing the matching data D21, D22, D23, D24, or averaging, or weighted summing, etc. If the main body matching data D1 is larger than the first preset data threshold tau 1 and the background matching data is larger than the second preset data threshold tau 2, a qualified detection result of the element to be detected can be obtained.

In some embodiments, the background area is divided into N sub-areas, where the value of N is greater than or equal to 4; the background matching data comprises sub-region matching data of the position image to be detected and the template image in any sub-region; the second preset data threshold includes preset region data thresholds for each of the N sub-regions. If the first comparison result indicates that the main body matching data is greater than the first preset data threshold value, and the second comparison result indicates that the background matching data is greater than the second preset data threshold value, obtaining a qualified detection result of the element to be detected may include: if the first comparison result shows that the main body matching data is larger than a first preset data threshold value, and the second comparison result shows that the sub-region matching data of each sub-region is larger than a preset region data threshold value of each sub-region, a qualified detection result of the element to be detected is obtained.

The sub-region matching data can be used for representing the similarity or matching degree of the position image to be detected and the template image in the corresponding sub-region. The preset region data threshold value can be used for judging whether the matching degree of the position image to be detected and the template image in the corresponding sub-region reaches a preset standard or not.

Specifically, the background area is divided into at least 4 sub-areas, and for each sub-area, correlation matching can be performed on the position image to be detected and the template image in each sub-area to obtain sub-area matching data of each sub-area. For each sub-region, a respective preset region data threshold for each sub-region may be preset. The main body matching data is compared with a first preset data threshold value to obtain a first comparison result, and the sub-region matching data of each sub-region is compared with a corresponding preset region data threshold value of each sub-region to obtain a second comparison result.

If the main body matching data is determined to be greater than the first preset data threshold according to the first comparison result, and the sub-region matching data of each sub-region is determined to be greater than the corresponding preset region data threshold of each sub-region according to the second comparison result, the to-be-detected position image can be considered to contain the to-be-detected element which is qualified in detection, and the qualified detection result of the to-be-detected element is obtained.

Illustratively, as shown in fig. 4b, the main body matching data of the position image to be detected and the template image in the main body area 5 is denoted as D1, the sub-area matching data of the position image to be detected and the template image in the sub-area 1 is denoted as D21, the sub-area matching data in the sub-area 2 is denoted as D22, the sub-area matching data in the sub-area 3 is denoted as D23, and the sub-area matching data in the sub-area 4 is denoted as D24. Assuming that the first preset data threshold of the main body area 5 is τ1, the preset area data threshold of the sub-area 1 is τ21, the preset area data threshold of the sub-area 2 is τ22, the preset area data threshold of the sub-area 3 is τ23, and the preset area data threshold of the sub-area 4 is τ24. If the main body matching data is larger than the first preset data threshold τ1, the sub-region matching data D21 is larger than the preset region data threshold τ21, the sub-region matching data D22 is larger than the preset region data threshold τ22, the sub-region matching data D23 is larger than the preset region data threshold τ23, and the sub-region matching data D24 is larger than the preset region data threshold τ24, a qualified detection result of the element to be detected can be obtained.

If the main body matching data is smaller than or equal to a first preset data threshold tau 1 or any sub-region matching data is smaller than or equal to a preset region data threshold of the corresponding sub-region, a detection result of unqualified elements to be detected can be obtained.

By way of example, with continued reference to fig. 4b, the same preset region data threshold may be set for sub-region 1 and sub-region 4, and for sub-region 2 and sub-region 3.

Illustratively, with continued reference to fig. 4d, the same preset region data thresholds may be set for sub-region 1, sub-region 3, sub-region 7, and sub-region 9, for sub-region 2 and sub-region 8, and for sub-region 4 and sub-region 6.

It should be noted that, the preset area data threshold of each of the N sub-areas may be set according to an actual application scenario or an application requirement, which is not specifically limited in the present specification.

In some embodiments, the first preset data threshold is derived from the subject matching reference data of the detected element, and the second preset data threshold is derived from the background matching reference data of the detected element; the detected elements include elements whose detection results are acceptable and/or elements whose detection results are unacceptable.

The element with the qualified detection result can be used as a normal sample, and the element with the unqualified detection result can be used as an abnormal sample.

Specifically, the subject matching data of detected elements whose detection results are acceptable and/or detected elements whose detection results are unacceptable may be acquired as the subject matching reference data, and the background matching data of these detected elements may be acquired as the background matching reference data. And analyzing the overall distribution condition of the main body matching reference data and the background matching reference data to obtain a first preset data threshold value and a second preset data threshold value.

For example, the subject matching data of the detected element whose detection result is qualified may be taken as the subject matching reference data, and the background matching data of the detected element whose detection result is qualified may be taken as the background matching reference data. The first preset data threshold value can be obtained by analyzing any one of the average value, the median, the standard deviation, the specific percentile, the minimum value and the like of the main body matching reference data; the second preset data threshold may be obtained by analyzing any one of an average value, a median value, a specific percentile, a standard deviation, a minimum value, and the like of the background matching reference data.

For example, the subject matching data of the detected element whose detection result is failed may be taken as the subject matching reference data, and the background matching data of the detected element whose detection result is failed may be taken as the background matching reference data. The first preset data threshold value can be obtained by analyzing any one of the average value, the median, the specific percentile, the standard deviation, the maximum value and the like of the main body matching reference data; the second preset data threshold may be obtained by analyzing any one of the average, median, specific percentile, standard deviation, maximum, etc. of the background matching reference data.

For example, the subject matching data of the detected element whose detection result is acceptable and the detected element whose detection result is unacceptable may be used as the subject matching reference data, and the background matching data of the detected element whose detection result is acceptable and the detected element whose detection result is unacceptable may be used as the background matching reference data. The first preset data threshold value can be obtained by analyzing any one of the average value, the median, the standard deviation, the variance and the like of the main body matching reference data; the second preset data threshold may be obtained by analyzing any one of the average, median, standard deviation, variance, etc. of the background matching reference data.

Illustratively, the background region is divided into not less than 4 sub-regions. And assuming background matching data of the detected element with qualified detection result as background matching reference data, wherein the background matching reference data comprises sub-region reference data of each sub-region. For each sub-region, by analyzing any one of an average value, a median, a specific percentile, a minimum value, and the like of sub-region reference data of each sub-region, a preset region data threshold of each sub-region can be obtained, and the second preset data threshold includes a preset region data threshold of each sub-region.

It should be noted that, the manner of obtaining the first preset data threshold according to the overall distribution condition of the main body matching reference data and obtaining the second preset data threshold according to the overall distribution condition of the background matching reference data may be determined according to an actual application scenario or an application requirement, and the present disclosure is not limited specifically.

In some embodiments, referring to fig. 6a, determining the detection result of the element to be detected based on the subject matching data and the background matching data may further include the following steps.

And S610, performing difference calculation according to the main body matching data and the preliminary matching data corresponding to the position image to be detected to obtain first difference data.

And S620, performing difference calculation according to the background matching data and the preliminary matching data corresponding to the position image to be detected, and obtaining second difference data.

S630, if the first difference data is larger than a third preset data threshold value and the second difference data is larger than a fourth preset data threshold value, a qualified detection result of the element to be detected is obtained.

The first difference data may be used to indicate a variation of the subject matching data with respect to the preliminary matching data corresponding to the position image to be detected, and the second difference data may be used to indicate a variation of the background matching data with respect to the preliminary matching data corresponding to the position image to be detected.

Specifically, the comparison analysis and the difference calculation are performed on the main body matching data and the preliminary matching data of the position image to be detected, so that the difference degree between the main body matching data and the preliminary matching data of the position image to be detected can be obtained, and the first difference data is obtained. And similarly, performing contrast analysis and difference calculation on the background matching data and the preliminary matching data corresponding to the position image to be detected, so as to obtain second difference data. A corresponding third preset data threshold is preset for the main body area, a corresponding fourth data threshold is preset for the background area, the first difference data is compared with the third preset data threshold, and the second difference data is compared with the fourth preset data threshold.

If the first difference data is larger than the third preset data threshold value and the second difference data is larger than the fourth preset data threshold value, the fact that in the fine matching result, the matching degree of the position image to be detected and the template image in the main area is higher than the overall matching degree indicated by the rough matching result (namely, the preliminary matching data of the position image to be detected), a certain threshold value is reached, and meanwhile, the matching degree of the position image to be detected and the template image in the background area is higher than the overall matching degree indicated by the rough matching result, and a certain threshold value is reached. Therefore, the to-be-detected element is considered to be included in the to-be-detected position image, and the to-be-detected element in the to-be-detected position image shows enough accuracy and consistency with the standard state or the reference state (namely the template image) of the to-be-detected element on the main body characteristic and the background environment, so that the qualified detection result of the to-be-detected element can be obtained.

In some embodiments, the first difference data may be obtained by calculating a difference between the subject matching data and the preliminary matching data corresponding to the position image to be detected, and the second difference data may be obtained by calculating a difference between the background matching data and the preliminary matching data corresponding to the position image to be detected.

In other embodiments, the first difference data may also be obtained by calculating a ratio between the main body matching data and the preliminary matching data corresponding to the position image to be detected, and the second difference data may also be obtained by calculating a ratio between the background matching data and the preliminary matching data corresponding to the position image to be detected.

Further, the background area is divided into not less than 4 sub-areas, sub-area difference data between sub-area matching data of each sub-area and preliminary matching data of the position image to be detected can be calculated for each sub-area, and the second difference data includes sub-area difference data of each of the plurality of sub-areas. For each sub-region, a corresponding sub-region difference threshold may also be set, and the fourth preset data threshold includes a sub-region difference threshold of each sub-region. If the first difference data is larger than the third preset data threshold value and the sub-region difference data of each sub-region is larger than the corresponding sub-region difference threshold value of each sub-region, a qualified detection result of the element to be detected can be obtained.

It should be noted that, the third preset data threshold may be obtained according to the main body matching reference data of the detected element, and the fourth preset data threshold may be obtained according to the background matching reference data of the detected element, which is not described in detail. The third preset data threshold and the fourth preset data threshold may also be set according to an actual application scenario or an application requirement, and the present disclosure is not limited specifically.

In this specification, the preliminary matching data of the position image to be detected may also be referred to as a rough matching score.

Illustratively, referring to fig. 6b, the main flow of the circuit board element detection method may include:

(1) Rough matching: and performing rough matching detection on the whole template image and the initial image to be detected by adopting a mode of combining original image matching (namely pixel correlation matching based on pixel values of the original image) and edge detection post-matching. Carrying out standard correlation matching according to the whole template image and pixel data of the window image to be detected to obtain matching scores of the whole positions; and simultaneously, respectively carrying out Sobel edge detection on the whole template image and the window image to be detected, and carrying out standard correlation matching on the edge data of the whole template image and the window image to be detected obtained after edge detection to obtain an edge position matching score after edge detection. And combining the integral position matching score and the edge position matching score to obtain a rough matching score of the window image to be detected, wherein the integral matching score accounts for a ratio lambda 1, and the edge matching score accounts for a ratio lambda 2. And determining the optimal matching position according to the rough matching score of each window image to be detected, and obtaining a position image to be detected.

(2) Fine matching: and adopting a matching detection mode of fusion of body matching (namely, main body area matching) and background matching (namely, background area matching). The overall template map is divided into a template body region (i.e., a main body region) and a template background region (i.e., a background region), wherein the background region part obtains different background subregions according to different splitting modes. Taking the division structure shown in fig. 4b as an example, the background area portion is divided into 4 sub-areas. And dividing the position image to be detected obtained by rough matching into a body region and a background region comprising 4 sub-regions in the same mode according to the structural distribution of the whole template diagram. Aiming at the body area, carrying out body matching in a mode of combining pixel correlation matching which is the same as rough matching and edge detection post-matching to obtain a body matching score; and carrying out background matching on each sub-region of the background region by adopting a mode of combining pixel correlation matching and edge detection post-matching which are the same as rough matching, so as to obtain the sub-region matching score of each sub-region (namely sub-region matching data of each sub-region) to obtain the background matching score. And respectively calculating the variation of the body matching score relative to the rough matching score and the variation of the sub-region matching score of each sub-region relative to the rough matching score so as to judge by utilizing the matching score of the relative variation.

(3) And (3) result judgment: and judging by combining the body matching score variation and the background matching score variation. If the change amount of the body matching score is greater than the threshold value τ1 of the body region, and the change amount of the sub-region matching score of the sub-region 1 in the background matching score is greater than the threshold value τ2 of the sub-region 1, the change amount of the sub-region matching score of the sub-region 2 is greater than the threshold value τ3 of the sub-region 2, the change amount of the sub-region matching score of the sub-region 3 is greater than the threshold value τ4 of the sub-region 3, and the change amount of the sub-region matching score of the sub-region 4 is greater than the threshold value τ5 of the sub-region 4, the element to be detected is detected to be qualified.

Therefore, the recognition algorithm of component error, leakage and deflection based on the template matching principle is realized by utilizing the modes of edge detection, component body information matching and component surrounding information matching fusion, the rapid detection of components can be realized under the condition that only one positive sample is needed, and higher detection precision, higher detection stability and better robustness are realized.

In the above embodiment, the degree of matching between the position image to be detected and the template image in the main area portion is quantized more carefully according to the difference data between the main matching data and the preliminary matching data of the position image to be detected, and the degree of matching between the position image to be detected and the template image in the background area portion is quantized more carefully according to the difference data between the background matching data and the preliminary matching data, so that the degree of matching between the position image to be detected and the template image can be evaluated more comprehensively and more accurately, thereby improving the accuracy and reliability of detection.

The embodiment of the present disclosure provides a circuit board element detection device, referring to fig. 7, a circuit board element detection device 700 may include: an image acquisition module 710, a matching data determination module 720, and a detection result determination module 730.

An image acquisition module 710, configured to acquire a position image to be detected and a template image of an element to be detected; the to-be-detected position image is an area image of which preliminary matching data between the to-be-detected initial image and the template image meets a preset matching condition, and the to-be-detected position image and the template image are respectively divided into a main area and a background area according to a preset division mode.

The matching data determining module 720 is configured to determine main matching data of the position image to be detected and the template image in the main area and background matching data in the background area.

The detection result determining module 730 is configured to determine a detection result of the element to be detected based on the subject matching data and the background matching data.

The specific limitation of the circuit board element detection device can be referred to the limitation of the circuit board element detection method hereinabove, and the description thereof will not be repeated here. The respective modules in the above-described circuit board element inspection apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

The present embodiment further provides a computer device, referring to fig. 8, where the computer device 800 may include a memory 810, a processor 820, and a first computer program 830 stored in the memory 810 and capable of running on the processor 820, and when the processor 820 executes the first computer program 830, the method for detecting a circuit board element in any one of the foregoing embodiments is implemented.

The present embodiment further provides a chip, referring to fig. 9, where the chip 900 may include a storage unit 910, a processing unit 920, and a second computer program 930 stored on the storage unit 910 and capable of running on the processing unit 920, and when the processing unit 920 executes the second computer program 930, the method for detecting a circuit board element in any of the foregoing embodiments is implemented.

The present specification embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the circuit board element detection method in any of the foregoing embodiments.

It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.

In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.

In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly, through intermediaries, or both, may be in communication with each other or in interaction with each other, unless expressly defined otherwise. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.

While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.

Claims (15)

1. A method of inspecting a circuit board component, the method comprising:

acquiring a position image to be detected and a template image of an element to be detected; the to-be-detected position image is an area image, in which preliminary matching data between the to-be-detected initial image and the template image meet preset matching conditions, and the to-be-detected position image and the template image are respectively divided into a main area and a background area according to a preset dividing mode;

Determining main body matching data of the position image to be detected and the template image in the main body area and background matching data of the template image in the background area;

And determining a detection result of the element to be detected based on the main body matching data and the background matching data.

2. The method according to claim 1, characterized in that the position image to be detected is acquired by:

determining a window image to be detected in the initial image to be detected according to a preset sliding window; wherein the preset sliding window is determined according to the size of the template image;

performing template matching on the template image and the window image to be detected to obtain preliminary matching data between the window image to be detected and the template image;

And if the preliminary matching data corresponding to the window image to be detected meets the preset matching condition, determining the window image to be detected as the position image to be detected.

3. The method according to claim 2, wherein the performing template matching on the template image and the window image to be detected to obtain preliminary matching data between the window image to be detected and the template image includes:

Performing pixel correlation matching on the pixel data of the template image and the pixel data of the window image to be detected to obtain pixel matching data of the window image to be detected;

Performing correlation matching by using edge data obtained by edge detection in the template image and edge data obtained by edge detection of the window image to be detected to obtain edge matching data of the window image to be detected;

And determining preliminary matching data between the window image to be detected and the template image according to the pixel matching data and the edge matching data.

4. A method according to claim 3, wherein the pixel matching data corresponds to pixel matching weight data and the edge matching data corresponds to edge matching weight data; the determining preliminary matching data between the window image to be detected and the template image according to the pixel matching data and the edge matching data comprises the following steps:

And carrying out weighted summation calculation according to the pixel matching data, the pixel matching weight data, the edge matching data and the edge matching weight data to obtain preliminary matching data of the window image to be detected.

5. The method of claim 1, wherein the subject match data and the background match data are determined by:

Performing template matching in the main body area according to the position image to be detected and the template image to obtain main body matching data;

and carrying out template matching in the background area according to the position image to be detected and the template image to obtain the background matching data.

6. The method of claim 1, wherein the preset dividing manner is used for indicating that the background area is arranged around the main area, and the background area is divided into N sub-areas, and the value of N is greater than or equal to 4.

7. The method of claim 6, wherein if the value of N is equal to 4, the background region includes a first sub-region, a second sub-region, a third sub-region, and a fourth sub-region, the first sub-region and the second sub-region being located on left and right sides of the main region; the third sub-area and the fourth sub-area are positioned on the upper side and the lower side of the main body area; or alternatively

If the value of N is equal to two of 8,8 sub-areas located at the left and right sides of the main body area, two of the 8 sub-areas are located at the upper and lower sides of the main body area, and four of the 8 sub-areas are located at four vertexes of the main body area.

8. The method according to any one of claims 1 to 7, wherein the determining a detection result of the element to be detected based on the subject matching data and the background matching data includes:

Determining a first comparison result between the main body matching data and a first preset data threshold value and a second comparison result between the background matching data and a second preset data threshold value;

And if the first comparison result indicates that the main body matching data is larger than the first preset data threshold value, and the second comparison result indicates that the background matching data is larger than the second preset data threshold value, obtaining the detection result of the qualified element to be detected.

9. The method of claim 8, wherein the background region is divided into N sub-regions, and the value of N is greater than or equal to 4; the background matching data comprise sub-region matching data of the position image to be detected and the template image in any sub-region; the second preset data threshold value comprises the preset area data threshold value of each of the N sub-areas; if the first comparison result indicates that the main body matching data is greater than the first preset data threshold, and the second comparison result indicates that the background matching data is greater than the second preset data threshold, the detection result of the element to be detected is obtained, including:

And if the first comparison result indicates that the main body matching data is larger than the first preset data threshold value, and the second comparison result indicates that the sub-region matching data of each sub-region is larger than the preset region data threshold value of each sub-region, obtaining the detection result of the qualified element to be detected.

10. The method of claim 8, wherein the first predetermined data threshold is derived from body matched reference data of the detected element and the second predetermined data threshold is derived from background matched reference data of the detected element; the detected elements include elements whose detection results are acceptable, and/or elements whose detection results are unacceptable.

11. The method according to any one of claims 1 to 7, wherein the determining a detection result of the element to be detected based on the subject matching data and the background matching data, further comprises:

performing difference calculation according to the main body matching data and the preliminary matching data corresponding to the position image to be detected to obtain first difference data;

Performing difference calculation according to the background matching data and the preliminary matching data corresponding to the position image to be detected to obtain second difference data;

and if the first difference data is larger than a third preset data threshold value and the second difference data is larger than a fourth preset data threshold value, obtaining the detection result of the element to be detected.

12. A circuit board element inspection device, the device comprising:

The image acquisition module is used for acquiring a position image to be detected and a template image of the element to be detected; the to-be-detected position image is an area image, in which preliminary matching data between the to-be-detected initial image and the template image meet preset matching conditions, and the to-be-detected position image and the template image are respectively divided into a main area and a background area according to a preset dividing mode;

the matching data determining module is used for determining main body matching data of the position image to be detected and the template image in the main body area and background matching data of the background area;

And the detection result determining module is used for determining the detection result of the element to be detected based on the main body matching data and the background matching data.

13. A computer device comprising a memory and a processor, the memory storing a first computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 11 when the first computer program is executed.

14. A chip comprising a memory unit and a processing unit, the memory unit storing a second computer program, characterized in that the processing unit implements the steps of the method of any of claims 1 to 11 when the second computer program is executed.

15. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 11.