CN113298793B - Circuit board surface defect detection method based on multi-view template matching - Google Patents
Circuit board surface defect detection method based on multi-view template matching Download PDFInfo
- Publication number
- CN113298793B CN113298793B CN202110619139.4A CN202110619139A CN113298793B CN 113298793 B CN113298793 B CN 113298793B CN 202110619139 A CN202110619139 A CN 202110619139A CN 113298793 B CN113298793 B CN 113298793B
- Authority
- CN
- China
- Prior art keywords
- circuit board
- defect
- sub
- template
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 77
- 238000001514 detection method Methods 0.000 title abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000000007 visual effect Effects 0.000 claims description 18
- 230000000694 effects Effects 0.000 abstract description 2
- 238000009825 accumulation Methods 0.000 abstract 1
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 238000001931 thermography Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method for detecting surface defects of a circuit board based on multi-view template matching, which is characterized in that image information of the circuit board to be detected and the template circuit board under different view angles is collected and is divided into a plurality of sub-areas according to the same size and number to be matched, defect indexes constructed by each matching result are fused to obtain comprehensive defect indexes, if the defect indexes are larger than a set threshold value, the defect of the sub-area corresponding to the comprehensive defect indexes is judged to exist, the defect number of the circuit board to be detected is obtained through accumulation, a large number of data samples are not needed, the defects of the surface of the circuit board are effectively identified, the influence of factors such as lighting conditions, assembly modes, test states, sample number and the like on the detection effect is weakened, the identification accuracy is remarkably improved, and the defect detection efficiency of various and small-batch circuit boards is effectively improved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a matching detection technology.
Background
In the production and manufacturing process of the circuit board, the tiny components on the surface of the circuit board are easy to generate defects of various types such as missing, wrong installation, damage and the like, and the reliability of the electronic product is affected.
The invention discloses a circuit board defect detection method based on a template matching algorithm in digital technology and application, wherein a device library is constructed according to an acquired device template, and a circuit board to be detected is matched with images in the device library. In order to achieve higher detection accuracy, the types and states of all components in the circuit board to be detected are required to be completely consistent with those in a device library, and the method is difficult to be applied to defect detection of circuit boards with different light environments, component assembly modes and various types.
The invention discloses a method for identifying defects of components in a circuit board by an infrared thermal imaging method, which is used for extracting temperature characteristics of the components by the thermal imaging technology, carrying out pattern classification based on a vector machine and identifying abnormal states of the components in the circuit board when working. The method has applicability in the electrified state of the circuit board, and defects cannot be detected early in the production and debugging processes.
The electronic quality detection of PCB surface quality based on sub-pixels discloses a circuit board defect detection method based on deep learning, which collects image information of wires and bonding pads in a circuit board optical plate, trains an artificial neural network model and identifies the type of defects. The method requires a large amount of data supervision learning, has a small effective number for various types of circuit boards, small-batch production or special application environments, has a defect of some types of samples, and cannot obtain an effective model.
And when a single-view-angle-based template matching method is adopted to detect the defects, if the defect judgment threshold is smaller, the number of generated misidentifications is larger. With the increase of the defect judgment threshold, the generated false recognition number gradually decreases, and when the judgment threshold is increased to a certain degree, the matching result is subjected to missing recognition, and a small amount of false recognition still exists at the moment. In the process of matching the single visual angle template, the regions of the real defects do not necessarily have the maximum defect indexes due to non-negligible error factors, and the real defects cannot be accurately identified only by increasing the judging threshold.
Disclosure of Invention
The invention provides a method for detecting surface defects of a circuit board based on multi-view template matching, which is used for accurately detecting defects such as missing, wrong installation, damage and the like of components on the surface of the circuit board, and the invention adopts the following technical scheme for realizing the purposes.
Collecting image information of a circuit board to be tested and image information of a template circuit board under different visual angles, dividing the image of the circuit board to be tested and the image of the template circuit board into a plurality of subregions according to the same size and number, matching each subregion of the circuit board to be tested with a subregion of the corresponding template circuit board, adopting a template matching method under a single visual angle, constructing corresponding defect indexes according to each matching result, fusing a plurality of defect indexes to obtain a comprehensive defect index, and judging that the image corresponding to the comprehensive defect index has defects if the image is larger than a set threshold.
Selecting n different reference viewing angles, wherein n is more than or equal to 2, and fixing a camera at each viewing angle
Collecting images T of template circuit boards with n different reference visual angles, which are respectively marked as T 1 ，T 2 ……T n
Collecting images S of n circuit boards to be tested with different reference visual angles, respectively recorded as S 1 ，S 2 ……S n
The image T of the template circuit board and the image S of the circuit board to be tested are adjusted to be the same in size, and each image T and S is divided into a plurality of sub-areas Tij and Sij with the same size
Matching the gray value Tij (x, y) corresponding to the sub-region Tij with the gray value Sij (x, y) corresponding to the sub-region Sij, calculating a correlation coefficient R (i, j) between Tij and Sij by adopting a standardized correlation matching method, and defining a defect index D (i, j) =1-R (i, j)
Equation one
Let Dk (x, y) be the defect index obtained by k-th reference view matching, add the defect indexes D (i, j) corresponding to sub-regions Sij of n images S, calculate the integrated defect index D from equation two t (i,j)
Formula II
If the integrated defect index D t And (i, j) is larger than a set threshold value, judging the subarea as a defect area, and accumulating the quantity of all defect subareas to be used as the defect quantity of the circuit board to be tested.
The invention has the beneficial effects that: based on the template matching defect detection of a plurality of different visual angles, a large number of data samples are not needed, defects on the surface of the circuit board are effectively identified, the influence of factors such as the light receiving condition, the assembly mode, the test state, the sample number and the like of components on the detection effect is weakened, the identification accuracy is obviously improved, and the defect detection efficiency of various and small-batch circuit boards is effectively improved. In the surface defect recognition test of the circuit board comprising two defects, the surface defects of the circuit board are respectively recognized from four different single visual angles, the average false recognition number is 18, the templates are matched together from two different visual angles, the false recognition number is 1, the templates are matched together from 4 different visual angles, and the false recognition is not generated.
Drawings
Fig. 1 is a schematic diagram of sub-region matching, and fig. 2 is a camera layout diagram.
Detailed Description
The technical scheme of the invention is specifically described below with reference to the accompanying drawings.
4 different reference viewing angles are selected, the included angle between the adjacent viewing angles is 90 degrees, and a camera is fixed in each viewing angle, as shown in fig. 2, a CMOS image sensor is adopted, and the resolution is 4032 pixels at maximum 3024 pixels.
2 circuit boards KS103 of the same type were selected as subjects, each 42mm long and 20mm wide. One piece was free of defects and served as a template circuit board. The other block contains two surface defects, namely, the R10 bit device is wrongly installed, the packaging size is 1206, the C6 bit device is wrongly installed, and the packaging size is 1206, so that the circuit board to be tested is obtained.
The molds were acquired separately with cameras in 4 different reference perspectivesThe images T of the board circuit board are respectively marked as T 1 、T 2 、T 3 、T 4 And the image S of the circuit board to be tested are respectively marked as S 1 、S 2 、S 3 、S 4 。
The template image T and the image S to be matched are adjusted to be 1200 pixels in size and are respectively divided into a plurality of sub-areas T with the same size of 40 pixels in size and 40 pixels in size ij 、S ij 。
Each sub-region T in the template image T ij Corresponding gray value T ij Sub-region S of (x, y) and corresponding position in the image S to be matched ij Corresponding gray value S ij (x, y) matching, as shown in FIG. 1, T is found by a normalized correlation matching method ij And S is equal to ij Correlation coefficient R (i, j) between, while defining defect index D (i, j) =1-R (i, j);
according to the template matching result under the single visual angle, when defects of the types of device misloading, device missing and the like exist on the surface of the circuit board, the defect index of the corresponding area is not less than 0.15, and in order to consider the accuracy and the completeness of identification, the defect threshold of damage identification under the single visual angle is set to be 0.15, namely when the defect index of a certain area is greater than 0.15, the area can be judged as a defect area.
At this time, there are many misidentifications in the defect detection results at a single viewing angle, namely 15 and 22.
If the defect indexes of each sub-region Sij under 2 different reference angles are added, the comprehensive defect index D is obtained t (i, j) the defect judgment threshold value is set to 0.3, the number of false identifications is significantly reduced, and only 1 false identification exists.
If the defect indexes of each sub-region Sij under 4 different reference viewing angles are added, the comprehensive defect index D is obtained t (i, j) setting the defect judgment threshold to 0.6, and accurately identifying the real defect without misidentification in the matching result.
When four different single visual angles are used for respectively carrying out surface defect recognition of the circuit board, the average false recognition number is 18, when 2 different visual angles are used for carrying out template matching together, the false recognition number is only 1, and when 4 different visual angles are used for carrying out template matching together, no false recognition is generated, so that the method can be effectively suitable for surface defect recognition of various circuit boards in small batches.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof, but rather as being included within the spirit and scope of the present invention.
Claims (1)
1. The method for detecting the surface defects of the circuit board based on the multi-view template matching is characterized by comprising the following steps of:
collecting image information of a circuit board to be tested and image information of a template circuit board under a plurality of different visual angles: selecting n different reference viewing angles, wherein n is more than or equal to 2, and each viewing angle is fixed with a camera; collecting images T of template circuit boards with n different reference visual angles, which are respectively marked as T 1 ，T 2 ……T n The method comprises the steps of carrying out a first treatment on the surface of the Collecting images S of n circuit boards to be tested with different reference visual angles, respectively recorded as S 1 ，S 2 ……S n ；
Dividing the image of the circuit board to be tested and the image of the template circuit board into a plurality of sub-areas according to the same size and number: the method comprises the steps of adjusting the sizes of an image T of a template circuit board and an image S of a circuit board to be tested to be the same, and dividing each image T and S into a plurality of sub-areas Tij and Sij with the same size;
matching each sub-region of the circuit board to be tested with the corresponding sub-region of the template circuit board: matching the gray value Tij (x, y) corresponding to the sub-region Tij with the gray value Sij (x, y) corresponding to the sub-region Sij, and adopting a standardized correlation matching method to obtain a first formulaCalculating a correlation coefficient R (i, j) between Tij and Sij, and defining a defect index D (i, j) =1-R (i, j);
adopting a template matching method under a single visual angle, and constructing a corresponding defect index according to each matching result;
fusing each defect index to obtain a comprehensive defect index: set D k (x, y) is the defect index obtained by the k-th reference view matching, the defect indexes D (i, j) corresponding to the sub-regions Sij of the n images S are added, and the result is represented by the formula II Calculating the comprehensive defect index D t (i,j)；
If the integrated defect index is larger than the set threshold value, judging that the sub-region corresponding to the integrated defect index has defects, and accumulating the number of all the sub-regions with defects to obtain the defect number of the circuit board to be tested.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110619139.4A CN113298793B (en) | 2021-06-03 | 2021-06-03 | Circuit board surface defect detection method based on multi-view template matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110619139.4A CN113298793B (en) | 2021-06-03 | 2021-06-03 | Circuit board surface defect detection method based on multi-view template matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298793A CN113298793A (en) | 2021-08-24 |
| CN113298793B true CN113298793B (en) | 2023-11-24 |
Family
ID=77326858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110619139.4A Active CN113298793B (en) | 2021-06-03 | 2021-06-03 | Circuit board surface defect detection method based on multi-view template matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298793B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4809308A (en) * | 1986-02-20 | 1989-02-28 | Irt Corporation | Method and apparatus for performing automated circuit board solder quality inspections |
| CN1431482A (en) * | 2002-01-10 | 2003-07-23 | 欧姆龙株式会社 | Surface state checking method and circuit board checker |
| JP2006140279A (en) * | 2004-11-11 | 2006-06-01 | Omron Corp | Solder inspection method and substrate testing apparatus using the same |
| CN101251658A (en) * | 2008-03-12 | 2008-08-27 | 友达光电股份有限公司 | Display quality testing apparatus and testing method |
| TW201035537A (en) * | 2009-03-31 | 2010-10-01 | Lg Display Co Ltd | System and method for testing liquid crystal display device |
| CN104954783A (en) * | 2015-06-11 | 2015-09-30 | 东莞市升宏智能科技有限公司 | Optical system of camera module detection equipment |
| CN105957059A (en) * | 2016-04-20 | 2016-09-21 | 广州视源电子科技股份有限公司 | Electronic component missing detection method and system |
| CN106092970A (en) * | 2016-06-07 | 2016-11-09 | 京东方科技集团股份有限公司 | A kind of Systems for optical inspection and optical detection apparatus |
| CN108982512A (en) * | 2018-06-28 | 2018-12-11 | 芜湖新尚捷智能信息科技有限公司 | A kind of circuit board detecting system and method based on machine vision |
| CN109100370A (en) * | 2018-06-26 | 2018-12-28 | 武汉科技大学 | A kind of pcb board defect inspection method based on sciagraphy and connected domain analysis |
| CN109118482A (en) * | 2018-08-07 | 2019-01-01 | 腾讯科技（深圳）有限公司 | A kind of panel defect analysis method, device and storage medium |
| CN109461149A (en) * | 2018-10-31 | 2019-03-12 | 泰州市创新电子有限公司 | The intelligent checking system and method for lacquered surface defect |
| CN109785294A (en) * | 2018-12-24 | 2019-05-21 | 苏州江奥光电科技有限公司 | A kind of pcb board defective locations detection system and method |
| CN111814850A (en) * | 2020-06-22 | 2020-10-23 | 浙江大华技术股份有限公司 | Defect detection model training method, defect detection method and related device |
| KR20210016247A (en) * | 2019-07-31 | 2021-02-15 | 삼성디스플레이 주식회사 | Spot detecting apparatus, method of detecting spot, and display device |
-
2021
- 2021-06-03 CN CN202110619139.4A patent/CN113298793B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4809308A (en) * | 1986-02-20 | 1989-02-28 | Irt Corporation | Method and apparatus for performing automated circuit board solder quality inspections |
| CN1431482A (en) * | 2002-01-10 | 2003-07-23 | 欧姆龙株式会社 | Surface state checking method and circuit board checker |
| JP2006140279A (en) * | 2004-11-11 | 2006-06-01 | Omron Corp | Solder inspection method and substrate testing apparatus using the same |
| CN101251658A (en) * | 2008-03-12 | 2008-08-27 | 友达光电股份有限公司 | Display quality testing apparatus and testing method |
| TW201035537A (en) * | 2009-03-31 | 2010-10-01 | Lg Display Co Ltd | System and method for testing liquid crystal display device |
| CN104954783A (en) * | 2015-06-11 | 2015-09-30 | 东莞市升宏智能科技有限公司 | Optical system of camera module detection equipment |
| CN105957059A (en) * | 2016-04-20 | 2016-09-21 | 广州视源电子科技股份有限公司 | Electronic component missing detection method and system |
| CN106092970A (en) * | 2016-06-07 | 2016-11-09 | 京东方科技集团股份有限公司 | A kind of Systems for optical inspection and optical detection apparatus |
| CN109100370A (en) * | 2018-06-26 | 2018-12-28 | 武汉科技大学 | A kind of pcb board defect inspection method based on sciagraphy and connected domain analysis |
| CN108982512A (en) * | 2018-06-28 | 2018-12-11 | 芜湖新尚捷智能信息科技有限公司 | A kind of circuit board detecting system and method based on machine vision |
| CN109118482A (en) * | 2018-08-07 | 2019-01-01 | 腾讯科技（深圳）有限公司 | A kind of panel defect analysis method, device and storage medium |
| CN109461149A (en) * | 2018-10-31 | 2019-03-12 | 泰州市创新电子有限公司 | The intelligent checking system and method for lacquered surface defect |
| CN109785294A (en) * | 2018-12-24 | 2019-05-21 | 苏州江奥光电科技有限公司 | A kind of pcb board defective locations detection system and method |
| KR20210016247A (en) * | 2019-07-31 | 2021-02-15 | 삼성디스플레이 주식회사 | Spot detecting apparatus, method of detecting spot, and display device |
| CN111814850A (en) * | 2020-06-22 | 2020-10-23 | 浙江大华技术股份有限公司 | Defect detection model training method, defect detection method and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298793A (en) | 2021-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tsai et al. | A shift-tolerant dissimilarity measure for surface defect detection | |
| Park et al. | Ambiguous surface defect image classification of AMOLED displays in smartphones | |
| Sun et al. | Automated thermal fuse inspection using machine vision and artificial neural networks | |
| Nayak et al. | Identification of PCB faults using image processing | |
| CN111656883B (en) | Learning completion model generation system and method for component image recognition | |
| TW202239281A (en) | Electronic substrate defect detection | |
| CN103091331A (en) | System and method for visual inspection on burrs and stain defects of radio frequency identification (RFID) antennae | |
| CN111566493B (en) | Method and system for circuit breaker condition monitoring | |
| CN107895362B (en) | Machine vision method for detecting quality of miniature wiring terminal | |
| CN109596530B (en) | Dark field polarization imaging method for classifying pockmarks and dust defects on optical surface | |
| JP6401648B2 (en) | Defect classification apparatus and defect classification method | |
| CN108311409A (en) | A kind of printed circuit board (PCB) detecting system based on artificial intelligence image recognition | |
| Liao et al. | A flexible PCB inspection system based on statistical learning | |
| JP6616906B1 (en) | Detection device and detection system for defective photographing data | |
| CN111739003B (en) | Machine vision method for appearance detection | |
| Mukhopadhyay et al. | PCB inspection in the context of smart manufacturing | |
| Iwahori et al. | Defect classification of electronic board using dense SIFT and CNN | |
| CN113298793B (en) | Circuit board surface defect detection method based on multi-view template matching | |
| KR101782364B1 (en) | Vision inspection method based on learning data | |
| CN115100497A (en) | Robot-based method, device, equipment and medium for routing inspection of abnormal objects in channel | |
| TWI698803B (en) | Image identifying method and related monitoring camera and monitoring camera system | |
| WO2023080587A1 (en) | Deep learning-based mlcc stacked alignment inspection system and method | |
| TW202034421A (en) | Color filter inspection device, inspection device, color filter inspection method, and inspection method | |
| Nakagawa et al. | Defect classification of electronic board using multiple classifiers and grid search of SVM parameters | |
| JP3892258B2 (en) | Automatic inspection device and automatic inspection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |