CN113298793A - Circuit board surface defect detection method based on multi-view template matching - Google Patents
Circuit board surface defect detection method based on multi-view template matching Download PDFInfo
- Publication number
- CN113298793A CN113298793A CN202110619139.4A CN202110619139A CN113298793A CN 113298793 A CN113298793 A CN 113298793A CN 202110619139 A CN202110619139 A CN 202110619139A CN 113298793 A CN113298793 A CN 113298793A
- Authority
- CN
- China
- Prior art keywords
- circuit board
- template
- defect
- matching
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 83
- 238000001514 detection method Methods 0.000 title abstract description 18
- 238000000034 method Methods 0.000 claims description 23
- 230000000007 visual effect Effects 0.000 claims description 23
- 239000002131 composite material Substances 0.000 claims 1
- 238000012360 testing method Methods 0.000 abstract description 3
- 230000000694 effects Effects 0.000 abstract description 2
- 238000005286 illumination Methods 0.000 abstract description 2
- 230000003313 weakening effect Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 238000001931 thermography Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000013095 identification testing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Abstract
The invention discloses a circuit board surface defect detection method based on multi-view template matching, which comprises the steps of collecting image information of a circuit board to be detected and a template circuit board under different views, dividing the image information into a plurality of sub-areas according to the same size and number for matching, fusing the defect index constructed by each matching result to obtain a comprehensive defect index, judging that the sub-area corresponding to the comprehensive defect index has defects if the image information is larger than a set threshold value, accumulating the defect number of the circuit board to be detected, effectively identifying the defects on the surface of the circuit board without a large number of data samples, weakening the influence of illumination conditions, assembly modes, test states, sample numbers and other factors on the detection effect, obviously improving the identification accuracy and effectively improving the defect detection efficiency of various types and small batches of circuit boards.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a matching detection technology.
Background
In the production and manufacturing process of the circuit board, various defects such as missing installation, wrong installation, damage and the like are easily generated on the surface of the circuit board, and the reliability of an electronic product is influenced.
In digital technology and application, circuit board component defect detection research discloses a circuit board defect detection method based on a template matching algorithm, wherein a device library is constructed according to an acquired device template, and a circuit board to be detected is matched with images in the device library. In order to achieve higher detection accuracy, the types and states of all components in the circuit board to be detected must be ensured to be completely consistent with those in the component library, and the method is difficult to be suitable for detecting defects of various circuit boards with different light environments and different component assembly modes.
In infrared technology, an infrared thermograph-based airborne circuit board fault mode diagnosis research discloses a method for identifying defects of components in a circuit board by an infrared thermal imaging method, wherein temperature characteristics of the components are extracted by a thermal imaging technology, mode classification is carried out based on a vector machine, and abnormal states of the components in the circuit board during working are identified. The method has applicability in the power-on state of the circuit board, and can not carry out early detection on defects in the production and debugging processes.
In electronic quality, the sub-pixel-based PCB surface quality detection discloses a circuit board defect detection method based on deep learning, which is used for collecting image information of a wire and a bonding pad in a circuit board light plate, training an artificial neural network model and identifying the type of defects. The method needs a large amount of data for supervision and learning, and has less effective quantity for circuit boards produced in various types and small batches or in special application environments, and effective models cannot be obtained due to the fact that some types of samples are missing.
And when the defect detection test is carried out on the circuit board with two surface defects, and the defect detection is carried out by adopting a template matching method based on a single visual angle, if the defect judgment threshold value is smaller, the number of generated false identifications is larger. The number of the generated false identifications is gradually reduced along with the increase of the defect judgment threshold, and when the judgment threshold is increased to a certain degree, the matching result is subjected to missing identification, and a small amount of false identifications still exist at the moment. It is shown that in the process of matching a single view angle template, due to the non-negligible error factor, the real defect area does not necessarily have the maximum defect index, and the real defect cannot be accurately identified only by increasing the judgment threshold.
Disclosure of Invention
The invention provides a method for detecting surface defects of a circuit board based on multi-view template matching, aiming at solving the problems in the prior art, and accurately detecting the defects of missing, misloading, damage and the like of surface elements of the circuit board.
The method comprises the steps of collecting image information of a circuit board to be detected and image information of a template circuit board under different visual angles, dividing the image of the circuit board to be detected and the image of the template circuit board into a plurality of sub-areas according to the same size and number, matching each sub-area of the circuit board to be detected and the corresponding sub-area of the template circuit board, adopting a template matching method under a single visual angle, constructing a corresponding defect index according to each matching result, fusing a plurality of defect indexes to obtain a comprehensive defect index, and judging that the image corresponding to the comprehensive defect index has defects if the defect index is larger than a set threshold value.
Selecting n different reference visual angles, wherein n is more than or equal to 2, and each visual angle is fixed with one camera
Collecting n images T of the template circuit board with different reference visual angles, and respectively recording the images T as T1，T2……Tn
Collecting n images S of the circuit board to be tested at different reference viewing angles, and respectively recording the images S as S1，S2……Sn
Adjusting the size of the image T of the template circuit board and the size of the image S of the circuit board to be tested to be the same, and respectively dividing each image T and S into a plurality of subregions Tij and Sij with the same size
Matching the gray value Tij (x, y) corresponding to the subregion Tij with the gray value Sij (x, y) corresponding to the subregion Sij, adopting a standardized correlation matching method, calculating a correlation coefficient R (i, j) between Tij and Sij by a formula I, and defining a defect index D (i, j) ═ 1-R (i, j)
Formula one
Setting Dk (x, y) as a defect index obtained by matching the kth reference view angle, adding the defect indexes D (i, j) corresponding to the subareas Sij of the n images S, and calculating a comprehensive defect index D by a formula IIt(i,j)
Formula two
If the index of comprehensive defects DtAnd (i, j) if the number of the sub-areas is larger than the set threshold value, judging the sub-areas to be defect areas, and accumulating the number of all the defect sub-areas to be used as the defect number of the circuit board to be detected.
The invention has the beneficial effects that: the method has the advantages that the template matching defect detection based on a plurality of different visual angles is realized, a large number of data samples are not needed, the defects on the surface of the circuit board are effectively identified, the influence of factors such as the illumination condition, the assembly mode, the test state and the number of samples on the detection effect of components is weakened, the identification accuracy is obviously improved, and the defect detection efficiency of various types and small batches of circuit boards is effectively improved. In a circuit board surface defect identification test containing two defects, the circuit board surface defects are respectively identified from four different single visual angles, the generated average misidentification number is 18, the templates are matched together from two different visual angles, the generated misidentification number is 1, and no misidentification is generated through the 4 different visual angles.
Drawings
Fig. 1 is a diagram of sub-region matching, and fig. 2 is a diagram of camera layout.
Detailed Description
The technical scheme of the invention is specifically explained in the following by combining the attached drawings.
Selecting 4 different reference viewing angles, wherein the included angle between adjacent viewing angles is 90 °, fixing a camera in each viewing angle, as shown in fig. 2, and adopting a CMOS image sensor, the resolution is 4032 pixel × 3024 pixel at the maximum.
2 circuit boards KS103 with the same model are selected as research objects, and each circuit board is 42mm long and 20mm wide. One piece is defect-free and serves as a template circuit board. The other piece comprises two surface defects, namely wrong mounting of an R10-bit device, packaging size of 1206, missing mounting of a C6-bit device and packaging size of 1206, and is used as a circuit board to be tested.
Respectively acquiring images T of the template circuit board by using cameras in 4 different reference visual angles, and respectively recording the images T as T1、T2、T3、T4And the image S of the circuit board to be tested are respectively marked as S1、S2、S3、S4。
Adjusting the template image T and the image S to be matched to make the sizes of the template image T and the image S to be matched be 1200 pixels by 600 pixels, and respectively dividing the template image T and the image S to be matched into a plurality of sub-areas T with the same size of 40 pixels by 40 pixelsij、Sij。
Each sub-region T in the template image TijCorresponding gray value Tij(x, y) and the sub-region S of the corresponding position in the image S to be matchedijCorresponding gray value Sij(x, y) matching, as shown in FIG. 1, by the normalized correlation matching method, T is obtainedijAnd SijThe correlation coefficient R (i, j) between the two points, and defining defect index D (i, j) as 1-R (i, j);
the template matching result under the single visual angle can be used for obtaining that when the surface of the circuit board has defects of device misloading, device missing and the like, the defect indexes of corresponding areas are not less than 0.15, in order to consider the accuracy and the integrity of identification, the defect threshold value of damage identification under the single visual angle is set to be 0.15, namely when the defect index of a certain area is more than 0.15, the area can be judged to be a defect area.
At this time, there are many false identifications in the defect detection result at a single viewing angle, which are 15 and 22 positions respectively.
If the defect indexes of each sub-area Sij under 2 different reference visual angles are added, a comprehensive defect index D is obtainedt(i, j), the defect judgment threshold is set to 0.3, the number of false identifications is obviously reduced, and only 1 false identification exists.
If the defect indexes of each sub-area Sij under 4 different reference visual angles are added, a comprehensive defect index D is obtainedt(i, j), the defect judgment threshold is set to be 0.6, no error identification is generated in the matching result, and the real defect can be accurately identified.
When the surface defect identification of the circuit board is carried out by four different single visual angles respectively, the average misidentification number is 18, when the template matching is carried out by 2 different visual angles, the misidentification number is only 1, and when the template matching is carried out by 4 different visual angles, no misidentification is generated.
The above-described embodiments are not intended to limit the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention are included in the scope of the present invention.
Claims (5)
1. A method for detecting surface defects of a circuit board based on multi-view template matching is characterized by comprising the following steps: the method comprises the steps of collecting image information of a circuit board to be detected and image information of a template circuit board under a plurality of different visual angles, dividing the image of the circuit board to be detected and the image of the template circuit board into a plurality of sub-areas according to the same size and quantity, matching each sub-area of the circuit board to be detected and the corresponding sub-area of the template circuit board, adopting a template matching method under a single visual angle, constructing corresponding defect indexes according to each matching result, fusing each defect index to obtain a comprehensive defect index, judging that the sub-area corresponding to the comprehensive defect index has defects if the defect index is larger than a set threshold value, and accumulating the quantity of all the sub-areas with the defects to obtain the defect quantity of the circuit board to be detected.
2. The multi-based of claim 1The method for detecting the surface defects of the circuit board matched with the view angle template is characterized in that the method for collecting the image information of the circuit board to be detected and the image information of the template circuit board under a plurality of different view angles comprises the following steps: selecting n different reference visual angles, wherein n is more than or equal to 2, and each visual angle is fixed with one camera; collecting n images T of the template circuit board with different reference visual angles, and respectively recording the images T as T1，T2……Tn(ii) a Collecting n images S of the circuit board to be tested at different reference viewing angles, and respectively recording the images S as S1，S2……Sn。
3. The method for detecting the surface defects of the circuit board based on the multi-view template matching according to claim 2, wherein the step of dividing the image of the circuit board to be detected and the image of the template circuit board into a plurality of sub-regions according to the same size and number comprises the following steps: and adjusting the size of the image T of the template circuit board and the size of the image S of the circuit board to be detected to be the same, and dividing each image T and S into a plurality of subregions Tij and Sij with the same size respectively.
4. The method for detecting surface defects of circuit boards based on multi-view template matching according to claim 3, wherein the matching of each sub-region of the circuit board to be detected with a corresponding sub-region of the template circuit board comprises: matching the gray value Tij (x, y) corresponding to the subregion Tij with the gray value Sij (x, y) corresponding to the subregion Sij, adopting a standardized correlation matching method and using a formula I
And calculating a correlation coefficient R (i, j) between Tij and Sij, and defining a defect index D (i, j) as 1-R (i, j).
5. The method for detecting the surface defects of the circuit board based on the multi-view template matching according to claim 4, wherein the fusing each defect index to obtain a comprehensive defect index comprises: let Dk(x, y) is a defect index obtained by matching the kth reference view angle, and the subareas Sij of the n images S are pairedAdding the corresponding defect indexes D (i, j) according to the formula two
Calculating the composite defect index Dt(i,j)。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110619139.4A CN113298793B (en) | 2021-06-03 | 2021-06-03 | Circuit board surface defect detection method based on multi-view template matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110619139.4A CN113298793B (en) | 2021-06-03 | 2021-06-03 | Circuit board surface defect detection method based on multi-view template matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298793A true CN113298793A (en) | 2021-08-24 |
| CN113298793B CN113298793B (en) | 2023-11-24 |
Family
ID=77326858
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110619139.4A Active CN113298793B (en) | 2021-06-03 | 2021-06-03 | Circuit board surface defect detection method based on multi-view template matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298793B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4809308A (en) * | 1986-02-20 | 1989-02-28 | Irt Corporation | Method and apparatus for performing automated circuit board solder quality inspections |
| CN1431482A (en) * | 2002-01-10 | 2003-07-23 | 欧姆龙株式会社 | Surface state checking method and circuit board checker |
| JP2006140279A (en) * | 2004-11-11 | 2006-06-01 | Omron Corp | Solder inspection method and substrate testing apparatus using the same |
| CN101251658A (en) * | 2008-03-12 | 2008-08-27 | 友达光电股份有限公司 | Display quality testing apparatus and testing method |
| TW201035537A (en) * | 2009-03-31 | 2010-10-01 | Lg Display Co Ltd | System and method for testing liquid crystal display device |
| CN104954783A (en) * | 2015-06-11 | 2015-09-30 | 东莞市升宏智能科技有限公司 | Optical system of camera module detection equipment |
| CN105957059A (en) * | 2016-04-20 | 2016-09-21 | 广州视源电子科技股份有限公司 | Electronic component missing detection method and system |
| CN106092970A (en) * | 2016-06-07 | 2016-11-09 | 京东方科技集团股份有限公司 | A kind of Systems for optical inspection and optical detection apparatus |
| CN108982512A (en) * | 2018-06-28 | 2018-12-11 | 芜湖新尚捷智能信息科技有限公司 | A kind of circuit board detecting system and method based on machine vision |
| CN109100370A (en) * | 2018-06-26 | 2018-12-28 | 武汉科技大学 | A kind of pcb board defect inspection method based on sciagraphy and connected domain analysis |
| CN109118482A (en) * | 2018-08-07 | 2019-01-01 | 腾讯科技（深圳）有限公司 | A kind of panel defect analysis method, device and storage medium |
| CN109461149A (en) * | 2018-10-31 | 2019-03-12 | 泰州市创新电子有限公司 | The intelligent checking system and method for lacquered surface defect |
| CN109785294A (en) * | 2018-12-24 | 2019-05-21 | 苏州江奥光电科技有限公司 | A kind of pcb board defective locations detection system and method |
| CN111814850A (en) * | 2020-06-22 | 2020-10-23 | 浙江大华技术股份有限公司 | Defect detection model training method, defect detection method and related device |
| KR20210016247A (en) * | 2019-07-31 | 2021-02-15 | 삼성디스플레이 주식회사 | Spot detecting apparatus, method of detecting spot, and display device |
-
2021
- 2021-06-03 CN CN202110619139.4A patent/CN113298793B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4809308A (en) * | 1986-02-20 | 1989-02-28 | Irt Corporation | Method and apparatus for performing automated circuit board solder quality inspections |
| CN1431482A (en) * | 2002-01-10 | 2003-07-23 | 欧姆龙株式会社 | Surface state checking method and circuit board checker |
| JP2006140279A (en) * | 2004-11-11 | 2006-06-01 | Omron Corp | Solder inspection method and substrate testing apparatus using the same |
| CN101251658A (en) * | 2008-03-12 | 2008-08-27 | 友达光电股份有限公司 | Display quality testing apparatus and testing method |
| TW201035537A (en) * | 2009-03-31 | 2010-10-01 | Lg Display Co Ltd | System and method for testing liquid crystal display device |
| CN104954783A (en) * | 2015-06-11 | 2015-09-30 | 东莞市升宏智能科技有限公司 | Optical system of camera module detection equipment |
| CN105957059A (en) * | 2016-04-20 | 2016-09-21 | 广州视源电子科技股份有限公司 | Electronic component missing detection method and system |
| CN106092970A (en) * | 2016-06-07 | 2016-11-09 | 京东方科技集团股份有限公司 | A kind of Systems for optical inspection and optical detection apparatus |
| CN109100370A (en) * | 2018-06-26 | 2018-12-28 | 武汉科技大学 | A kind of pcb board defect inspection method based on sciagraphy and connected domain analysis |
| CN108982512A (en) * | 2018-06-28 | 2018-12-11 | 芜湖新尚捷智能信息科技有限公司 | A kind of circuit board detecting system and method based on machine vision |
| CN109118482A (en) * | 2018-08-07 | 2019-01-01 | 腾讯科技（深圳）有限公司 | A kind of panel defect analysis method, device and storage medium |
| CN109461149A (en) * | 2018-10-31 | 2019-03-12 | 泰州市创新电子有限公司 | The intelligent checking system and method for lacquered surface defect |
| CN109785294A (en) * | 2018-12-24 | 2019-05-21 | 苏州江奥光电科技有限公司 | A kind of pcb board defective locations detection system and method |
| KR20210016247A (en) * | 2019-07-31 | 2021-02-15 | 삼성디스플레이 주식회사 | Spot detecting apparatus, method of detecting spot, and display device |
| CN111814850A (en) * | 2020-06-22 | 2020-10-23 | 浙江大华技术股份有限公司 | Defect detection model training method, defect detection method and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298793B (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6922168B2 (en) | Surface mount line quality control system and its control method | |
| CN112053318B (en) | Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning | |
| KR102022496B1 (en) | Process management and monitoring system using vision image detection and a method thereof | |
| US11132786B2 (en) | Board defect filtering method based on defect list and circuit layout image and device thereof and computer-readable recording medium | |
| CN111656883B (en) | Learning completion model generation system and method for component image recognition | |
| JP2004502250A (en) | Image processing system for use with an inspection system | |
| CN103091331A (en) | System and method for visual inspection on burrs and stain defects of radio frequency identification (RFID) antennae | |
| KR20190122550A (en) | Inspection management system, inspection management apparatus and inspection management method | |
| JP6645007B2 (en) | Board identification apparatus, board identification method, and board traceability system | |
| JP5168215B2 (en) | Appearance inspection device | |
| Li et al. | Automatic industry PCB board DIP process defect detection with deep ensemble method | |
| Liao et al. | A flexible PCB inspection system based on statistical learning | |
| CN108445010A (en) | Automatic optical detection method and device | |
| Hezaveh et al. | Roof damage assessment using deep learning | |
| WO2019146097A1 (en) | Sensing device and sensing system for defective photographic data | |
| Sarawade et al. | Detection of faulty integrated circuits in PCB with thermal image processing | |
| Iwahori et al. | Defect classification of electronic board using dense SIFT and CNN | |
| CN113298793A (en) | Circuit board surface defect detection method based on multi-view template matching | |
| CN115947066B (en) | Belt tearing detection method, device and system | |
| CN114226262A (en) | Flaw detection method, flaw classification method and flaw detection system | |
| Mogharrebi et al. | Missing component detection on PCB using neural networks | |
| Takacs et al. | Novel outlier filtering method for AOI image databases | |
| Zhang et al. | A Cascaded Zoom-In Method for Defect Detection of Solder Joints | |
| Kleber et al. | Optical sensor evaluation for vision based recognition of electronics waste | |
| Kumar et al. | Automated quality inspection of PCB assembly using image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |