KR20240037494A - Printed circuit board defect detection method using semi-supervised learning - Google Patents
Printed circuit board defect detection method using semi-supervised learning Download PDFInfo
- Publication number
- KR20240037494A KR20240037494A KR1020220116065A KR20220116065A KR20240037494A KR 20240037494 A KR20240037494 A KR 20240037494A KR 1020220116065 A KR1020220116065 A KR 1020220116065A KR 20220116065 A KR20220116065 A KR 20220116065A KR 20240037494 A KR20240037494 A KR 20240037494A
- Authority
- KR
- South Korea
- Prior art keywords
- defect
- printed circuit
- circuit board
- detection method
- image
- Prior art date
Links
- 230000007547 defect Effects 0.000 title claims abstract description 49
- 238000001514 detection method Methods 0.000 title claims abstract description 12
- 238000013145 classification model Methods 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 3
- 238000013450 outlier detection Methods 0.000 claims description 3
- 229910000679 solder Inorganic materials 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000002950 deficient Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
- G01N2021/95638—Inspecting patterns on the surface of objects for PCB's
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Pathology (AREA)
- Data Mining & Analysis (AREA)
- Analytical Chemistry (AREA)
- Immunology (AREA)
- Chemical & Material Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Biochemistry (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
반지도 학습을 이용한 인쇄회로기판 결함 감지방법은 정상적인 인쇄회로기판의 이미지 형태가 학습된 오토 인코더(Auto Encoder)를 이용하여 인쇄회로기판의 이미지에서의 결함위치를 추론하는 단계와, 인쇄회로기판의 이미지에 “MLP Mixer”분류모델을 이용하여 결함의 이름을 추론하는 단계를 포함한다.The printed circuit board defect detection method using semi-supervised learning includes the steps of inferring the location of the defect in the image of the printed circuit board using an auto encoder that has learned the image shape of the normal printed circuit board, and It includes the step of inferring the name of the defect using the “MLP Mixer” classification model on the image.
Description
본 발명은 인쇄회로기판 결함 감지방법에 관한 것으로서, 더 상세하게는 반지도 학습을 이용한 인쇄회로기판 결함 감지방법에 관한 것이다.The present invention relates to a method for detecting printed circuit board defects, and more specifically, to a method for detecting printed circuit board defects using semi-supervised learning.
전자 장치에 장착되는 인쇄 회로 기판에 솔더 페이스트가 인쇄되는 공정은 스크린 프린터에 의해 수행된다.The process of printing solder paste on a printed circuit board mounted on an electronic device is performed by a screen printer.
스크린 프린터가 인쇄 회로 기판에 솔더 페이스트를 인쇄하는 공정은 다음과 같다. The process by which a screen printer prints solder paste on a printed circuit board is as follows.
스크린 프린터는 인쇄 회로 기판을 고정시키기 위한 테이블에 인쇄 회로 기판을 위치시키고, 스텐실의 개구가 대응되는 인쇄 회로 기판의 패드 상에 위치하도록 스텐실을 인쇄 회로 기판 상에 정렬시킨다. The screen printer places the printed circuit board on a table for holding the printed circuit board and aligns the stencil on the printed circuit board so that the openings of the stencil are located on corresponding pads of the printed circuit board.
다음으로 스크린 프린터는 스퀴지(squeegee)를 이용하여 스텐실의 개구를 통해 솔더 페이스트를 인쇄 회로 기판에 인쇄한다. Next, the screen printer uses a squeegee to print solder paste onto the printed circuit board through the openings in the stencil.
다음으로 스크린 프린터는 스텐실과 인쇄 회로 기판을 분리시킨다.Next, the screen printer separates the stencil and printed circuit board.
한편, 인쇄 회로 기판에 인쇄되는 솔더 페이스트의 형상은 SPI(solder paste inspection) 기술을 통해 검사될 수 있다. SPI 기술은, 광학 기술을 통해 인쇄 회로 기판에 인쇄된 솔더 페이스트의 2차원 또는 3차원 영상을 획득하고, 획득된 영상으로부터 인쇄 회로 기판에 인쇄되는 솔더 페이스트의 형상을 검사하는 기술이다.Meanwhile, the shape of the solder paste printed on the printed circuit board can be inspected through SPI (solder paste inspection) technology. SPI technology is a technology that acquires two-dimensional or three-dimensional images of solder paste printed on a printed circuit board through optical technology and inspects the shape of the solder paste printed on the printed circuit board from the acquired image.
본 발명은 상기와 같은 기술적 과제를 해결하기 위해 제안된 것으로, 2가지 추론 모델(inference model)을 사용하여 객체의 위치(Location)와 클래스(class)를 예측하며 지도학습과 비지도 학습의 조합으로 이루어지는 인쇄회로기판 결함 감지방법을 제공한다.The present invention was proposed to solve the technical challenges described above. It predicts the location and class of an object using two inference models and uses a combination of supervised learning and unsupervised learning. Provides a method for detecting printed circuit board defects.
상기 문제점을 해결하기 위한 본 발명의 일 실시예에 따르면, 인쇄회로기판의 이미지에서 이상치 감지모델을 이용하여 결함의 위치를 추론한 후, 분류 모델(classification model)을 사용하여 결함의 이름을 추론하는 것을 특징으로 하는 인쇄회로기판 결함 감지방법이 제공된다.According to an embodiment of the present invention to solve the above problem, the location of the defect is inferred using an outlier detection model in the image of the printed circuit board, and then the name of the defect is inferred using a classification model. A method for detecting defects in a printed circuit board is provided.
또한, 본 발명의 다른 실시예에 따르면, 정상적인 인쇄회로기판의 이미지 형태가 학습된 오토 인코더(Auto Encoder)를 이용하여 인쇄회로기판의 이미지에서의 결함위치를 추론하는 단계와, 인쇄회로기판의 이미지에 “MLP Mixer”분류모델을 이용하여 결함의 이름을 추론하는 단계를 포함하는 인쇄회로기판 결함 감지방법이 제공된다.In addition, according to another embodiment of the present invention, inferring the location of a defect in an image of a printed circuit board using an auto encoder in which the image shape of a normal printed circuit board has been learned; A printed circuit board defect detection method including the step of inferring the name of the defect using the “MLP Mixer” classification model is provided.
또한, 본 발명에서 결함의 이름을 추론하는 단계는, 결함이 발견된 위치 상하좌우의 일정 공간을 지정하여 결함 상자를 정의하는 단계와, 결함 상자를 4개 또는 9개의 패치(patch)로 나누어 “MLP Mixer”분류모델의 입력으로 사용하는 단계와, 채널 믹싱(channel mixing)과 토큰 믹싱(token mixing)의 특징 추출 기법을 사용하여, 획득된 특징들로 결함 상자가 어떤 종류의 결함인지 판단하는 단계를 포함하는 것을 특징으로 한다.In addition, in the present invention, the step of inferring the name of a defect includes defining a defect box by specifying a certain space above, below, and on the left and right of the location where the defect was found, and dividing the defect box into 4 or 9 patches. A step of using it as an input to the “MLP Mixer” classification model, and a step of determining what type of defect the defect box is using the obtained features using feature extraction techniques of channel mixing and token mixing. It is characterized by including.
본 발명의 반지도 학습을 이용한 인쇄회로기판 결함 감지방법은 2가지 추론 모델(inference model)을 사용하여 객체의 위치(Location)와 클래스(class)를 예측하며 지도학습과 비지도 학습의 조합으로 이루어진다.The printed circuit board defect detection method using semi-supervised learning of the present invention predicts the location and class of the object using two inference models and consists of a combination of supervised learning and unsupervised learning. .
도 1은 본 발명의 반지도 학습을 이용한 인쇄회로기판 결함 감지방법의 시스템 모델을 나타낸 도면
도 2는 본 발명의 반지도 학습을 이용한 인쇄회로기판 결함 감지방법의 순서도1 is a diagram showing a system model of the printed circuit board defect detection method using semi-supervised learning of the present invention.
Figure 2 is a flowchart of the printed circuit board defect detection method using semi-supervised learning of the present invention.
이하, 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자가 본 발명의 기술적 사상을 용이하게 실시할 수 있을 정도로 상세히 설명하기 위하여, 본 발명의 실시예를 첨부한 도면을 참조하여 설명하기로 한다.Hereinafter, in order to explain the present invention in detail so that a person skilled in the art can easily implement the technical idea of the present invention, embodiments of the present invention will be described with reference to the accompanying drawings.
본 발명에서는 반지도 학습(semi supervised learning)을 사용한 효과적인 인쇄회로기판(PCB) 결함 감지 기법을 제시한다. The present invention presents an effective printed circuit board (PCB) defect detection technique using semi-supervised learning.
제안된 기법은 크게 2가지 추론 모델(inference model)을 사용하여 객체의 위치(Location)와 클래스(class)를 예측하며 지도학습과 비지도 학습의 조합으로 이루어져 있다. The proposed technique largely uses two inference models to predict the location and class of an object and consists of a combination of supervised learning and unsupervised learning.
위치 추론(Location inference)의 경우 이상치 감지 모델을 사용하며, 클래스 추론(class inference)의 경우 분류 모델(classification model)을 사용하여, 결함의 위치와 이름을 효과적으로 추론한다.For location inference, an outlier detection model is used, and for class inference, a classification model is used to effectively infer the location and name of the defect.
도 1은 본 발명의 반지도 학습을 이용한 인쇄회로기판 결함 감지방법의 시스템 모델을 나타낸 도면이고, 도 2는 본 발명의 반지도 학습을 이용한 인쇄회로기판 결함 감지방법의 순서도이다.Figure 1 is a diagram showing a system model of the printed circuit board defect detection method using semi-supervised learning of the present invention, and Figure 2 is a flowchart of the printed circuit board defect detection method using semi-supervised learning of the present invention.
도 1 및 도 2를 참조하면, 본 발명은 도 1과 같이 2가지 네트워크를 사용하여 인쇄회로기판 결함의 위치(Location)와 클래스(class)를 추론하는 기법이다. Referring to Figures 1 and 2, the present invention is a technique for inferring the location and class of a printed circuit board defect using two networks as shown in Figure 1.
도 1-① 오토 인코더(Auto Encoder)는 이상치를 감지하는 모델이다. 이 모델은 비지도 학습을 사용하여 정상적인 인쇄회로기판 형태를 학습한다. Figure 1-① Auto Encoder is a model that detects outliers. This model uses unsupervised learning to learn the normal printed circuit board shape.
이후 추론 단계에서 결함이 존재하는 입력이 들어온다면, 결함 이미지를 결함이 없는 이미지로 복원하게 된다. If a defective input is received in the subsequent inference stage, the defective image is restored to a defect-free image.
이후 복원된 이미지와 결함이 존재하는 이미지의 뺄셈(Subtraction)을 통하여 결함의 위치를 찾아낸다. 이 단계에서 사용자는 인쇄회로기판의 결함 위치를 확인할 수 있지만, 그 결함이 어떤 결함인지는 알 수 없다. Afterwards, the location of the defect is found through subtraction between the restored image and the image where the defect exists. At this stage, the user can check the location of the defect on the printed circuit board, but cannot tell what kind of defect it is.
따라서 도 1의 시스템에서는 결함이 발견된 위치 상하좌우의 일정 공간을 지정하여 도 1-②와 같이 결함 상자를 정의하게 된다. 정의된 결함 상자는 분류기 학습에 사용된다.Therefore, in the system of Figure 1, a certain space above, below, left, and right of the location where the defect was found is designated to define a defect box as shown in Figure 1-②. The defined fault box is used to train the classifier.
분류기는 도 1-③과 같은 “MLP Mixer” 를 사용하였으며, 결함 상자를 4개 혹은 9개의 패치(patch)로 나누어 입력으로 사용한다. 이후 채널 믹싱(channel mixing)과 토큰 믹싱(token mixing) 등의 특징 추출 기법을 사용하여, 얻게 되는 특징들로 결함 상자가 어떤 종류의 결함인지 판단한다.The classifier uses the “MLP Mixer” as shown in Figure 1-③, and divides the defect box into 4 or 9 patches and uses them as input. Afterwards, feature extraction techniques such as channel mixing and token mixing are used to determine what type of defect the defect box is based on the obtained features.
본 발명은 기존의 영상 처리 기법을 사용하던 방법들과 달리 주변 환경의 변화에 강인하고, 모든 인쇄회로기판에 대하여 “Ground Truth” 가 필요했던 방식과 달리 데이터가 축적되면 각 입력에 대응하는 정답에 가장 가까운 이미지를 생성함으로써 매번 문턱값을 설정할 필요가 없다.Unlike methods using existing image processing techniques, the present invention is robust to changes in the surrounding environment, and unlike methods that require “ground truth” for all printed circuit boards, when data is accumulated, the correct answer corresponding to each input is provided. By generating the closest image, there is no need to set a threshold each time.
이와 같이, 본 발명이 속하는 기술분야의 당업자는 본 발명이 그 기술적 사상이나 필수적 특징을 변경하지 않고서 다른 구체적인 형태로 실시될 수 있다는 것을 이해할 수 있을 것이다. 그러므로 이상에서 기술한 실시예들은 모든 면에서 예시적인 것이며 한정적인 것이 아닌 것으로서 이해해야만 한다. 본 발명의 범위는 상기 상세한 설명보다는 후술하는 특허청구범위에 의하여 나타내어지며, 특허청구범위의 의미 및 범위 그리고 그 등가개념으로부터 도출되는 모든 변경 또는 변형된 형태가 본 발명의 범위에 포함되는 것으로 해석되어야 한다.As such, a person skilled in the art to which the present invention pertains will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.
Claims (3)
A printed circuit board defect detection method characterized by inferring the location of the defect using an outlier detection model in an image of the printed circuit board and then inferring the name of the defect using a classification model.
인쇄회로기판의 이미지에 “MLP Mixer”분류모델을 이용하여 결함의 이름을 추론하는 단계;
를 포함하는 인쇄회로기판 결함 감지방법.
Inferring the location of a defect in an image of a printed circuit board using an auto encoder that has learned the image shape of a normal printed circuit board; and
Inferring the name of the defect using the “MLP Mixer” classification model on the image of the printed circuit board;
A printed circuit board defect detection method comprising:
상기 결함의 이름을 추론하는 단계는,
결함이 발견된 위치 상하좌우의 일정 공간을 지정하여 결함 상자를 정의하는 단계;
상기 결함 상자를 4개 또는 9개의 패치(patch)로 나누어 상기 “MLP Mixer”분류모델의 입력으로 사용하는 단계; 및
채널 믹싱(channel mixing)과 토큰 믹싱(token mixing)의 특징 추출 기법을 사용하여, 획득된 특징들로 상기 결함 상자가 어떤 종류의 결함인지 판단하는 단계;를 포함하는 것을 특징으로 하는 인쇄회로기판 결함 감지방법.According to paragraph 2,
The step of inferring the name of the defect is,
Defining a defect box by designating a certain space above, below, left, and right of the location where the defect was found;
Dividing the defect box into 4 or 9 patches and using them as input to the “MLP Mixer” classification model; and
A printed circuit board defect comprising: determining what type of defect the defect box is based on the obtained features using feature extraction techniques of channel mixing and token mixing. Detection method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020220116065A KR20240037494A (en) | 2022-09-15 | 2022-09-15 | Printed circuit board defect detection method using semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020220116065A KR20240037494A (en) | 2022-09-15 | 2022-09-15 | Printed circuit board defect detection method using semi-supervised learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| KR20240037494A true KR20240037494A (en) | 2024-03-22 |
Family
ID=90481152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020220116065A KR20240037494A (en) | 2022-09-15 | 2022-09-15 | Printed circuit board defect detection method using semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR20240037494A (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102249841B1 (en) | 2020-05-27 | 2021-05-10 | 주식회사 엠아이티 | A Defect detection method of soldering lump that accumulatively improves accuracy using deep learning |
-
2022
- 2022-09-15 KR KR1020220116065A patent/KR20240037494A/en unknown
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102249841B1 (en) | 2020-05-27 | 2021-05-10 | 주식회사 엠아이티 | A Defect detection method of soldering lump that accumulatively improves accuracy using deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10684321B2 (en) | Printed circuit board inspecting apparatus, method for detecting anomaly in solder paste and computer readable recording medium | |
| TW202334900A (en) | Training data generation apparatus, training data generation method, and program | |
| KR102228957B1 (en) | Printed circuit board inspecting apparatus, method for determining fault type of screen printer and computer readable recording medium | |
| JP2015012532A5 (en) | ||
| KR20230027500A (en) | Board inspection method using a learning model | |
| JP2020087211A (en) | Learning model creation device, type determination system, and learning model creation method | |
| CN117437227B (en) | Image generation and defect detection method, device, medium, equipment and product | |
| CN117274245B (en) | AOI optical detection method and system based on image processing technology | |
| CN114387230A (en) | PCB defect detection method based on re-verification detection | |
| KR20240037494A (en) | Printed circuit board defect detection method using semi-supervised learning | |
| US20230298155A1 (en) | Information processing apparatus and information processing method | |
| KR20080011235A (en) | Substrate inspection device | |
| TW202113599A (en) | System for generating detection model according to standard data to confirm soldering state and method thereof | |
| JPH08327559A (en) | Device and method for inspecting pattern | |
| JP3279099B2 (en) | Mark recognition device and mark recognition method | |
| TWI773035B (en) | System and method for image recognition | |
| JPWO2020158630A1 (en) | Detector, learner, computer program, detection method and learner generation method | |
| JP2009150855A (en) | Inspection method of inspection target, and inspection device for the inspection target | |
| JP4812482B2 (en) | Printing state inspection apparatus and printing state inspection method | |
| KR20200011714A (en) | Method And Apparatus for Classifying Defective Electronic Component | |
| JP4084257B2 (en) | Printed circuit board inspection equipment | |
| JP2991520B2 (en) | Inspection device for print blur of printed matter | |
| CN116129435B (en) | Character defect detection method, device, equipment and storage medium | |
| TWI745068B (en) | Method for establishing seal identification model and server terminal for establishing seal identification model | |
| CN117333762B (en) | Image reproduction identification method based on multi-feature fusion |