CN111080615B - PCB defect detection system and detection method based on convolutional neural network - Google Patents
PCB defect detection system and detection method based on convolutional neural network Download PDFInfo
- Publication number
- CN111080615B CN111080615B CN201911273062.9A CN201911273062A CN111080615B CN 111080615 B CN111080615 B CN 111080615B CN 201911273062 A CN201911273062 A CN 201911273062A CN 111080615 B CN111080615 B CN 111080615B
- Authority
- CN
- China
- Prior art keywords
- pcb
- image
- defect
- detection system
- defect detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention discloses a PCB defect detection system based on a convolutional neural network, which comprises: the first image input module is used for inputting PCB images to be detected; the key point detection module is used for detecting at least two defect detection key points in the PCB image and extracting a PCB defect target image from the PCB image through each detected defect detection key point; the standard image acquisition module is used for acquiring and outputting a standard image corresponding to the PCB defect target image; the second image input module is used for inputting the PCB defect target image and the standard image into a PCB image segmentation module at the same time; the PCB image segmentation module is used for carrying out defect image segmentation on the PCB defect target image based on the standard image, and finally obtaining a specific area with defects in the PCB defect target image.
Description
Technical Field
The invention relates to a PCB defect detection system, in particular to a PCB defect detection system and method based on a convolutional neural network for detecting defects of a PCB.
Background
PCB (Printed Circuit Board) the Chinese name printed circuit board, also called printed circuit board, is an important electronic component, is a support for electronic components, and is a carrier for electrical connection of electronic components. Whether the PCB itself has defects or not directly affects the equipment performance of the equipment using the PCB, so the defect detection of the PCB is particularly necessary.
At present, the defect detection of the PCB circuit board is mainly traditional manual visual inspection, the manual visual inspection has higher omission rate and false detection rate, the visual inspection has very low detection efficiency, and a large amount of manpower is consumed, or the production cost of enterprises is directly improved, and the market competitiveness of products is reduced.
Disclosure of Invention
The invention aims to provide a PCB defect detection system based on a convolutional neural network and a method for detecting defects of a PCB printed circuit board by the PCB defect detection system based on the convolutional neural network so as to solve the technical problems.
To achieve the purpose, the invention adopts the following technical scheme:
the utility model provides a PCB defect detection system based on convolutional neural network for whether there is manufacturing defect to PCB printed circuit board detects, include:
a first image input module for inputting PCB image I to be detected o ；
The key point detection module is connected with the first image input module and is used for detecting the PCB image I o At least two defect detection key points are detected, and each detected defect detection key point is used for detecting the defect detection key point from the PCB image I o Extracting PCB defect target image I' t ；
A standard image acquisition module for acquiring the PCB defect target image I' t Corresponding standard image I s And output;
the second image input module is respectively connected with the key point detection module and the standard image acquisition module and is used for inputting the PCB defect target image I '' t And the standard image I s Simultaneously inputting the images into a PCB image segmentation module;
the PCB image segmentation module is connected with the second image input module and is used for being based on the standard image I s For the PCB defect target image I' t Performing defect image segmentation, and finally obtaining the PCB defect target image I '' t A specific area of defect is present.
The invention also provides a PCB defect detection method based on the convolutional neural network, which is realized by applying the PCB defect detection system and comprises the following steps:
step S1, the PCB defect detection system acquires the PCB image I to be detected o ；
Step S2, the PCB defect detection system detects the PCB image I o Detecting defect key points to obtain at least two defect detection key points;
step S3, the PCB defect detection system detects key points based on each defect, and the key points are detected from the PCB image I o Extracting the PCB defect target image I' t ；
Step S4, the PCB defect detection system detects the target image I 'of the PCB defect' t And a preset standard image I s Performing matching detection, and performing matching detection on the PCB defect target image I '' t Performing defect image segmentation, and finally obtaining the PCB defect target image I '' t The specific region in which the defect exists.
As a preferred embodiment of the present invention, the step S1 includes an image preprocessing process, and the image preprocessing process specifically includes the following steps:
step S11, the PCB defect detection system displays the PCB image I according to a preset scaling scale o Shrinking to obtain a scaled PCB image I r ；
Step S12, the PCB defect detection system detects the PCB image I r Performing image normalization processing to obtain a normalized PCB image I nd 。
As a preferred embodiment of the present invention, in the step S2, the PCB defect detection system performs the step of detecting the PCB image I nd The specific method for detecting the defect key points comprises the following steps:
step S21, the PCB defect detection system will said PCB image I nd Inputting into a preset key point detection model, and then outputting a score graph S representing the confidence of the defect key point d ；
Step S22, the PCB defect detection system calculates the score map S d The coordinates of the pixel point with the highest confidence in each channel are selected as the defect detection key point corresponding to the corresponding channel, and the defect detection key point is calculated and obtained in the PCB image I r Specific coordinates of (a);
step S23, the PCB defect detection system reversely calculates to obtain each defect detection key point in the PCB image I according to the scaling in the step S11 o Is defined in (a) is a specific coordinate of (b).
As a preferred embodiment of the present invention, in the step S3, the PCB defect detection system extracts the PCB image I from the PCB image I o Extracting the PCB defect target image I' t The specific method steps of (a) are as follows:
step S31, the PCB defect detection system detects the key points on the PCB image I based on each defect detection O From the PCB image I O Cut out target image I t ；
Step S32, the PCB defect detection system detects the target image I t Is adjusted to a preset size to obtain a PCB defect target image I 'after being adjusted in size' t 。
As a preferred embodiment of the present invention, in the step S4, the PCB defect detection system detects the PCB defect target image I' t The specific method for dividing the defect image comprises the following steps:
step S41, the PCB defect detection system acquires the PCB defect target image I' t And the PCB defect target image I' t Corresponding to the standard image I s ；
Step S42, the PCB defect detection system detects the target image I 'of the PCB defect' t And the standard image I s Connected in the channel direction to form a channel image I c ；
Step S43, the PCB defect detection system detects the channel image I c Performing image normalization processing to obtain a normalized PCB image I ns ；
Step S44, the PCB defect detection system detects the PCB image I ns Input into an image segmentation model for the PCB image I ns Predicting whether there is a defect, and outputting a defect probability map S S ；
Step S45, the PCB defect detection system judges the PCB image I ns Whether the pixel value corresponding to each pixel point is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains the PCB defect target image I 'according to the judging result' t The specific region in which the defect exists.
As a preferable embodiment of the present invention, the threshold value in the step S45 is 0.5.
The PCB defect detection system provided by the invention is based on the deep learning convolutional neural network, can automatically detect whether the product defects exist on the PCB circuit board, solves the problems of easy missed detection and false detection in the conventional manual visual detection mode, greatly improves the detection efficiency and the detection accuracy, is beneficial to improving the product quality of production enterprises and improves the market competitiveness of the products.
Drawings
In order to more clearly illustrate the technical solution of the embodiments of the present invention, the drawings that are required to be used in the embodiments of the present invention will be briefly described below. It is evident that the drawings described below are only some embodiments of the present invention and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a schematic diagram of a PCB defect detection system provided by the present invention;
FIG. 2 is a diagram of steps in a method for detecting PCB defects on a PCB printed circuit board using the PCB defect detection system provided by the present invention;
FIG. 3 is a diagram showing an input PCB image I in the PCB defect detection method according to the present invention o A method step diagram for image preprocessing is carried out;
FIG. 4 is a diagram showing an image I of a PCB in the PCB defect detection method according to the present invention nd Performing a specific step diagram of defect key point detection;
FIG. 5 is a diagram showing the image I of a PCB in the method for detecting defects of a PCB according to the present invention o Extracting PCB defect target image I' t Is characterized by comprising the following specific steps;
FIG. 6 is a target image I 'of PCB defects in the PCB defect detection method according to the present invention' t And carrying out a specific method step diagram of defect image segmentation.
Detailed Description
The technical scheme of the invention is further described below by the specific embodiments with reference to the accompanying drawings.
Wherein the drawings are for illustrative purposes only and are shown in schematic, non-physical, and not intended to be limiting of the present patent; for the purpose of better illustrating embodiments of the invention, certain elements of the drawings may be omitted, enlarged or reduced and do not represent the size of the actual product; it will be appreciated by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numbers in the drawings of embodiments of the invention correspond to the same or similar components; in the description of the present invention, it should be understood that, if the terms "upper", "lower", "left", "right", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, only for convenience in describing the present invention and simplifying the description, rather than indicating or implying that the apparatus or elements being referred to must have a specific orientation, be constructed and operated in a specific orientation, so that the terms describing the positional relationships in the drawings are merely for exemplary illustration and should not be construed as limiting the present patent, and that the specific meaning of the terms described above may be understood by those of ordinary skill in the art according to specific circumstances.
In the description of the present invention, unless explicitly stated and limited otherwise, the term "coupled" or the like should be interpreted broadly, as it may be fixedly coupled, detachably coupled, or integrally formed, as indicating the relationship of components; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between the two parts or interaction relationship between the two parts. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
Referring to fig. 1, the present embodiment provides a PCB defect detection system based on convolutional neural network, for detecting whether there is a manufacturing defect of a PCB printed circuit board, the PCB defect detection system includes:
a first image input module 1 for inputting a PCB image I to be detected o ；
The key point detection module 2 is connected with the first image input module 1 and is used for detecting the key point of the PCB image I o At least two defect detection key points are detected, and each detected defect detection key point is used for detecting the defect of the PCB image I o Extracting PCB defect target image I' t ；
A standard image acquisition module 3 for acquiring a PCB defect target image I' t Corresponding standard image I s And output;
the second image input module 4 is respectively connected with the key point detection module 2 and the standard image acquisition module 3 and is used for inputting the PCB defect target image I '' t And standard image I s Simultaneously inputting the images into a PCB image segmentation module;
a PCB image segmentation module 5 connected to the second image input module 4 for standard image I-based s Target image I 'of PCB defect' t Performing defect image segmentation, and finally obtaining a PCB defect target image I '' t In which there is a defectSpecific areas.
The invention also provides a PCB defect detection method based on the convolutional neural network, which is realized by applying the PCB defect detection system, referring to FIG. 2, and specifically comprises the following steps:
step S1, a PCB defect detection system acquires a PCB image I to be detected o ；
Step S2, the PCB defect detection system detects the PCB image I o Performing defect key point detection to obtain at least two defect detection key points;
step S3, the PCB defect detection system detects key points based on each defect, and the key points are detected from the PCB image I o Extracting PCB defect target image I' t ；
Step S4, the PCB defect detection system detects the target image I 'of the PCB defect' t And a preset standard image I s Performing matching detection and performing target image I 'on PCB defects' t Performing defect image segmentation, and finally obtaining a PCB defect target image I '' t A specific area of defect is present.
Referring to fig. 3, the step S1 includes an image preprocessing process, which specifically includes the following steps:
step S11, the PCB defect detection system displays the PCB image I according to a predetermined scaling o Shrinking to obtain a scaled PCB image I r ；
Step S12, the PCB defect detection system detects the PCB image I r Performing image normalization processing to obtain a normalized PCB image I nd 。
In the above technical scheme, specifically, the PCB defect detection system acquires the PCB image I through a camera o ∈R 1920 ×1080×3 (numerals 1920, 1080 and 3 respectively represent PCB image I) o Length, width, number of channels);
in this embodiment, the PCB defect detection system performs the input of PCB image I at a 3-fold scale o PCB image I scaled down to 640 x 360 size r ∈R 640×360×3 (numerals 640, 360 and 3 represent PCB image I, respectively r Length, width andnumber of channels);
subsequently, the PCB defect detection system will PCB image I r Dividing the pixel values of all the pixel points in the array by 255 (normalizing the pixel values from 0-255 to 0-1) to obtain a normalized PCB image I nd ∈R 640×360×3 (numerals 640, 360 and 3 represent PCB image I, respectively nd Length, width and number of channels).
Referring to fig. 4, in step S2, the PCB defect detection system detects a PCB image I nd The specific method for detecting the defect key points comprises the following steps:
step S21, the PCB defect detection system detects the PCB image I nd Inputting into a preset key point detection model, and then outputting a score graph S representing the confidence of the defect key point d ；
Step S22, the PCB defect detection system calculates a score map S d The coordinates of the pixel point with the highest confidence in each channel are selected as the defect detection key point corresponding to the corresponding channel, and the defect detection key point is calculated and obtained in the PCB image I r Specific coordinates of (a);
step S23, the PCB defect detection system reversely calculates to obtain the PCB image I of each defect detection key point according to the scaling in step S11 o Is defined in (a) is a specific coordinate of (b).
In this embodiment, the number of detection key points of the system for defect detection may be two, and taking the system for detecting two key points of defect detection as an example, specifically, in the above technical solution, the system will preprocess the PCB image I nd Inputting into the key point detection model, and inputting PCB image I nd Extracting the image feature through the key point detection model to output a feature image with the size of 160 x 90 x 270, and then up-sampling the feature image by the system to obtain an input PCB image I nd The feature map size of the same size is converted from 160×90×270 to 640×360×2. Then the system sequentially carries out further image feature extraction on the converted feature images through two convolution layers and a Sigmoid function activation layer to output a score image S with the size of 640 x 360 x 2 d . The score map S d Two channels are provided, which respectively correspond to the PCB defect target image I' t The two vertices of the upper left and lower right corners (i.e., the target detection area) of each pixel in each channel represent the confidence that the pixel is predicted to be the vertex (upper left or lower right corner) corresponding to that channel.
The system then calculates a score map S d The coordinates of the pixel point with the highest confidence in each channel are selected as the corresponding defect detection key point of the channel, so that the two vertexes of the upper left corner and the lower right corner of the target detection area are obtained in the PCB image I r Specific coordinates { (x) 1 ,y 1 ),(x 2 ,y 2 )}；
x 1 For representing first defect detection key points in PCB image I r A transverse coordinate value of (a);
y 1 for representing first defect detection key points in PCB image I r A longitudinal coordinate value of (a);
x 2 for representing the second defect detection key point in PCB image I r Transverse coordinate values of (2);
y 2 for representing the second defect detection key point in PCB image I r Is defined by the longitudinal coordinate values of (a).
Finally, the system calculates and obtains the first defect detection key point in the original PCB image I based on the 3-part scaling o Coordinates (3 x) 1 ,3y 1 )；
And calculate and get the PCB image I with the second defect detection key point at the beginning o Coordinates (3 x) 2 ,3y 2 )。
In the technical scheme, the key point detection model is trained through a first convolution neural network, and the first convolution neural network realizes detection and identification of the defect detection key points through a first main network and the key point detection network. In this embodiment, the specific network structure and network parameters of the first convolutional neural network adopted by the system are shown in the following table a:
| layer(s) | Filter device | Step size | Output size |
| Input | 640x360x3 | ||
| HRNet18- |
4 | 160x90x270 | |
| UpSampling | 640x360x270 | ||
| Conv1 | 1x1x256 | 1 | 640x360x256 |
| Conv2 | 1x1x4 | 1 | 640x360x2 |
| Sigmoid | 640x360x2 |
Table a
In table a, english input is an input layer of the first convolutional neural network, and is used for inputting PCB image I with a size of 640×360×3 nd 。
HRNet18-Det represents a keypoint detection model that is pre-trained, and its training method is a model training method existing in the prior art, and the training method is not a scope of the invention claimed, so specific method steps for training the keypoint detection model are not described herein. PCB image I with size of 640 x 360 x 3 nd And extracting the image features through the key point detection model to output a feature map with the size of 160 x 90 x 270.
UpSampling is an UpSampling layer, and a feature map with a size of 160 x 90 x 270 is processed by the UpSampling layer to obtain an input PCB image I nd Feature maps of the same size;
conv1 is a first convolution layer, the convolution kernel size is 1 x 256, the step length is 1, and the feature map output by the up-sampling layer processing is convolved by the first convolution layer to obtain the feature map with the size of 640 x 360 x 256.
Conv2 is a second convolution layer with a convolution kernel size of 1×1×4 and a step size of 1, and converts a feature map with a size of 640×360×256 into a feature map with a size of 640×360×2.
Sigmoid is a function activation layer, and a feature map with the size of 640 x 360 x 2 outputs a score map S with the size of 640 x 360 x 2 after being processed by the function activation layer d 。
Referring to fig. 5, in step S3, the PCB defect detection system detects a defect from the PCB image I o Extraction of the Chinese medicineOutputting a PCB defect target image I' t The specific method comprises the following steps:
step S31, the PCB defect detection system detects the specific coordinates of the key points in the PCB image based on the specific coordinates of the key points in the PCB image, and the key points are detected from the PCB image I O Cut out target image I t ；
Step S32, the PCB defect detection system detects the target image I t Is adjusted to a preset size to obtain a PCB defect target image I 'after being adjusted to the preset size' t 。
In the present embodiment, specifically, the system generates the original PCB image I based on the coordinates of the two vertices of the upper left corner and the lower right corner of the target detection area o Cut out target image I t ＝I o [3y 1 :3y 2 ,3x 1 :3x 2 ,:]；
The system then takes the target image I t PCB defect target image I 'sized 640 x 360 x 3' t ∈R 640 ×360×3 。
Referring to fig. 6, in step S4, the PCB defect detection system detects a PCB defect target image I' t The specific method for dividing the defect image comprises the following steps:
step S41, the PCB defect detection system acquires a PCB defect target image I' t And PCB defect target image I' t Corresponding standard image I s ；
Step S42, the PCB defect detection system detects the target image I 'of the PCB defect' t And standard image I s Connected in the channel direction to form a channel image I c ；
Step S43, the PCB defect detection system displays the channel image I c Performing image normalization processing to obtain a normalized PCB image I ns ；
Step S44, the PCB defect detection system detects the PCB image I ns Input into an image segmentation model for PCB image I ns Predicting whether there is a defect, and outputting a defect probability map S S ；
Step S45, the PCB defect detection system judges the PCB image I ns Each of (3)Whether the pixel value corresponding to one pixel point is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains a PCB defect target image I 'according to the judging result' t A specific area of defect is present.
In the above technical scheme, the image segmentation model is trained by a second convolutional neural network, and the second convolutional neural network realizes image segmentation of the PCB image by a second backbone network and the image segmentation network. In this embodiment, the specific network structure and network parameters of the second convolutional neural network adopted by the system are shown in the following table b:
table b
In the above table b, english input represents the input layer of the second convolutional neural network, and the input image of the input layer is the PCB defect target image I' t What PCB defect target image I' t Corresponding standard image I s PCB defect target image I' t And standard image I s Since the size of the input layer is 640×360×3, the size of the image input to the input layer is 640×360×6 (length×width×channel number);
HRNet18-Seg is used for representing an image segmentation model, and an input image of 6 channels is subjected to image feature extraction of the image segmentation model to output a feature map with the size of 160 x 90 x 270;
UpSampling is the UpSampling layer. The system then upsamples the feature map with a size of 160 x 90 x 270 to obtain a feature map with a size of 640 x 360 x 270.
Conv1 is a first convolution layer in the second convolution neural network, and the system further extracts image features from the feature map with the size of 640 x 360 x 270 through the first convolution layer to obtain a feature map with the size of 640 x 360 x 256. The convolution kernel size of the first convolution layer is 1 x 256, with a step size of 1.
Conv2 is a second convolution layer in the second convolution neural network, and the system further extracts image features of the feature map with the size of 640 x 360 x 256 through the second convolution layer to obtain the feature map with the size of 640 x 360 x 1. The convolution kernel size of the second convolution layer is 1 x 1, the step size is 1.
Finally, the system outputs a defect probability map S with the size of 640 x 360 x 1 based on the feature map with the size of 640 x 360 x 1 through the Sigmoid function activation layer S 。
In the above technical solution, the preset threshold is preferably 0.5. That is, when the defect probability map S S The pixel value of the pixel point is larger than 0.5, which indicates that the pixel point has defects, if the defect probability map S S If the pixel value of the pixel point is less than or equal to 0.5, the pixel point is free from defects.
In addition, in the above technical solution, the training methods of the key point detection model and the image segmentation model are both existing training methods of the recognition models, and specific training processes of the two recognition models by the convolutional neural network are not described herein.
In summary, the invention can automatically detect whether the PCB has product defects, solves the problem that the traditional manual visual inspection mode is easy to cause missed inspection and false inspection, and greatly improves the detection efficiency and the detection accuracy.
It should be understood that the above description is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be apparent to those skilled in the art that various modifications, equivalents, variations, and the like can be made to the present invention. However, such modifications are intended to fall within the scope of the present invention without departing from the spirit of the present invention. In addition, some terms used in the specification and claims of the present application are not limiting, but are merely for convenience of description.
Claims (7)
1. A convolutional neural network-based PCB defect detection system for detecting whether a manufacturing defect exists in a PCB printed circuit board, comprising:
a first image input module for inputting PCB image I to be detected o ；
The key point detection module is connected with the first image input module and is used for detecting the PCB image I o At least two defect detection key points are detected, and each detected defect detection key point is used for detecting the defect detection key point from the PCB image I o Extracting PCB defect target image I' t ；
A standard image acquisition module for acquiring the PCB defect target image I' t Corresponding standard image I s And output;
the second image input module is respectively connected with the key point detection module and the standard image acquisition module and is used for inputting the PCB defect target image I '' t And the standard image I s Simultaneously inputting the images into a PCB image segmentation module;
the PCB image segmentation module is connected with the second image input module and is used for being based on the standard image I s For the PCB defect target image I' t Performing defect image segmentation, and finally obtaining the PCB defect target image I '' t A specific region in which a defect exists;
the key point detection module realizes detection and identification of the defect detection key points through a preset key point detection model, the key point detection model is trained through a first convolution neural network, and the first convolution neural network realizes detection and identification of the defect detection key points through a first trunk network and the key point detection network;
the PCB image segmentation module achieves the PCB defect target image I 'through a preset image segmentation model' t The image segmentation model is trained by a second convolutional neural network, and the second convolutional neural network realizes the segmentation of the image through a second backbone network and the image segmentation networkThe PCB defect target image I' t Is divided into defect images;
the PCB defect detection system detects a defect from PCB image I o Extracting PCB defect target image I' t The specific method comprises the following steps:
step S31, the PCB defect detection system detects the specific coordinates of the key points in the PCB image based on the specific coordinates of the key points in the PCB image, and the key points are detected from the PCB image I O Cut out target image I t ；
Step S32, the PCB defect detection system detects the target image I t Is adjusted to a preset size to obtain a PCB defect target image I 'after being adjusted to the preset size' t ；
The PCB defect detection system is based on the coordinates of two vertexes of the upper left corner and the lower right corner of the target detection area, and is used for detecting the defects of the PCB from the original PCB image I o Cut out target image I t ＝I o [3y 1 :3y 2 ,3x 1 :3x 2 ,:]Wherein x is 1 、y 1 X represents the horizontal axis coordinate and the vertical axis coordinate of the vertex of the upper left corner respectively 2 、y 2 Respectively representing the horizontal axis coordinate and the vertical axis coordinate of the vertex of the upper right angle,
the system then takes the target image I t PCB defect target image I 'sized 640 x 360 x 3' t ∈R 640 ×360×3 ；
The PCB defect detection system is used for detecting a target image I 'of PCB defects' t The specific method for dividing the defect image comprises the following steps:
step S41, the PCB defect detection system acquires a PCB defect target image I' t And PCB defect target image I' t Corresponding standard image I s ；
Step S42, the PCB defect detection system detects the target image I 'of the PCB defect' t And standard image I s Connected in the channel direction to form a channel image I c ；
Step S43, the PCB defect detection system displays the channel image I c Performing image normalization processing to obtain a normalized PCB image I ns ；
Step S44, PCB defectThe detection system detects the PCB image I ns Input into an image segmentation model for PCB image I ns Predicting whether there is a defect, and outputting a defect probability map S S ；
Step S45, the PCB defect detection system judges the PCB image I ns Whether the pixel value corresponding to each pixel point is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains a PCB defect target image I 'according to the judging result' t A specific area of defect is present.
2. A method for detecting defects of a PCB based on a convolutional neural network, implemented by applying the system for detecting defects of a PCB according to claim 1, comprising the steps of:
step S1, the PCB defect detection system acquires the PCB image I to be detected o ；
Step S2, the PCB defect detection system detects the PCB image I o Detecting defect key points to obtain at least two defect detection key points;
step S3, the PCB defect detection system detects key points based on each defect, and the key points are detected from the PCB image I o Extracting the PCB defect target image I' t ；
Step S4, the PCB defect detection system detects the target image I 'of the PCB defect' t And a preset standard image I s Performing matching detection, and performing matching detection on the PCB defect target image I '' t Performing defect image segmentation, and finally obtaining the PCB defect target image I '' t The specific region in which the defect exists.
3. The method of detecting a PCB defect according to claim 2, wherein the step S1 includes an image preprocessing process, and the image preprocessing process specifically includes the steps of:
step S11, the PCB defect detection system displays the PCB image I according to a preset scaling scale o Shrinking to obtain a scaled PCB image I r ；
Step S12, the PCB defect detection system detects the PCB image I r Performing image normalization processing to obtain a normalized PCB image I nd 。
4. The method of claim 3, wherein in the step S2, the PCB defect detection system detects the PCB image I nd The specific method for detecting the defect key points comprises the following steps:
step S21, the PCB defect detection system detects the PCB image I nd Inputting into a preset key point detection model, and then outputting a score graph S representing the confidence of the defect key point d ；
Step S22, the PCB defect detection system calculates the score map S d The coordinates of the pixel point with the highest confidence in each channel are selected as the defect detection key point corresponding to the corresponding channel, and the defect detection key point is calculated and obtained in the PCB image I r Specific coordinates of (a);
step S23, the PCB defect detection system reversely calculates to obtain each defect detection key point in the PCB image I according to the scaling in the step S11 o Is defined in (a) is a specific coordinate of (b).
5. The PCB defect detection method of claim 4, wherein in step S3, the PCB defect detection system extracts the PCB image I from the PCB image I o Extracting the PCB defect target image I' t The specific method comprises the following steps:
step S31, the PCB defect detection system detects the key points on the PCB image I based on each defect detection O From the PCB image I O Cut out the targetImage I t ；
Step S32, the PCB defect detection system detects the target image I t Is adjusted to a preset size to obtain a PCB defect target image I 'after being adjusted in size' t 。
6. The method of claim 5, wherein in the step S4, the PCB defect detection system detects the PCB defect target image I' t The specific method for dividing the defect image comprises the following steps:
step S41, the PCB defect detection system acquires the PCB defect target image I' t And the PCB defect target image I' t Corresponding to the standard image I s ；
Step S42, the PCB defect detection system detects the target image I 'of the PCB defect' t And the standard image I s Connected in the channel direction to form a channel image I c ；
Step S43, the PCB defect detection system detects the channel image I c Performing image normalization processing to obtain a normalized PCB image I ns ；
Step S44, the PCB defect detection system detects the PCB image I ns Input into an image segmentation model for the PCB image I ns Predicting whether there is a defect, and outputting a defect probability map S S ；
Step S45, the PCB defect detection system judges the PCB image I ns Whether the pixel value corresponding to each pixel point is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains the PCB defect target image I 'according to the judging result' t The specific region in which the defect exists.
7. The PCB defect detection method of claim 6, wherein the threshold in step S45 is 0.5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911273062.9A CN111080615B (en) | 2019-12-12 | 2019-12-12 | PCB defect detection system and detection method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911273062.9A CN111080615B (en) | 2019-12-12 | 2019-12-12 | PCB defect detection system and detection method based on convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111080615A CN111080615A (en) | 2020-04-28 |
| CN111080615B true CN111080615B (en) | 2023-06-16 |
Family
ID=70314112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911273062.9A Active CN111080615B (en) | 2019-12-12 | 2019-12-12 | PCB defect detection system and detection method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111080615B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111899241B (en) * | 2020-07-28 | 2022-03-18 | 华中科技大学 | Quantitative on-line detection method and system for defects of PCB (printed Circuit Board) patches in front of furnace |
| CN113609897A (en) * | 2021-06-23 | 2021-11-05 | 阿里巴巴新加坡控股有限公司 | Defect detection method and defect detection system |
| WO2023102723A1 (en) * | 2021-12-07 | 2023-06-15 | 宁德时代新能源科技股份有限公司 | Image processing method and system |
| CN114489914B (en) * | 2022-02-17 | 2022-11-29 | 广州嘉普信息科技有限公司 | Method for generating template key area of PCB intelligent detection system and electronic equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130142769A (en) * | 2012-06-20 | 2013-12-30 | 한국기술교육대학교 산학협력단 | Apparatus and method for vision inspecting using image division |
| CN106897994A (en) * | 2017-01-20 | 2017-06-27 | 北京京仪仪器仪表研究总院有限公司 | A kind of pcb board defect detecting system and method based on layered image |
| CN107389701A (en) * | 2017-08-22 | 2017-11-24 | 西北工业大学 | A kind of PCB visual defects automatic checkout system and method based on image |
| CN107610085A (en) * | 2016-07-11 | 2018-01-19 | 富强 | A kind of welding point defect detecting system based on computer vision |
| CN108122247A (en) * | 2017-12-25 | 2018-06-05 | 北京航空航天大学 | A kind of video object detection method based on saliency and feature prior model |
| CN109101983A (en) * | 2018-08-02 | 2018-12-28 | 大连恒锐科技股份有限公司 | A kind of shoe pattern and footprint critical point detection method based on deep learning |
| KR101950681B1 (en) * | 2018-05-04 | 2019-02-21 | 주식회사 스켈터랩스 | Defect inspection device, system, method and computer readable storage medium |
| CN109829900A (en) * | 2019-01-18 | 2019-05-31 | 创新奇智(北京)科技有限公司 | A kind of steel coil end-face defect inspection method based on deep learning |
| CN109859207A (en) * | 2019-03-06 | 2019-06-07 | 华南理工大学 | A kind of defect inspection method of high density flexible substrate |
| CN109919908A (en) * | 2019-01-23 | 2019-06-21 | 华灿光电（浙江）有限公司 | The method and apparatus of light-emitting diode chip for backlight unit defects detection |
| CN110320285A (en) * | 2019-07-08 | 2019-10-11 | 创新奇智(青岛)科技有限公司 | A kind of steel rail defect recognition methods, system and electronic equipment based on ultrasonic signal |
| CN110443777A (en) * | 2018-10-09 | 2019-11-12 | 广东仙童智能机器人科技有限公司 | A kind of PCB image alignment method, device and computer storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111684492B (en) * | 2017-06-26 | 2024-03-15 | 医科达有限公司 | Method for improving cone beam CT image quality using deep convolutional neural network |
-
2019
- 2019-12-12 CN CN201911273062.9A patent/CN111080615B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130142769A (en) * | 2012-06-20 | 2013-12-30 | 한국기술교육대학교 산학협력단 | Apparatus and method for vision inspecting using image division |
| CN107610085A (en) * | 2016-07-11 | 2018-01-19 | 富强 | A kind of welding point defect detecting system based on computer vision |
| CN106897994A (en) * | 2017-01-20 | 2017-06-27 | 北京京仪仪器仪表研究总院有限公司 | A kind of pcb board defect detecting system and method based on layered image |
| CN107389701A (en) * | 2017-08-22 | 2017-11-24 | 西北工业大学 | A kind of PCB visual defects automatic checkout system and method based on image |
| CN108122247A (en) * | 2017-12-25 | 2018-06-05 | 北京航空航天大学 | A kind of video object detection method based on saliency and feature prior model |
| KR101950681B1 (en) * | 2018-05-04 | 2019-02-21 | 주식회사 스켈터랩스 | Defect inspection device, system, method and computer readable storage medium |
| CN109101983A (en) * | 2018-08-02 | 2018-12-28 | 大连恒锐科技股份有限公司 | A kind of shoe pattern and footprint critical point detection method based on deep learning |
| CN110443777A (en) * | 2018-10-09 | 2019-11-12 | 广东仙童智能机器人科技有限公司 | A kind of PCB image alignment method, device and computer storage medium |
| CN109829900A (en) * | 2019-01-18 | 2019-05-31 | 创新奇智(北京)科技有限公司 | A kind of steel coil end-face defect inspection method based on deep learning |
| CN109919908A (en) * | 2019-01-23 | 2019-06-21 | 华灿光电（浙江）有限公司 | The method and apparatus of light-emitting diode chip for backlight unit defects detection |
| CN109859207A (en) * | 2019-03-06 | 2019-06-07 | 华南理工大学 | A kind of defect inspection method of high density flexible substrate |
| CN110320285A (en) * | 2019-07-08 | 2019-10-11 | 创新奇智(青岛)科技有限公司 | A kind of steel rail defect recognition methods, system and electronic equipment based on ultrasonic signal |
Non-Patent Citations (2)
| Title |
|---|
| Jian CHENG;Pei-song WANG;Gang LI;Qing-hao HU;Han-qing LU.Recent advances in efficient computation of deep convolutional neural networks.Frontiers of Information Technology & Electronic Engineering.2015,(第01期),全文. * |
| 邓桂平;申莉;田忠春;杨建荣.智能电能表质量一致性检测系统设计.电测与仪表.2015,(第S1期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111080615A (en) | 2020-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080615B (en) | PCB defect detection system and detection method based on convolutional neural network | |
| CN111709909B (en) | General printing defect detection method based on deep learning and model thereof | |
| CN108108746B (en) | License plate character recognition method based on Caffe deep learning framework | |
| CN104751142B (en) | A kind of natural scene Method for text detection based on stroke feature | |
| CN106599028B (en) | Book content searching and matching method based on video image processing | |
| CN106548182B (en) | Pavement crack detection method and device based on deep learning and main cause analysis | |
| US20080232715A1 (en) | Image processing apparatus | |
| CN109190458B (en) | Method for detecting head of small person based on deep learning | |
| CN111915704A (en) | Apple hierarchical identification method based on deep learning | |
| CN110346699B (en) | Insulator discharge information extraction method and device based on ultraviolet image processing technology | |
| CN105590319A (en) | Method for detecting image saliency region for deep learning | |
| CN110991448A (en) | Text detection method and device for nameplate image of power equipment | |
| CN114155546B (en) | Image correction method and device, electronic equipment and storage medium | |
| CN108830133A (en) | Recognition methods, electronic device and the readable storage medium storing program for executing of contract image picture | |
| CN113657528B (en) | Image feature point extraction method and device, computer terminal and storage medium | |
| CN111210417B (en) | Cloth defect detection method based on convolutional neural network | |
| CN110222647B (en) | Face in-vivo detection method based on convolutional neural network | |
| CN111461076A (en) | Smoke detection method and smoke detection system combining frame difference method and neural network | |
| CN116824347A (en) | Road crack detection method based on deep learning | |
| CN113177941B (en) | Steel coil edge crack identification method, system, medium and terminal | |
| CN111079749B (en) | End-to-end commodity price tag character recognition method and system with gesture correction | |
| CN111666813A (en) | Subcutaneous sweat gland extraction method based on three-dimensional convolutional neural network of non-local information | |
| CN114581446B (en) | Battery core abnormity detection method and system of laminated battery | |
| CN115810197A (en) | Multi-mode electric power form recognition method and device | |
| CN114299020A (en) | Scooter display screen defect detection method and system based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |