CN116309427A - PCB surface defect detection method based on improved YOLOv5 algorithm - Google Patents

Abstract

The invention discloses a PCB surface defect detection method based on an improved YOLOv5 algorithm, which comprises the following steps: the method comprises the steps of obtaining a public PCB data set, preprocessing, generating initial Anchor frame parameters which are most in line with the characteristics of the data set by using a K-means clustering algorithm, and expanding the data set by using a data enhancement strategy of Mosaic+Mixup; establishing a YOLOv5 network structure, introducing a lightweight network GhostNet to replace a residual structure in a CSP1 module in the YOLOv5 backbone network, introducing a CIOU loss function to solve the degradation problem of the original YOLOv5 loss function GIOU, adding the correlation among SE attention mechanism learning channels, and improving the detection precision; introducing a weighted bidirectional feature network, obtaining a multi-level feature network by utilizing weighted feature fusion and trans-scale connection, and improving the original FPN+PAN structure of YOLOv 5; determining a network loss function and a performance evaluation index of the YOLOv5 algorithm according to the CIOU_loss; sending the PCB surface defect sample into an improved YOLOv5 network model for training to obtain a target detection optimal model; and finally, detecting the surface defects of the PCB by using the test set, outputting a detection result, and evaluating the detection result. The average precision mean value of the improved YOLOv5 reaches 96.88%, the single picture detection time is not more than 50 milliseconds, and the method is superior to the traditional machine learning algorithm, and can meet the real-time detection requirement on PCB defects in actual engineering.

Description

PCB surface defect detection method based on improved YOLOv5 algorithm

Technical Field

The invention relates to the field of artificial intelligent target detection, in particular to a PCB surface defect detection method based on an improved YOLOv5 algorithm.

Background

Along with the development of the technology level, electronic devices gradually enter into the life of people, especially in the application of high-tech systems such as precision instruments, supercomputers, radar systems and the like, and PCB boards are in an important position. From the end of the 90 s to the present, china has become the manufacturing, import and consumption country of electronic products, and global PCB production bases and factories of factories are gradually migrated from European and American areas, india areas, middle east areas and the like to China. China has now become the largest PCB production base in the world. By 2015, the total yield of the Chinese circuit board reaches about 2000 hundred million RMB, accounting for about 45% of the world. However, the whole life cycle of an electronic system often faces multiple physical field effects induced by complex environments such as temperature, humidity, vibration, electromagnetic field and the like, and sensitive characteristic parameters of an electronic assembly material are degraded, so that parameter deviation, system out-of-tolerance and even functional failure of a product occur. So in production, if a problematic PCB is not detected, the loss it causes may be immeasurable. Light weight can cause malfunction of the instrument and heavy weight can cause fire or major engineering accidents. The defect detection of PCBs, especially before application, is particularly important in engineering.

The defects of the PCB commonly existing at present are short circuit, open circuit, rat bite, burrs, false copper, leakage hole and the like. These common defects are often found in complex PCB circuits, which makes the detection difficult and also prone to missed or false detection. In industrial production, methods such as manual visual inspection, functional test and visual inspection are generally adopted. At present, the requirement of modern production cannot be met by artificial vision, functional testing also depends on professional detection equipment, the cost is high, the visual detection is based on a detection scheme of computer vision, the possibility of secondary damage is avoided by adopting image recognition detection without direct contact with a PCB, meanwhile, the detection efficiency of images is greatly improved along with the development of computer hardware, in the field of image detection, a target detection algorithm based on deep learning and a traditional detection algorithm are improved in detection precision and detection degree, and more recognition detection scenes adopt the scheme to carry out floor conversion.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a PCB surface defect detection method based on an improved YOLOv5 algorithm.

In order to solve the technical problems, the invention provides the following technical scheme:

the invention relates to a PCB surface defect detection method based on an improved YOLOv5 algorithm, which comprises the following specific steps:

step 1: and acquiring a disclosed data set, which is used for detecting surface defects of the PCB, marking pictures in the data set by using a Lambimg tool, generating a label in a YOLO format, and dividing the data set into a training set, a testing set and a verification set. And then preprocessing the PCB picture: generating initial Anchor frame parameters which are most in line with the characteristics of a data set by using a K-means clustering algorithm, processing PCB data information by adopting a data enhancement strategy of Mosaic+Mixup, firstly splicing four pictures by using a Mosaic method to obtain a picture which is rich in background and provided with a corresponding labeling frame, and then increasing the defect number on the picture by using a Mixup method to improve the detection effect on a small target;

step 2: a PCB detection model based on the YOLOv5 is built, a lightweight network GhostNet is introduced to replace a residual structure in a CSP1 module in the YOLOv5 backbone network, an SE attention mechanism is introduced, and a CIOU loss function is introduced to replace an original loss function GIOU. Compared with the traditional convolution, the implementation of the Ghost convolution is divided into two parts, firstly, a part of characteristic diagrams with fewer channels are obtained through normal convolution calculation, then, more characteristic diagrams are obtained through simple linear operation on the characteristic diagrams, and finally, the two groups of characteristic diagrams are spliced to form new output, so that the improvement on the YOLOv5 trunk part is completed;

step 3: introducing a weighted bidirectional feature network, obtaining a multi-level feature network by utilizing weighted feature fusion and cross-scale connection, improving the original FPN+PAN structure of YOLOv5, and improving the detection efficiency of a small target;

step 4: and determining a network loss function and a performance evaluation index of the YOLOv5 algorithm according to the CIOU_loss, wherein the network loss function consists of two parts, namely regression loss and classification loss. Regression lOSS is also called coordinate position lOSS, and CIOU_lOSS is used as position lOSS. Classification loss comprises class loss and confidence loss, and the class loss and the confidence loss are evaluated by adopting Focal_loss;

step 5: sending the PCB surface defect sample into an improved YOLOv5 network model for training to obtain a target detection optimal model;

step 6: and 5, taking sample data in the test set as input, carrying out target detection through the model improved in the step 5, outputting a detection result, and evaluating the detection result.

As a preferred technical solution of the present invention, the step 1 includes the following steps:

step 1.1: 1000 PCB data sets containing defect information are obtained, the size of each image is 640 x 640, an image classification program is compiled, training sets, verification sets and test sets are randomly distributed in a ratio of 7:2:1, and randomness of the defect data is guaranteed.

Step 1.2: anchor frame labeling is carried out on PCB images in a training set by using a Labelimg tool, and target classes are divided into six classes of missing_hole, open_ circuit, spurious _cap and mouse_ bite, short, spur, wherein the missing_hole label represents a leak hole, the open_circuit label represents an open circuit, the spilus_cap label represents pseudo copper, the mouse_bit label represents a mouse bite, the short label represents a short circuit and the spir label represents burrs.

Step 1.3: and after all the images are marked, converting the generated xml format defect information into txt format.

As a preferred technical solution of the present invention, the step 2 includes the following steps:

step 2.1: the extraction module used in the backbox part of the YOLOv5 network is C3, the module is formed by stacking a butteleneck module and a CBS convolution block, and too many convolution layers increase the calculation amount and the network parameter number, so that a lightweight network, ghostNet, is introduced to replace the residual structure in the CSP1 module in the YOLOv5 Backbone network.

Step 2.2: and adding an SE attention detection mechanism into the YOLOv5 network to solve the problem of loss caused by different weights of different channels of the feature map in the pooling process, adding an SE module into a Backbone part, learning the correlation among the channels, screening the attention to the channels, and effectively improving the detection precision.

Step 2.3: the CIOU loss function is introduced to replace the original network GIOU loss function, and the CIOU considers the overlapping area, the center point distance and the length-width ratio, so that the regression of the target frame is more stable, and the convergence speed is faster.

As a preferred embodiment of the present invention, the step 6 includes the following steps:

step 6.1: if the model detects six defects such as missing_hole, open_ circuit, spurious _reflector and mouse_ bite, short, spur on the PCB, the defects are marked by a red rectangular frame, pictures are stored, and the date, time and place of detection are marked.

Compared with the prior art, the invention has the following beneficial effects:

1: the invention utilizes a K-means clustering algorithm to generate an initial Anchor Anchor frame specific to a homemade data set, and uses Mosaic+Mixup data enhancement to improve data and diversity.

2: according to the invention, the Ghostnet network replaces the residual structure of the YOLOv5 backbone network, so that the network parameters and the calculated amount are reduced while the feature extraction is optimized.

3: according to the invention, the main network of the YOLOv5 algorithm is improved, and the SE attention detection module is added, so that the detection accuracy of the model is improved.

4: the invention improves the loss function of the YOLOv5 algorithm, introduces the CIOU function, and solves the degradation problem of the original GIOU function.

5: according to the invention, the original FPN+PAN structure of YOLOv5 is improved, a weighted bidirectional feature network is introduced, a multi-level feature network is obtained by utilizing weighted feature fusion and cross-scale connection, the feature fusion among different scales is enhanced, and the detection precision is improved.

Drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.

Wherein like reference numerals refer to like elements throughout.

Further, if detailed description of the known art is not necessary to illustrate the features of the present invention, it will be omitted. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the drawings, and the words "inner" and "outer" refer to directions toward or away from, respectively, the geometric center of a particular component.

In the drawings:

FIG. 1Yolov5 network architecture;

FIG. 2 a backbond section of the Yolov5 network;

FIG. 3 shows an improved CSP1 structure;

FIG. 4BiFPN structure;

fig. 5PCB detection flow chart.

Detailed Description

The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.

Example 1

In step 1, 1000 PCB datasets containing defect information are obtained, each image has a size of 640 x 640, an image classification program is written, and training sets, verification sets and test sets are randomly allocated in a ratio of 7:2:1, so that randomness of the defect data is ensured. Anchor frame labeling is carried out on PCB images in a training set by using a Labelimg tool, and target classes are divided into six classes of missing_hole, open_ circuit, spurious _cap and mouse_ bite, short, spur, wherein the missing_hole label represents a leak hole, the open_circuit label represents an open circuit, the spilus_cap label represents pseudo copper, the mouse_bit label represents a mouse bite, the short label represents a short circuit and the spir label represents burrs. After all the images are marked, converting the generated xml format defect information into txt format;

preprocessing a data set by using a K-means clustering algorithm and Mosaic+Mixup data enhancement, wherein the initial anchor frame of Yolov5 is generated based on a COCO data set, so that the initial anchor frame is rich in target category and is not suitable for a PCB detection scene in the text, and the accuracy of target frame information can be reduced by using the initial prediction frame, therefore, the anchor frame suitable for the PCB defect detection scene needs to be clustered again on the PCB data set by adopting the K-means clustering algorithm;

the data enhancement method of Mosaic+Mixup amplifies the PCB data set. The basic flow of the Mosaic data enhancement is that firstly, 4 defective images with marking frames are selected, the defective images are subjected to operations such as overturning, zooming and color gamut changing, then the four images are respectively put on four images with gray background, finally, the four images are intercepted by utilizing a matrix mode and spliced into a new image, the new image has rich background and contains the marking frames on the original image. However, the Mosaic data enhancement can only enrich the background of the image and cannot increase the defect number, so the Mosaic+Mixup data enhancement strategy is used to enrich the defect number. According to the above, the defect part is extracted from the image enhanced by the Mosaic data, and then the data enhancement processing is performed on the defect part by using mix up to increase the number of defects on the image, so as to amplify the data set.

As described in step 2, it can be seen from fig. 1 that there are many layers of CBS basic Convolution blocks in the YOLOv5 network model, and the CBS Convolution blocks consist of Convolution layers (Conv), batch normalization layers (BN), and an activation function SiLu. However, too many convolution layers increase the calculation amount and the network parameter amount, so that a lightweight GhostNet network is introduced to replace a residual structure in a CSP1 structure in a Yolov5 backbone network, and the improved CSP1 is shown in FIG. 3. The GhostNet convolution module replaces a traditional convolution layer by adopting a mode of combining traditional convolution with a lightweight redundancy feature generator, so that network parameters and calculation amount are relatively less, and the GhostNet convolution module is easier to deploy to a terminal.

And adding an SE attention detection mechanism into the YOLOv5 network to solve the problem of loss caused by different weights of different channels of the feature map in the pooling process, adding an SE module into a Backbone part, learning the correlation among the channels, screening the attention to the channels, and effectively improving the detection precision.

The CIOU loss function is introduced to replace the original network GIOU loss function, and the CIOU considers the overlapping area, the center point distance and the length-width ratio, so that the regression of the target frame is more stable, and the convergence speed is faster.

As shown in step 3, in the convolutional neural network, feature maps obtained by convolutional layers of different parameters contain feature information of different targets. The feature map obtained after the deep convolution has higher resolution and mainly contains position information, but lacks semantic information, and the content obtained by the shallow convolution is just opposite to the former. The original YOLOv5 algorithm fuses the FPN and PAN in the neck layer in a bidirectional manner, and extracts information of different feature layers. Therefore, it is necessary to fuse feature information of the deep feature map and the shallow feature map. Too small a number of defects on the PCB may result in the omission of YOLOv5 extracted feature information. In order to strengthen feature fusion among different scales and improve detection precision, the invention introduces a weighted bidirectional feature network (bidirectional feature network), and as shown in fig. 4, a multi-level feature network is obtained by utilizing the weighted feature fusion and cross-scale connection. Global features of semantic information may enhance the accuracy of identifying small target defects.

As shown in step 4, the network loss function consists of two parts, regression loss and classification loss, respectively. Regression loss is also called coordinate position loss, and CIOU_loss is used as position loss. The classification loss comprises a classification loss and a confidence loss, and the classification loss and the confidence loss are evaluated by adopting the Focal_loss, and the calculation formulas are as follows:

CIOU _loss ＝1-CIOU

Focal _loss ＝-(1-p _t ) ^γ lgp _t

wherein IOU represents the intersection ratio of the predicted frame and the real frame, ρ ² (b，b ^gt ) Representing the Euclidean distance between the predicted frame of the object to be detected and the center point of the real frame, c representing the diagonal distance between the predicted frame of the object to be detected and the minimum circumscribed rectangle of the real frame, alpha being the weight coefficient, v representing the parameter for measuring the consistency of the length-width ratio,

representing the aspect ratio of the real frame, +.>

Representing aspect ratio of prediction frame, p _t Representing the predicted sample probability, γ is a constant parameter.

And 5, sending the PCB surface defect sample into the improved YOLOv5 network model for training to obtain the target detection optimal model.

And (6) taking sample data in the test set as input, performing target detection through the model improved in the step (4), outputting a target detection result, and evaluating the detection result.

In order to objectively evaluate the performance of the improved YOLOv5 model, a common precision (P), recall (R), average Precision (AP) and average precision average value mAP are adopted as evaluation indexes, and the calculation formulas of the indexes are as follows:

where TP represents the number of positive samples predicted in the true positive samples, FP represents the number of positive samples predicted in the true negative samples, FN represents the number of negative samples predicted in the true positive samples, k represents the total number of categories detected, AP (i) represents the AP value of the ith category, and ap=map, in particular when k=1.

In the description of the present invention, it should be understood that the terms "coaxial," "bottom," "one end," "top," "middle," "another end," "upper," "one side," "top," "inner," "front," "center," "two ends," etc. indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.

Furthermore, the terms "first," "second," "third," "fourth," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, whereby features defining "first," "second," "third," "fourth" may explicitly or implicitly include at least one such feature.

In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "configured," "connected," "secured," "screwed," and the like are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly through intermediaries, or in communication with each other or in interaction with each other, unless explicitly defined otherwise, the meaning of the terms described above in this application will be understood by those of ordinary skill in the art in view of the specific circumstances.

Finally, it should be noted that: the above is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that the present invention is described in detail with reference to the foregoing embodiments, and modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (4)

1. A PCB surface defect detection method based on an improved YOLOv5 algorithm is characterized by comprising the following steps:

step 1: and acquiring a disclosed data set, which is used for detecting surface defects of the PCB, marking pictures in the data set by using a Lambimg tool, generating a label in a YOLO format, and dividing the data set into a training set, a testing set and a verification set. Generating initial Anchor frame parameters which are most in line with the characteristics of a data set by using a K-means clustering algorithm, increasing the diversity of the data set by using a Mosaic+Mixup data enhancement mode, and improving the detection effect of a small target;

step 2: a PCB detection model based on YOLOv5 is established, a residual structure in a CSP1 module in a lightweight network GhostNet instead of the YOLOv5 Backbone network is introduced, a SE attention mechanism is introduced, a CIOU loss function is introduced to replace an original loss function GIOU, and the YOLOv5 network Backbone part is improved;

step 3: introducing a weighted bidirectional feature network, obtaining a multi-level feature network by utilizing weighted feature fusion and trans-scale connection, and improving the original FPN+PAN structure of YOLOv 5;

step 4: and determining a network loss function and a performance evaluation index of the YOLOv5 algorithm according to the CIOU, wherein the network loss function consists of two parts, namely regression loss and classification loss. Regression loss is also called coordinate position loss, and CIOU_loss is used as position loss. Classification loss comprises class loss and confidence loss, and the class loss and the confidence loss are evaluated by adopting Focal_loss;

step 5: sending the PCB surface defect sample into an improved YOLOv5 network model for training to obtain a target detection optimal model;

step 6: and 5, taking sample data in the test set as input, carrying out target detection through the model improved in the step 5, outputting a target detection result, and evaluating the detection result.

2. The method for detecting surface defects of a PCB based on the modified YOLOv5 algorithm of claim 1, wherein the step 1 comprises the steps of:

step 1.1: 1000 PCB data sets containing defect information are obtained, the size of each image is 640 x 640, an image classification program is compiled, training sets, verification sets and test sets are randomly distributed in a ratio of 7:2:1, and randomness of the defect data is guaranteed;

step 1.2: anchor frame marking is carried out on PCB images in a training set by using a Labelimg tool, and target classes are divided into six classes of missing_hole, open_ circuit, spurious _cap and mouse_ bite, short, spur, wherein the missing_hole label represents a leak hole, the open_circuit label represents an open circuit, the spilus_cap label represents pseudo copper, the mouse_bit label represents a mouse bite, the short label represents a short circuit and the spike label represents burrs;

step 1.3: and after all the images are marked, converting the generated xml format defect information into txt format.

3. The method for detecting surface defects of a PCB based on the modified YOLOv5 algorithm of claim 1, wherein the step 2 comprises the steps of:

step 2.1: the extraction module used by the backbox part in the YOLOv5 network is C3, the module is formed by stacking a Bottleneck module and a CBS convolution block, and the calculated amount and the network parameter number are increased due to too many convolution layers, so that a lightweight network GhostNet is introduced to replace a residual structure in a CSP1 module in the YOLOv5 Backbone network;

step 2.2: adding an SE attention detection mechanism into the YOLOv5 network to solve the problem of loss caused by different weights of different channels of the feature map in the pooling process, adding an SE module into a Backbone part, learning the correlation among the channels, screening out the attention to the channels, and effectively improving the detection precision;

step 2.3: the CIOU loss function is introduced to replace the original network GIOU loss function, and the CIOU considers the overlapping area, the center point distance and the length-width ratio, so that the regression of the target frame is more stable, and the convergence speed is faster.

4. The method for detecting surface defects of a PCB based on the modified YOLOv5 algorithm of claim 1, wherein the step 6 comprises the steps of:

step 6.1: if the model detects six defects such as missing_hole, open_ circuit, spurious _reflector and mouse_ bite, short, spur on the PCB, the defects are marked by a red rectangular frame, pictures are stored, and the date, time and place of detection are marked.

Images