CN117094972A - Lightweight PCB defect detection method and system based on high-order spatial interaction - Google Patents

Lightweight PCB defect detection method and system based on high-order spatial interaction Download PDF

Info

Publication number
CN117094972A
CN117094972A CN202311063487.3A CN202311063487A CN117094972A CN 117094972 A CN117094972 A CN 117094972A CN 202311063487 A CN202311063487 A CN 202311063487A CN 117094972 A CN117094972 A CN 117094972A
Authority
CN
China
Prior art keywords
pcb
convolution
model
defect detection
lightweight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311063487.3A
Other languages
Chinese (zh)
Inventor
续欣莹
魏嘉敏
谢珺
李鹏越
张文杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Taiyuan University of Technology
Original Assignee
Taiyuan University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Taiyuan University of Technology filed Critical Taiyuan University of Technology
Priority to CN202311063487.3A priority Critical patent/CN117094972A/en
Publication of CN117094972A publication Critical patent/CN117094972A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30141—Printed circuit board [PCB]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of PCB defect detection, in particular to a lightweight PCB defect detection method and system based on high-order spatial interaction, which solve the technical problems in the background art. The system comprises a data acquisition module, a parameter adjustment module, a PCB defect detection model and a result display module. The invention improves the interaction capability, the expression capability of the characteristics and the multi-scale perception capability of the high-order space of the PCB defect detection model, effectively recovers the detail information of the characteristic diagram and improves the accuracy of target detection, and simultaneously reduces the calculated amount and the parameter number of the PCB defect detection model. The PCB defect detection system can intuitively display the detection result and accuracy, so that the detection is quicker and more efficient, the operability of the PCB defect detection is improved, and the working efficiency is improved.

Description

Lightweight PCB defect detection method and system based on high-order spatial interaction
Technical Field
The invention relates to the technical field of PCB defect detection, in particular to a lightweight PCB defect detection method and system based on high-order spatial interaction.
Background
The printed circuit board (Printed circuit boards, PCB) is an important component of the electronic product, and its quality directly affects the performance and reliability of the electronic product, but various defects such as holes, mouse bites, open circuits, short circuits, burrs, residual copper, etc. may be generated in the manufacturing process of the PCB, so that effective defect detection of the PCB is a key step for improving the quality of the product and reducing the cost.
Early researches on PCB defect detection are mainly based on traditional detection technology and machine learning technology, however, the traditional PCB defect detection technology and the machine learning technology are not suitable for the current development trend, and have the problems of low efficiency, low accuracy, susceptibility to subjective factors, difficulty in processing complex and changeable defect scenes and the like. The progress of the deep learning technology provides a new thought and method for detecting the defects of the PCB, and the target detection network based on the deep learning can rapidly, accurately and stably detect various defects on the PCB. The main current target detection schemes are Two types, namely Two types of Two-stage detection and One-stage detection. Two-stage is a detection algorithm based on candidate frames, and is divided into Two stages: the first stage is to generate regions that may contain objects, and the second stage is to classify objects in these regions. RCNN and Faster R-CNN are typical Two-stage algorithm, which has high accuracy but still has poor performance in terms of detection speed and small target detection problem. The One-stage algorithm has only One stage, does not need to generate and classify candidate areas, directly outputs the class probability and the position coordinate value of the object, can obtain a final detection result by One-time detection, and has the advantage of high detection speed. YOLO series, SSD, etc. are representative algorithms. Currently, deep learning-based methods have made some progress in PCB defect detection, but it still has some challenges, such as too small a data set, interference with small objects and complex backgrounds, real-time requirements, etc. Thus, PCB defect detection has great development space and potential in industrial applications.
Disclosure of Invention
The method aims to solve the problem that the existing algorithm still has poor detection speed and small target detection; the invention provides a lightweight PCB defect detection method and system based on high-order spatial interaction.
The invention discloses a lightweight PCB defect detection method based on high-order space interaction, which comprises the following steps:
s1, acquiring a PCB defect data set, performing data enhancement operation on the data set, completing data preprocessing, and dividing a training set, a verification set and a test set according to a proportion;
s2, constructing a lightweight PCB defect detection model based on high-order spatial interaction, namely a YOLOv5-HorL model, wherein the YOLOv5-HorL model is formed by introducing a HorFPN module, a lightweight CARAFE up-sampling operator and a lightweight convolution GhostConv on the basis of the YOLOv5 model;
s3, training the YOLOv5-HorL model through a training set, adjusting super parameters, calculating an average precision mean value of the adjusted YOLOv5-HorL model on a verification set, judging whether the mAP of the current model is optimal, and if so, storing the current model as an optimal PCB defect detection model;
s4, inputting the test set into an optimal PCB defect detection model to obtain the defect position, the defect type and the corresponding detection accuracy of the PCB image to be detected.
In the lightweight PCB defect detection method based on high-order spatial interaction, a YOLOv5-HorL model is applied to a PCB image to generate a defect detection result. The step S1 data preprocessing uses a series of data enhancement operations to expand the data set of the PCB defect data set so as to avoid the problem of over fitting caused by insufficient data quantity. The HorFPN module can realize high-efficiency high-order space interaction and improve the performance of a target detection task; the lightweight CARAFE upsampling operator predicts the upsampling weight of each position by using a small convolution network, and then uses the weights to weight and combine the surrounding features, thereby realizing the upsampling of the feature map; lightweight convolution, ghostConv, is a decomposition of a standard convolution kernel into two parts: a main convolution kernel and some simple linear transformations. The main convolution kernel is responsible for extracting the main information of the input features, while the linear transformation is used to generate more feature maps, thereby enhancing the expressive power of the model. In this way, a lightweight convolution GhostConv can achieve the same or similar effect as a standard convolution with less computational resources. And introducing a HorFPN module, a lightweight CARAFE up-sampling operator and a lightweight convolution GhostConv into the network of the YOLOv5-HorL model, improving the interaction capacity of a model high-order space, the expression capacity of features and the multiscale perceptibility, effectively recovering the detail information of the feature map, improving the accuracy of target detection, and reducing the calculation amount and the parameter number of the model. Therefore, the detection cost and the false detection rate can be reduced, and the accuracy of PCB defect detection is improved.
Preferably, in step S1, the data enhancement operation includes random flipping, random rotation, and HSV adjustment. Random flipping is to randomly select a direction on the original image, then flip it along that direction as a new image, and adjust the direction and position of the corresponding annotation frame. The detection capability of the model for targets with different symmetries and viewing angles is increased. The random rotation is to randomly select an angle on the original image, then rotate the original image by a certain angle as a new image, and adjust the angle and the position of the corresponding labeling frame. The detection capability of the model on targets in different directions and postures is increased. HSV adjustment is to randomly select some parameters on the original image and then transform it into some color space as a new image. The detection capability of the model for targets with different illumination conditions and background colors is increased.
Preferably, in step S1, the PCB defect dataset contains six defects of hole, rat bite, open circuit, short circuit, burr and residual copper, about 1-5 defects per picture, and the number of each defect is balanced. And the PCB defect dataset contains PCBs of various sizes, a maximum of 12.5cm by 12cm, and a minimum of 5.3cm by 4.8cm.
Preferably, in step S2, the construction step of the YOLOv5-HorL model is as follows:
s201, constructing a PCB defect detection model based on a YOLOv5 algorithm;
s202, introducing a HorFPN module to replace an FPN module in the Yolov5 algorithm, introducing a CARAFE upsampling operator to replace a default upsampling mode in the Yolov5 algorithm, and introducing a light convolution GhostConv to replace a common convolution in the Yolov5 algorithm. The HorFPN module replaces the FPN module to be used for spatial convolution of feature fusion, so that efficient high-order spatial interaction is achieved, and performance of a target detection task is improved.
Preferably, in step S2, the HorFPN module is a recursive gated convolution g n Conv modified; recursive gated convolution g n Conv uses gating convolution to achieve input adaptation and higher order spatial interaction, while using a large convolution kernel or global filter to achieve large-scale spatial interaction, recursive gating convolution g n The operation of Conv comprises the following steps:
a) For input features x, g n Conv projects it into n+1 subspaces p 0 And q k (k＝0,1,...,n-1)；
b) Performing gated convolution circularly to obtain p k+1 (k＝0,1,...,n-1)；
c) All p k (k=0, 1,., n) are stitched together and subjected to a linear projection to obtain an output y;
wherein p and q are projection features, and the angle marks are subspaces.
Preferably, in step S2, the CARAFE upsampling operator predicts the upsampling weight of each position by using the convolutional network, and then uses the obtained upsampling weights to weight and combine the surrounding features, so as to implement upsampling of the feature map, and the operational procedure of the CARAFE upsampling operator is as follows:
W l′ ＝Ψ(N(X l ,k encoder )) (1)
X' l′ ＝φ(N(X l ,k up ),W l′ ) (2)
in formulas (1) and (2), ψ is an upsampling kernel prediction module, φ is a content aware reorganization module, kernel W l′ To upsample kernel, X l ' ′ N (X) is the output characteristic diagram after content recombination l K) represents X in the feature map l Neighborhood of k encoder For the size of the encoder convolution kernel, k up Is the size of the upsampled kernel.
Preferably, in step S2, the lightweight convolution GhostConv uses a group of low-rank convolution kernels as a complete convolution kernel, so as to accelerate the convolution operation, and the operation procedure of the lightweight convolution GhostConv is as follows:
Y＝[F m *X,F c *(F m *X)] (3)
in the formula (3), X is an input feature map, F m Filter as main convolution layer, F c Filter for linear transformation, which represents convolution operation, [ ·, ]]Representing stitching of two feature maps along the channel dimension.
Preferably, in step S3, the training process uses an SGD optimizer, and the loss functions include a classification loss function, a regression loss function, and a confidence loss function, where the classification loss function and the confidence loss function are calculated using a cross entropy loss function, and the regression loss function is calculated using a GIOU loss function. In actual training, the weight attenuation is 0.0005, the initial learning rate is set to be 0.01, the minimum learning rate is set to be 0.00008, and the model is saved after the model converges.
Preferably, in step S4, the evaluation indexes used in the accuracy detection process include accuracy rate, recall rate, average accuracy mean value, calculated amount and parameter number. The accuracy and recall are indicators of the ability of the classifier to identify the sample from different angles. The accuracy rate represents how many of the data predicted as positive samples are true positive samples, i.e., the "find pairs" ratio; the recall indicates how many of the true positive samples are predicted to be positive samples, i.e., the proportion of "find full". The accuracy and recall rate reflect the performance of the network from two levels respectively, and the accuracy of the network can be evaluated more effectively by integrating the two indexes. Average accuracy is typically used to integrate the two indices. The average accuracy mean is the average of all classes of APs. The larger mAP is, the better the detection algorithm can recognize different kinds of objects. mAP is generally used as the main index for comparison by the target detection algorithm. The calculated quantity and the parameter quantity are respectively used for measuring the time complexity and the space complexity of the model. The calculated amount refers to the floating point operation times per second, and the parameter amount refers to the total number of parameters required to be trained in the network model.
The invention also discloses a lightweight PCB defect detection system based on high-order space interaction, which comprises a data acquisition module, a parameter adjustment module, a PCB defect detection model and a result display module; the data acquisition module is used for acquiring PCB image data; the parameter adjusting module suppresses the IOU and the confidence coefficient by adjusting the non-maximum value so as to meet different detection requirements of users; the PCB defect detection model is constructed based on a YOLOv5-HorL model in the lightweight PCB defect detection method based on high-order spatial interaction, and a PCB image is input into the PCB defect detection model for defect detection; the result display module is used for displaying the type, accuracy, size and position indexes of the PCB defect, generating a detection result and displaying the detection result to a user, and finally automatically storing the detection result. The system operates in a real-time manner and automatically generates a detection result in real time.
Compared with the prior art, the technical scheme provided by the invention has the following advantages: according to the lightweight PCB defect detection method based on high-order space interaction, a HorFPN module, a lightweight CARAFE up-sampling operator and a lightweight convolution GhostConv are introduced into a YOLOv5-HorL network, so that the interaction capacity, the feature expression capacity and the multi-scale perception capacity of a high-order space of a PCB defect detection model are improved, the detail information of the feature map is effectively recovered, the accuracy of target detection is improved, and meanwhile, the calculated amount and the parameter number of the PCB defect detection model are reduced. Therefore, the detection cost and the false detection rate can be reduced, and the accuracy of PCB defect detection is improved. The lightweight PCB defect detection method and system based on the high-order space interaction can intuitively display the detection result and accuracy, so that the detection is quicker and more efficient, the operability of PCB defect detection is greatly improved, and the working efficiency is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a general flow chart of a lightweight PCB defect detection method based on high-order spatial interaction provided in an embodiment of the present invention;
FIG. 2 is a network structure diagram of a YOLOv5-HorL model in a specific implementation of a lightweight PCB defect detection method based on high-order spatial interaction provided in an embodiment of the present invention;
fig. 3 is a diagram illustrating g in a specific implementation of a lightweight PCB defect detection method based on high-order spatial interaction according to an embodiment of the present invention n A structure diagram of Conv recursive gating convolution;
fig. 4 is a block diagram of a CARAFE upsampling operator in a specific implementation of a lightweight PCB defect detection method based on higher-order spatial interaction according to an embodiment of the present invention;
fig. 5 is a structural diagram of a light-weighted convolution of a GhostConv in a specific implementation of a light-weighted PCB defect detection method based on high-order spatial interaction according to an embodiment of the present invention;
fig. 6 is a schematic block diagram of a lightweight PCB defect detection system based on high-order spatial interaction according to an embodiment of the present invention;
fig. 7 is a detailed block diagram of a lightweight PCB defect detection system based on high-order spatial interaction according to an embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the invention will be more clearly understood, a further description of the invention will be made. It should be noted that, without conflict, the embodiments of the present invention and features in the embodiments may be combined with each other.
In the description, it should be noted that the terms "first," "second," and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. It should be noted that, unless explicitly stated or limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms described above will be understood by those of ordinary skill in the art as the case may be.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the invention.
Specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
In one embodiment, as shown in fig. 1-7, a lightweight PCB defect detection method based on high-order spatial interaction, as shown in fig. 1, comprises the steps of:
s1, acquiring a PCB defect data set, performing data enhancement operation on the data set, completing data preprocessing, and dividing a training set, a verification set and a test set according to a proportion;
s2, constructing a lightweight PCB defect detection model based on high-order spatial interaction, namely a YOLOv5-HorL model, wherein the YOLOv5-HorL model is formed by introducing a HorFPN module, a lightweight CARAFE up-sampling operator and a lightweight convolution GhostConv on the basis of the YOLOv5 model;
s3, training the YOLOv5-HorL model through a training set, adjusting super parameters, calculating an average precision mean value of the adjusted YOLOv5-HorL model on a verification set, judging whether the mAP of the current model is optimal, and if so, storing the current model as an optimal PCB defect detection model;
s4, inputting the test set into an optimal PCB defect detection model to obtain the defect position, the defect type and the corresponding detection accuracy of the PCB image to be detected.
In the lightweight PCB defect detection method based on high-order spatial interaction, a YOLOv5-HorL model is applied to a PCB image to generate a defect detection result. In the specific embodiment, in step S1, 693 images suitable for the detection task are selected, and a series of data enhancement operations are used for data preprocessing, so as to expand the data set of the PCB defect data set, so as to avoid the problem of overfitting caused by insufficient data volume. The HorFPN module can realize high-efficiency high-order space interaction and improve the performance of a target detection task; the lightweight CARAFE upsampling operator predicts the upsampling weight of each position by using a small convolution network, and then uses the weights to weight and combine the surrounding features, thereby realizing the upsampling of the feature map; lightweight convolution, ghostConv, is a decomposition of a standard convolution kernel into two parts: a main convolution kernel and some simple linear transformations. The main convolution kernel is responsible for extracting the main information of the input features, while the linear transformation is used to generate more feature maps, thereby enhancing the expressive power of the model. In this way, a lightweight convolution GhostConv can achieve the same or similar effect as a standard convolution with less computational resources. And introducing a HorFPN module, a lightweight CARAFE up-sampling operator and a lightweight convolution GhostConv into the network of the YOLOv5-HorL model, improving the interaction capacity of a model high-order space, the expression capacity of features and the multiscale perceptibility, effectively recovering the detail information of the feature map, improving the accuracy of target detection, and reducing the calculation amount and the parameter number of the model. Therefore, the detection cost and the false detection rate can be reduced, and the accuracy of PCB defect detection is improved.
In this embodiment, the basic operation principle of the YOLOv5 model is to divide an input image into small lattices, and each lattice is responsible for predicting a certain number of bounding boxes and class probabilities. Then, the prediction of each lattice is subjected to feature extraction and classification through a Convolutional Neural Network (CNN), and finally, the final detection result is screened through non-maximum suppression (NMS). The structure of the YOLOv5 model is mainly divided into the following three parts:
1. the Backbone network (Backbone) employs a CSPDarknet, which is mainly composed of a Focus network structure and a CSPNet network structure. Firstly, carrying out feature extraction on an input picture through a Focus network, adjusting the number of expansion channels, outputting three effective feature layers with different scales through four residual error structures formed by CSPNet, and finally adding an SPP structure to improve the receptive field of the network;
2. the Neck network (Neck) adopts the combination of FPN (Feature Pyramid Network) and PAN (Path Aggregation Network) to form a bidirectional characteristic pyramid network, so that the characteristics of each scale can obtain richer and more balanced information;
3. the Prediction network (Prediction) adopts a design similar to YOLOv3 and YOLOv4, uses a convolution layer to receive feature maps from different scales, and outputs a four-dimensional tensor to represent the Prediction result of each grid unit.
In a specific embodiment, the training process of the PCB defect detection model in step S3 in the method includes the following steps:
s301, randomly dividing a PCB defect data set into a training set, a verification set and a test set according to the proportion of 8:1:1, and ensuring that the defect types and the defect quantity in each data set are uniformly distributed;
s302, loading a YOLOv5-HorL model, and initializing super parameters, wherein the super parameters comprise: weight attenuation coefficient, learning rate, training batch size, iteration round number and coefficient value of each partial loss function;
s303, after loading a network model, carrying out feature extraction and defect positioning and classification on an input training set image;
s304, training each round, calculating a loss function by the network, and then carrying out parameter optimization by adopting an SGD optimizer;
s305, calculating an Average Precision (mAP) of the network model on the verification set, judging whether the mAP of the model is optimal, and if so, storing the model.
Based on the above embodiments, in a preferred embodiment, in step S1, the data enhancing operation includes random flipping, random rotation, and HSV adjustment. Random flipping is to randomly select a direction on the original image, then flip it along that direction as a new image, and adjust the direction and position of the corresponding annotation frame. The detection capability of the model for targets with different symmetries and viewing angles is increased. The random rotation is to randomly select an angle on the original image, then rotate the original image by a certain angle as a new image, and adjust the angle and the position of the corresponding labeling frame. The detection capability of the model on targets in different directions and postures is increased. HSV adjustment is to randomly select some parameters on the original image and then transform it into some color space as a new image. The detection capability of the model for targets with different illumination conditions and background colors is increased.
Based on the above embodiments, in a preferred embodiment, in step S1, the PCB defect dataset includes six defects of hole missing, rat bite, open circuit, short circuit, burr and residual copper, and each picture has about 1-5 defects, and the number of each defect is balanced. And the PCB defect dataset contains PCBs of various sizes, a maximum of 12.5cm by 12cm, and a minimum of 5.3cm by 4.8cm.
Based on the above embodiments, in a preferred embodiment, in step S2, the structure of the YOLOv5-HorL model is as shown in fig. 2, and the construction steps of the YOLOv5-HorL model are as follows:
s201, constructing a PCB defect detection model based on a YOLOv5 algorithm;
s202, introducing a HorFPN module to replace an FPN module in the Yolov5 algorithm, introducing a CARAFE upsampling operator to replace a default upsampling mode in the Yolov5 algorithm, and introducing a light convolution GhostConv to replace a common convolution in the Yolov5 algorithm. The HorFPN module replaces the FPN module to be used for spatial convolution of feature fusion, so that efficient high-order spatial interaction is achieved, and performance of a target detection task is improved.
Based on the above embodiment, in a preferred embodiment, in step S2, H is as shown in FIG. 3The orFPN module is based on recursive gating convolution g proposed in HorNet n The Conv is improved to replace the space convolution used for feature fusion in the FPN so as to realize high-efficiency high-order space interaction and improve the performance of a target detection task; recursive gated convolution g n Conv uses gating convolution to achieve input adaptation and higher order spatial interaction, while using a large convolution kernel or global filter to achieve large-scale spatial interaction, recursive gating convolution g n The operation of Conv comprises the following steps:
a) For input features x, g n Conv projects it into n+1 subspaces p 0 And q k (k＝0,1,...,n-1)；
b) Performing gated convolution circularly to obtain p k+1 (k＝0,1,...,n-1)；
c) All p k (k=0, 1,., n) are stitched together and subjected to a linear projection to obtain an output y;
wherein p and q are projection features, and the angle marks are subspaces.
Based on the above embodiment, in a preferred embodiment, in step S2, as shown in fig. 4, the CARAFE upsampling operator predicts the upsampling weight of each position by using the convolutional network, and then weights the feature around the combination with the obtained upsampling weight, so as to implement upsampling of the feature map, where the operation procedure of the CARAFE upsampling operator is as follows:
W l′ ＝Ψ(N(X l ,k encoder )) (1)
X' l′ ＝φ(N(X l ,k up ),W l′ ) (2)
in formulas (1) and (2), ψ is an upsampling kernel prediction module, φ is a content aware reorganization module, kernel W l′ To upsample the kernel, X' l′ N (X) is the output characteristic diagram after content recombination l K) represents X in the feature map l Neighborhood of k encoder For the size of the encoder convolution kernel, k up Is the size of the upsampled kernel.
Based on the above embodiment, in a preferred embodiment, in step S2, as shown in fig. 5, the lightweight convolution GhostConv is to decompose a standard convolution kernel into two parts: a main convolution kernel and some simple linear transformations. The main convolution kernel is responsible for extracting the main information of the input features, while the linear transformation is used to generate more feature maps, thereby enhancing the expressive power of the model. In this way, a GhostConv can achieve the same or similar effect as a standard convolution with less computational resources. The lightweight convolution GhostConv takes a group of low-rank convolution kernels as a complete convolution kernel, so that the acceleration of convolution operation is realized, and the operation process of the lightweight convolution GhostConv is as follows:
Y＝[F m *X,F c *(F m *X)] (3)
in the formula (3), X is an input feature map, F m Filter as main convolution layer, F c Filter for linear transformation, which represents convolution operation, [ ·, ]]Representing stitching of two feature maps along the channel dimension.
Based on the above embodiments, in a preferred embodiment, in step S3, the training process uses an SGD optimizer, and the loss functions include a classification loss function, a regression loss function, and a confidence loss function, where the classification loss function and the confidence loss function are calculated using a cross entropy loss function, and the regression loss function is calculated using a GIOU loss function. In actual training, the weight attenuation is 0.0005, the initial learning rate is set to be 0.01, the minimum learning rate is set to be 0.00008, and the model is saved after the model converges.
On the basis of the above embodiment, in a preferred embodiment, in step S4, the evaluation indexes used in the accuracy detection process include accuracy (Precision), recall (Recall), mean Average-accuracy (mAP), calculated amount (flow), and parameter amount (parameters).
Wherein the precision and recall are indicators of the ability of the classifier to identify the sample from different angles. The accuracy rate represents how many of the data predicted as positive samples are true positive samples, i.e., the "find pairs" ratio; the recall indicates how many of the true positive samples are predicted to be positive samples, i.e., the proportion of "find full". The calculation formulas of the Precision and Recall rate (Precision) are respectively:
in formulas (4) and (5), TP represents a true case, i.e., predicted as a positive sample and identified correctly; TN represents a true negative example, i.e., predicted as a negative sample and correctly identified; FP represents a false positive case, i.e. predicted as a positive sample but identified an error; FN represents a false negative example, i.e. predicted as a negative sample but identifying an error. The accuracy and recall rate reflect the performance of the network from two levels respectively, and the accuracy of the network can be evaluated more effectively by integrating the two indexes. Average Accuracy (AP) is typically used to integrate the two metrics.
The average accuracy mean is the average of all classes of APs. The larger mAP is, the better the detection algorithm can recognize different kinds of objects. mAP is usually used as a main index for comparison of a target detection algorithm, and the average precision mean value has a calculation formula as follows:
in the formula (6), ap (j) represents the average accuracy of the jth class target, and M represents the total number of classes. The Average Precision (AP) represents the average of the precision of a certain class at different recall rates, and the average precision average (mAP) is the average of the Average Precision (AP) of all classes.
The calculated quantity and the parameter quantity are respectively used for measuring the time complexity and the space complexity of the model. The calculated amount refers to the floating point operation times per second, and the parameter amount refers to the total number of parameters required to be trained in the network model.
Table 1 index contrast of different target detection algorithms on PCB datasets
As shown in table 1, to verify the superiority of the method of the present invention, mAP, precision, recall, calculated amount and parameter amount comparisons of different algorithms were performed under the same experimental environment configuration. The results show that mAP of the YOLOv5-HorL is up to 96.26%, which is respectively increased by 43.36%, 5.95%, 14.99%, 1.95% and 4.99% compared with that of Faster R-CNN, SSD, YOLOv3, YOLOv4 and YOLOv5 s. Simultaneously Precision, recall is 97.90 percent and 91.84 percent respectively, which is greatly improved compared with the fast R-CNN, SSD, YOLOv, the Yolov4 and the Yolov5 s. The calculation amount and the parameter amount of the YOLOv5-HorL are also reduced to a certain extent, the YOLOv5s is further lightened, and higher detection precision is realized. The experimental result proves that the invention has the effectiveness and has more practical value.
The invention also discloses a lightweight PCB defect detection system based on high-order space interaction in a certain embodiment, which comprises a data acquisition module, a parameter adjustment module, a PCB defect detection model and a result display module, as shown in fig. 6 and 7; the data acquisition module is used for acquiring PCB image data; the parameter adjusting module suppresses the IOU and the confidence coefficient by adjusting the non-maximum value so as to meet different detection requirements of users; the PCB defect detection model is constructed based on a YOLOv5-HorL model in the lightweight PCB defect detection method based on high-order spatial interaction, and a PCB image is input into the PCB defect detection model for defect detection; the result display module is used for displaying the type, accuracy, size and position indexes of the PCB defect, generating a detection result and displaying the detection result to a user, and finally automatically storing the detection result. The system operates in a real-time manner and automatically generates a detection result in real time. The system is based on a processor, a memory and a computer program, and the lightweight PCB defect detection system is created by the computer program, wherein in the lightweight PCB defect detection system, a YOLOv5-HorL model is used for detecting a PCB image in real time and a detection result is given.
After the data acquisition module acquires the PCB image data, the image data is preprocessed, wherein the preprocessing comprises the data enhancement processing of the input image so as to reduce the over-fitting phenomenon; and carrying out normalization processing on the input image. The PCB image is generated by shooting the PCB by using an image collector, can be an image or video shot in real time, and can also be an image stored in a local server or cloud, and is acquired by reading or network transmission. The parameter adjusting module is used for adjusting the non-maximum suppression IOU and the Confidence coefficient Confidence so as to meet different detection requirements of users. In some embodiments, the non-maximum inhibitory IOU and Confidence adjustment range is 0-1. The result display module is used for receiving the detection result, automatically determining qualitative and quantitative indexes such as the type, the accuracy, the size and the position of the PCB defect and displaying the qualitative and quantitative indexes to a user, and finally automatically storing the detection result. The type is PCB defects obtained through detection, including holes, mouse bites, open circuits, short circuits, burrs and residual copper. In some embodiments, the automatically saved test result is a textual display of the quantitative qualitative data described above.
According to the lightweight PCB defect detection method and system based on high-order space interaction, firstly, data preprocessing is carried out on a PCB image to avoid the problem of over fitting caused by insufficient data quantity, then the processed PCB image is input into a PCB defect detection model to carry out defect detection, and finally, a detection result is automatically obtained; the accuracy and efficiency of PCB defect detection are effectively improved, and the intelligent level of defect detection is effectively improved.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Although described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the embodiments, and they should be construed as covering the scope of the appended claims.

Claims (10)

1. A lightweight PCB defect detection method based on high-order spatial interaction is characterized by comprising the following steps:
s1, acquiring a PCB defect data set, performing data enhancement operation on the data set, completing data preprocessing, and dividing a training set, a verification set and a test set according to a proportion;
s2, constructing a lightweight PCB defect detection model based on high-order spatial interaction, namely a YOLOv5-HorL model, wherein the YOLOv5-HorL model is formed by introducing a HorFPN module, a lightweight CARAFE up-sampling operator and a lightweight convolution GhostConv on the basis of the YOLOv5 model;
s3, training the YOLOv5-HorL model through a training set, adjusting super parameters, calculating an average precision mean value of the adjusted YOLOv5-HorL model on a verification set, judging whether the mAP of the current model is optimal, and if so, storing the current model as an optimal PCB defect detection model;
s4, inputting the test set into an optimal PCB defect detection model to obtain the defect position, the defect type and the corresponding detection accuracy of the PCB image to be detected.
2. The method for detecting defects of a lightweight PCB based on high-order spatial interaction of claim 1, wherein in step S1, the data enhancement operation includes random flipping, random rotation, and HSV adjustment.
3. The method for detecting defects of a lightweight PCB based on higher-order spatial interaction of claim 2, wherein in step S1, the PCB defect dataset comprises six defects of hole, mouse bite, open circuit, short circuit, burr and residual copper, each of which has about 1-5 defects, and the number of each defect is balanced.
4. The method for detecting defects of a lightweight PCB based on higher-order spatial interaction of claim 1, wherein in step S2, the construction of the YOLOv5-HorL model comprises the steps of:
s201, constructing a PCB defect detection model based on a YOLOv5 algorithm;
s202, introducing a HorFPN module to replace an FPN module in the Yolov5 algorithm, introducing a CARAFE upsampling operator to replace a default upsampling mode in the Yolov5 algorithm, and introducing a light convolution GhostConv to replace a common convolution in the Yolov5 algorithm.
5. The method for detecting defects of a lightweight PCB based on higher-order spatial interaction of claim 4, wherein in step S2, the HorFPN module is based on a recursive gating convolution g proposed in HorNet n Conv modified; recursive gated convolution g n Conv uses gating convolution to achieve input adaptation and higher order spatial interaction, while using a large convolution kernel or global filter to achieve large-scale spatial interaction, recursive gating convolution g n The operation of Conv comprises the following steps:
a) For input features x, g n Conv projects it into n+1 subspaces p 0 And q k (k＝0,1,...,n-1)；
b) Performing gated convolution circularly to obtain p k+1 (k＝0,1,...,n-1)；
c) All p k (k=0, 1,., n) are stitched together and subjected to a linear projection to obtain an output y;
wherein p and q are projection features, and the angle marks are subspaces.
6. The method for detecting defects of a lightweight PCB based on higher-order spatial interaction of claim 4, wherein in step S2, a CARAFE upsampling operator predicts upsampling weights of each position by using a convolutional network, and then weights and combines surrounding features by using the obtained upsampling weights, thereby implementing upsampling of a feature map, and an operation procedure of the CARAFE upsampling operator is as follows:
W l′ ＝Ψ(N(X l ,k encoder )) (1)
X l ' ′ ＝φ(N(X l ,k up ),W l′ ) (2)
in formulas (1) and (2), ψ is an upsampling kernel prediction module, φ is a content aware reorganization module, kernel W l′ To upsample kernel, X l ' ′ N (X) is the output characteristic diagram after content recombination l K) represents X in the feature map l Neighborhood of k encoder For the size of the encoder convolution kernel, k up Is the size of the upsampled kernel.
7. The method for detecting defects of a lightweight PCB based on higher-order spatial interaction of claim 4, wherein in step S2, a set of low-rank convolution kernels are used as a complete convolution kernel, so that acceleration of convolution operation is achieved, and an operation procedure of the lightweight convolution GhostConv is as follows:
Y＝[F m *X,F c *(F m *X)] (3)
in the formula (3), X is an input feature map, F m Filter as main convolution layer, F c Filter for linear transformation, which represents convolution operation, [ ·, ]]Representing stitching of two feature maps along the channel dimension.
8. The method according to claim 4-7, wherein in step S3, the training process uses an SGD optimizer, the loss functions include a classification loss function, a regression loss function and a confidence loss function, the classification loss function and the confidence loss function are calculated using a cross entropy loss function, and the regression loss function is calculated using a GIOU loss function.
9. The method for detecting defects of a lightweight PCB based on higher-order spatial interaction of claim 8, wherein in step S4, the evaluation indexes used in the accuracy detection process include accuracy, recall, average accuracy mean, calculated amount and parameter amount.
10. The lightweight PCB defect detection system based on high-order space interaction is characterized by comprising a data acquisition module, a parameter adjustment module, a PCB defect detection model and a result display module; the data acquisition module is used for acquiring PCB image data; the parameter adjusting module suppresses the IOU and the confidence coefficient by adjusting the non-maximum value so as to meet different detection requirements of users; the PCB defect detection model is constructed based on a YOLOv5-HorL model in the lightweight PCB defect detection method based on high-order spatial interaction as set forth in claim 4, and a PCB image is input into the PCB defect detection model for defect detection; the result display module is used for displaying the type, accuracy, size and position indexes of the PCB defect, generating a detection result and displaying the detection result to a user, and finally automatically storing the detection result.
CN202311063487.3A 2023-08-23 2023-08-23 Lightweight PCB defect detection method and system based on high-order spatial interaction Pending CN117094972A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311063487.3A CN117094972A (en) 2023-08-23 2023-08-23 Lightweight PCB defect detection method and system based on high-order spatial interaction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311063487.3A CN117094972A (en) 2023-08-23 2023-08-23 Lightweight PCB defect detection method and system based on high-order spatial interaction

Publications (1)

Publication Number Publication Date
CN117094972A true CN117094972A (en) 2023-11-21

Family

ID=88778058

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311063487.3A Pending CN117094972A (en) 2023-08-23 2023-08-23 Lightweight PCB defect detection method and system based on high-order spatial interaction

Country Status (1)

Country Link
CN (1) CN117094972A (en)

Similar Documents

Publication Publication Date Title
CN111524106B (en) Skull fracture detection and model training method, device, equipment and storage medium
CN110060237B (en) Fault detection method, device, equipment and system
Chen et al. Median filtering forensics based on convolutional neural networks
CN111368636B (en) Object classification method, device, computer equipment and storage medium
JP2015087903A (en) Apparatus and method for information processing
CN110008953B (en) Potential target area generation method based on convolution neural network multi-layer feature fusion
CN110807362A (en) Image detection method and device and computer readable storage medium
CN113487610B (en) Herpes image recognition method and device, computer equipment and storage medium
CN111126278A (en) Target detection model optimization and acceleration method for few-category scene
WO2024021461A1 (en) Defect detection method and apparatus, device, and storage medium
CN110781980A (en) Training method of target detection model, target detection method and device
CN113052006A (en) Image target detection method and system based on convolutional neural network and readable storage medium
Geng et al. An improved helmet detection method for YOLOv3 on an unbalanced dataset
CN114897782B (en) Gastric cancer pathological section image segmentation prediction method based on generation type countermeasure network
CN115439395A (en) Defect detection method and device for display panel, storage medium and electronic equipment
CN117690128B (en) Embryo cell multi-core target detection system, method and computer readable storage medium
CN117037004A (en) Unmanned aerial vehicle image detection method based on multi-scale feature fusion and context enhancement
CN112084887A (en) Attention mechanism-based self-adaptive video classification method and system
CN116563913A (en) Mask standard wearing detection method based on multilayer feature fusion
CN114387230A (en) PCB defect detection method based on re-verification detection
CN112966687B (en) Image segmentation model training method and device and communication equipment
CN110135428A (en) Image segmentation processing method and device
CN112825116B (en) Method, device, medium and equipment for detecting and tracking human face of monitoring video image
CN111582057A (en) Face verification method based on local receptive field
CN116721288A (en) Helmet detection method and system based on YOLOv5

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination