CN116740460A - Pcb defect detection system and detection method based on convolutional neural network - Google Patents
Pcb defect detection system and detection method based on convolutional neural network Download PDFInfo
- Publication number
- CN116740460A CN116740460A CN202310793897.7A CN202310793897A CN116740460A CN 116740460 A CN116740460 A CN 116740460A CN 202310793897 A CN202310793897 A CN 202310793897A CN 116740460 A CN116740460 A CN 116740460A
- Authority
- CN
- China
- Prior art keywords
- image
- pcb
- defect
- convolutional neural
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 93
- 238000001514 detection method Methods 0.000 title claims abstract description 70
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 39
- 230000007246 mechanism Effects 0.000 claims abstract description 15
- 238000012805 post-processing Methods 0.000 claims abstract description 14
- 238000000605 extraction Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 20
- 230000011218 segmentation Effects 0.000 claims description 18
- 238000007781 pre-processing Methods 0.000 claims description 17
- 239000013598 vector Substances 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 10
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000005549 size reduction Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000001186 cumulative effect Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000003213 activating effect Effects 0.000 claims description 2
- 230000001537 neural effect Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 5
- 230000002787 reinforcement Effects 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000011179 visual inspection Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013137 model compression technique Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 235000012434 pretzels Nutrition 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a PCB defect detection system and a PCB defect detection method based on a convolutional neural network, which belong to the technical field of convolutional neural networks, wherein S1, a PCB image is obtained; s2, inputting the preprocessed PCB image into a detection module; s3, the detection module adopts a convolutional neural network and an attention module to output a defect detection result; s4, performing post-processing on the defect detection result to obtain a defect area in the PCB image; the scheme can realize automatic detection, namely, the convolutional neural network and the attention mechanism are adopted to realize image feature extraction and classification recognition, and the PCB defects can be automatically and accurately positioned and recognized.
Description
Technical Field
The invention belongs to the technical field of convolutional neural networks, and particularly relates to a pcb defect detection system and method based on a convolutional neural network.
Background
PCB (Printed Circuit Board) the Chinese name printed circuit board, also called printed circuit board, is an important electronic component, is a support for electronic components, and is a carrier for electrical connection of electronic components. Whether the PCB itself has defects or not directly affects the equipment performance of the equipment using the PCB, so the defect detection of the PCB is particularly necessary.
At present, the defect detection of the PCB circuit board is mainly traditional manual visual inspection, the manual visual inspection has higher omission rate and false detection rate, the visual inspection has very low detection efficiency, and a large amount of manpower is consumed, or the production cost of enterprises is directly improved, and the market competitiveness of products is reduced.
Disclosure of Invention
The invention aims to provide a pcb defect detection system and a pcb defect detection method based on a convolutional neural network, which are used for solving the technical problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions: a pcb defect detection system based on convolutional neural network, comprising:
the image input module is used for acquiring PCB images;
the image preprocessing module is used for preprocessing the PCB image;
the detection module is used for detecting defects of the preprocessed PCB image by adopting a convolutional neural network and an attention mechanism and outputting a defect detection result;
and the post-processing module is used for carrying out post-processing on the defect detection result to obtain a defect area in the PCB image.
As a preferred embodiment, the image input module inputs an original image Io of the PCB to be detected;
the image preprocessing module is used for carrying out resolution conversion on the Io image and carrying out size reduction to the image I 'after size reduction, and carrying out enhancement and standardization processing on the image I' after size reduction, wherein the image preprocessing module comprises brightness enhancement, contrast enhancement, noise filtering, and normalization of dividing the pixel value of the image by 255 to be within the range of 0 and 1; obtaining a standardized image Is;
the detection module Is used for adopting a convolutional neural network structure based on a residual error module, adding an attention mechanism into a Decoder part to obtain effective characteristics, and respectively carrying out defect positioning, classification and segmentation on an Is image to obtain the following components:
judging whether the image has defects or not according to the confidence map Pd;
classifying the confidence map Pc, and judging the defect type;
dividing the confidence map Ps to divide the defect area;
the post-processing module is used for obtaining a boundary frame B of a region with highest segmentation confidence as a defect region according to Pd, pc and Ps results and according to whether defects and types of the defects exist in the threshold value Judge image;
and outputting a defect area B and a corresponding category existing in the original image Io of the PCB.
Based on the system, the scheme also discloses a pcb defect detection method of the convolutional neural network based on the system, which comprises the following steps:
s1, acquiring a PCB image, and preprocessing the PCB image;
s2, inputting the preprocessed PCB image into a detection module;
s3, the detection module adopts a convolutional neural network and an attention module to output a defect detection result;
and S4, carrying out post-processing on the defect detection result to obtain a defect area in the PCB image.
As a preferred embodiment, the method comprises the following steps:
s1, obtaining a PCB image, namely inputting the PCB image Io to be detected;
s2, image preprocessing comprises the following steps:
s21, carrying out size positioning on the Io image, and converting the Io image into fixed resolution;
s22, performing operations such as brightness, contrast enhancement, noise filtration and the like on the image I' with the size positioned, so that the image is clear and easy to distinguish;
s23, performing image normalization to normalize pixel values to a [0,1] range;
the step S3 further comprises the following steps:
s31, extracting image features by adopting a convolutional neural network based on a residual error module;
s32, adding an attention module in the decoding part;
s33, respectively generating a defect existence prediction graph Pd, a classification confidence graph Pc and a defect segmentation graph Ps, and respectively corresponding to positioning, classification and segmentation results;
s4, post-processing comprises the following steps:
s41, judging whether the image has defects according to the confidence coefficient graph Pd;
s42, judging the defect type according to the classification confidence coefficient map Pc;
s43, obtaining a boundary box of the defect area B according to the segmentation confidence map Ps;
s44, calculating a confidence coefficient mean value of pixels in the region B, and finally determining defect types;
s5, outputting to obtain a defect area B and a corresponding category in the original image Io of the PCB.
As a preferred embodiment, two indexes of the number of blemishes and the degree of edge blurring are set in step S2, if the number of blemishes of the image can be controlled to be the rated number and the degree of edge blurring is less than 0.9, the image is clear and easy to distinguish, 0 in the degree of edge blurring indicates that the image is completely clear, and 1 indicates that the degree of blurring is highest.
As a preferred implementation mode, a Scharr filter is used for detecting edges in an image, then the number of edge pixels is counted to be used as an index of the edge blurring degree, then probability Hough transformation is used for detecting centroid points in the image, the number of the centroid points is counted to be used as an index of the number of blemishes, and whether the image definition meets the standard is judged according to the two indexes.
Further, constructing the neural model based on the residual network comprises the following steps:
selecting the number of input channels of the model according to the size of the PCB image;
constructing a first convolution layer, selecting a filter and setting a stride;
alleviating the overfitting, and activating by adopting Batchnormal and ReLU;
constructing a second convolution layer, a corresponding filter, a stride, and then a Batchnormalization and a ReLU;
constructing a third convolution layer, selecting a corresponding filter and stride, and then, carrying out Batchnormalization and ReLU;
using a maximum pooling layer, step 2, downsampling;
constructing a residual block comprising 2-3 convolution layers;
repeatedly constructing a plurality of residual blocks, and downsampling for 2 times each time;
initializing model parameters: he initialization or Xavier initialization is used;
training a model: adopting an SGD optimizer, the learning rate is 0.01, and the learning rate is reduced by 64 batches;
inputting a PCB image, and extracting basic features through a convolution and pooling layer;
extracting higher level features by a plurality of residual blocks;
acquiring the output of the last convolution layer as the overall characteristic of the image;
l2 regularization is carried out on the model to avoid overfitting;
the special requirements of the PCB image are complemented by fine-tuning pre-training weights.
Further, the specific steps of the attention module are as follows:
the convolutional neural network completes feature extraction to obtain feature graphs with various scales;
tiling and reordering, namely tiling all feature images into a feature vector through full connected;
constructing a query vector q as a key vector to be searched for association;
key-value construction, namely decomposing the feature vector into key-value pairs (k, v);
calculating an inner product, namely calculating the inner product of q and all key vectors, and measuring the association degree;
normalization and weighting, namely calculating attention weight by using a Softmax function, and carrying out weighted summation with a value vector;
feature fusion, namely fusing attention to an output layer to realize semantic matching;
and decoding and predicting to realize final classification and segmentation.
Further, the device also comprises a rewarding module; namely, the construction of the reward function specifically comprises the following steps:
the method comprises the steps of obtaining a PCB image, preprocessing the PCB image, inputting the preprocessed PCB image into a detection module, and outputting a defect detection result by the detection module by adopting a convolutional neural network and an attention mechanism;
optimizing and adjusting the defect detection result, and obtaining a defect area in the PCB image by adopting the optimized detection result;
in the online reasoning stage, a state space and an action space are defined;
estimating the value in each state by using the DQN network, continuously exploring different actions, and obtaining the maximum cumulative rewards;
and updating the DQN network in real time, and continuously optimizing the rewarding function.
Compared with the prior art, the invention has the beneficial effects that:
1. the automatic detection is realized, namely the convolutional neural network and the attention mechanism are adopted to realize the image feature extraction and the classification recognition, so that the PCB defects can be automatically and accurately positioned and recognized. The scheme uses a convolutional neural network to extract image features, then strengthens the decoding process through a attention mechanism, improves the defect segmentation effect, and finally realizes automatic detection and identification. 2. The identification variety is rich, namely a multi-task detection network is adopted, and meanwhile, the defect positioning, classifying and dividing functions are realized. Different types of PCB defects may be identified. The proposal mentions that the detection module simultaneously realizes defect positioning, classification and segmentation, which shows that a plurality of defect categories can be identified, and the identification range is enlarged. 3. The method is expected to be efficient, and the convolutional neural network and the attention mechanism connected by the residual are adopted, so that efficient reasoning of model training is facilitated. Residual connection can effectively solve gradient dispersion problem, and attention mechanism can focus key characteristics, which helps to improve detection efficiency. 4. The method has the robustness, and the robustness and the stability of the model can be improved through data enhancement, rewarding mechanism and post-processing. Scheme mentioning data enhancement can improve model environment adaptation, reinforcement of learning rewards and post-processing helps to optimize and reduce detection errors. 5. There is further room for improvement, such as improved accuracy and speed, improved scheme details, etc. by employing model integration techniques.
Meanwhile, the scheme is added with a reward mechanism, and experience can be continuously obtained through interaction with the environment through reinforcement learning rewards.
The reinforcement learning reward function can optimize itself in real time by continuously exploring and learning, thereby improving the detection effect.
Reinforcement learning does not require a large amount of annotation data, and samples are obtained for model optimization only through environmental interaction exploration.
The reinforcement learning can make up for the data distribution difference of the model, and improves the stability and generalization capability of the detection effect.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
description of the embodiments
Examples
Referring to fig. 1, the invention provides a pcb defect detection system and a pcb defect detection method based on a convolutional neural network, comprising the following steps:
s1, obtaining a PCB image, namely inputting the PCB image Io to be detected;
s2, image preprocessing comprises the following steps:
s21, carrying out size positioning on the Io image, and converting the Io image into fixed resolution;
s22, performing operations such as brightness, contrast enhancement, noise filtration and the like on the image I' with the size positioned, so that the image is clear and easy to distinguish;
s23, performing image normalization to normalize pixel values to a [0,1] range;
the step S3 further comprises the following steps:
s31, extracting image features by adopting a convolutional neural network based on a residual error module;
s32, adding an attention module in the decoding part;
s33, respectively generating a defect existence prediction graph Pd, a classification confidence graph Pc and a defect segmentation graph Ps, and respectively corresponding to positioning, classification and segmentation results;
s4, post-processing comprises the following steps:
s41, judging whether the image has defects according to the confidence coefficient graph Pd;
s42, judging the defect type according to the classification confidence coefficient map Pc;
s43, obtaining a boundary box of the defect area B according to the segmentation confidence map Ps;
s44, calculating a confidence coefficient mean value of pixels in the region B, and finally determining defect types;
s5, outputting to obtain a defect area B and a corresponding category in the original image Io of the PCB.
Wherein the image pre-processing may be positioned with a fixed size, such as direct resize to 300x300 pixels for the entire PCB image. Dynamic positioning is also possible, such as maintaining the aspect ratio of the PCB image (typically around 2), with the longest edge resize to 300 pixels. The optional dimensions include 224x224, 320x320, 416x416, etc. according to the model input size values. If GPU parallelism is taken into account, a multiple size of 32 is preferably chosen, which is advantageous for acceleration. Image sharpness criterion-two indicators of number of blemishes (blemishes) and degree of edge blurriness (blurriness) are set. If the number of blemishes in the image can be controlled to be around 50 and the degree of edge blurring is less than 0.9 (0 means completely clear, 1 means highest degree of blurring), it indicates that the image is clearly discernible. The method comprises the specific operation steps of 1) detecting edges in an image by using Scharr filtering, 2) counting the number of edge pixel points to serve as an index of the edge blurring degree, 3) detecting centroid (center) points in the image by using probability Hough transformation, 4) counting the number of centroid points to serve as an index of the number of blemishes, and 5) judging whether the image definition meets the standards or not according to the two indexes.
The operations of performing brightness, contrast enhancement, noise filtering, and the like are specifically as follows: brightness enhancement, namely calculating a histogram of the image I', determining brightness distribution conditions, adopting linear conversion or gamma correction according to the distribution conditions to expand brightness values to be in a range of 0-255, and increasing local contrast by using a CLAHE (contrast adaptive histogram equalization) algorithm.
Contrast enhancement, computing a histogram of the image, determining a current contrast distribution, increasing contrast using histogram planning techniques, such as comprehensive log-reflected rate Conversion (CLR), and increasing global and local contrast using algorithms such as geometric equalization.
Noise filtering, which uses gaussian filtering to reduce gaussian noise and pretzel noise, median filtering or partial triangular filtering can be used for irregular noise in the PCB image, and time-based filtering can be used for sequential images.
The specific method for constructing the model structure based on the residual error network comprises the following steps:
if the PCB image size is 224x224, the model input channel number is 3 (RGB).
A convolution layer of 7*7 was constructed, filter size 64, stride 2. The remission overfitting used Batchnormalization and ReLU activation.
3*3 convolution layers were built, filter 32, stride 1, keep up with the Batchnormalization and ReLU. (4) 3*3 convolutional layers, filter 64, stride 1, then Batchnormalization and ReLU were constructed. (5) using the max-pooling layer of 3*3, stride 2 downsampling.
A residual block is constructed comprising 2-3 convolutional layers.
The construction of more residual blocks is repeated, 2 times each time downsampled.
The model parameters are then initialized. He initialization or Xavier initialization was used.
The model is then trained. And the SGD optimizer is adopted, the learning rate is 0.01, the learning rate is reduced, and the batch size is 64.
The PCB image is then input and the basic features are extracted by the convolution and pooling layers.
Then, higher level features are extracted through a plurality of residual blocks.
And then, the output of the last convolution layer is obtained and used as the overall characteristic of the image.
Then, L2 regularization is performed on the model to avoid overfitting.
Then, when deploying the model, the model compression technique is used to increase the speed.
The special requirements of the PCB image are then complemented by fine-tuning pre-training weights.
In the above scheme, in the decoder, the specific steps of adding the decoder into the attention module are as follows:
1) And in the encoding stage, the convolutional neural network finishes feature extraction and obtains feature graphs with various scales.
2) Tiling and reordering, namely tiling all feature graphs into a feature vector through full connected.
3) Query construction, namely constructing a query vector q as a key for searching the association.
4) Key-value construction by decomposing a feature vector into key-value pairs (k, v)
5) Calculating inner products, namely calculating the inner products of q and all keys, and measuring the association degree.
6) Normalization and weighting the attention weights are calculated using a Softmax function and weighted sum with the value vectrors.
7) Feature fusion, namely fusing the attention context to an output layer to realize a semantic matching function.
8) Decoding prediction-the decoder completes the final prediction.
The scheme has the advantages that: 1) extracting various scale features, 2) realizing feature global context, 3) constructing attention points, 4) establishing mapping pairs, 5) evaluating association degree, 6) calculating attention weight, 7) fusing context information, and 8) realizing final classification and segmentation.
In the implementation process, a reward module can be added, and the method specifically comprises the following steps:
the method comprises the steps of obtaining a PCB image, preprocessing the PCB image, inputting the preprocessed PCB image into a detection module, and outputting a defect detection result by the detection module by adopting a convolutional neural network and an attention mechanism;
optimizing and adjusting the defect detection result, and obtaining a defect area in the PCB image by adopting the optimized detection result;
in the online reasoning stage, a state space and an action space are defined;
estimating the value in each state by using the DQN network, continuously exploring different actions, and obtaining the maximum cumulative rewards;
and updating the DQN network in real time, and continuously optimizing the rewarding function.
Meanwhile, in the implementation process, a rewarding process monitoring module can be added, namely, in the working process of the rewarding module, the change of the rewarding weight of a rewarding mechanism is recorded and monitored for a certain pcb product within a specified period of time; on the basis of the total quantity of the awarded token, recording the trend and the direction of the increase of the awards of the change amplitude of the weight so as to be convenient for the working state of the goods value system in time; if the weight change is too large or the rewards are increased rapidly in a period of time, the accuracy of the finally checked product is required to be approved, the generation of bad and path dependence is avoided, and the running quality of the system is ensured; if the direction of a certain bonus representative suddenly decreases over a period of time, it is necessary to check whether information such as the number of the incoming product, manufacturer, customer, etc. is wrong.
Finally, it is noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered by the scope of the claims of the present invention.
Claims (9)
1. A pcb defect detection system based on convolutional neural network, comprising:
the image input module is used for acquiring PCB images;
the image preprocessing module is used for preprocessing the PCB image;
the detection module is used for detecting defects of the preprocessed PCB image by adopting a convolutional neural network and an attention mechanism and outputting a defect detection result;
and the post-processing module is used for carrying out post-processing on the defect detection result to obtain a defect area in the PCB image.
2. The convolutional neural network-based pcb defect detection system of claim 1, wherein: the image input module is used for inputting an original image Io of the PCB to be detected;
the image preprocessing module is used for carrying out resolution conversion on the Io image and carrying out size reduction to the image I 'after size reduction, and carrying out enhancement and standardization processing on the image I' after size reduction, wherein the image preprocessing module comprises brightness enhancement, contrast enhancement, noise filtering, and normalization of dividing the pixel value of the image by 255 to be within the range of 0 and 1; obtaining a standardized image Is;
the detection module Is used for adopting a convolutional neural network structure based on a residual error module, adding an attention mechanism into a Decoder part to obtain effective characteristics, and respectively carrying out defect positioning, classification and segmentation on an Is image to obtain the following components:
judging whether the image has defects or not according to the confidence map Pd;
classifying the confidence map Pc, and judging the defect type;
dividing the confidence map Ps to divide the defect area;
the post-processing module is used for obtaining a boundary frame B of a region with highest segmentation confidence as a defect region according to Pd, pc and Ps results and according to whether defects and types of the defects exist in the threshold value Judge image;
and outputting a defect area B and a corresponding category existing in the original image Io of the PCB.
3. A pcb defect detection method based on a convolutional neural network is characterized in that: the method comprises the following steps:
s1, acquiring a PCB image, and preprocessing the PCB image;
s2, inputting the preprocessed PCB image into a detection module;
s3, the detection module adopts a convolutional neural network and an attention module to output a defect detection result;
and S4, carrying out post-processing on the defect detection result to obtain a defect area in the PCB image.
4. A pcb defect detection method based on convolutional neural network according to claim 3, wherein: the method comprises the following steps:
s1, obtaining a PCB image, namely inputting the PCB image Io to be detected;
s2, image preprocessing comprises the following steps:
s21, carrying out size positioning on the Io image, and converting the Io image into fixed resolution;
s22, performing operations such as brightness, contrast enhancement, noise filtration and the like on the image I' with the size positioned, so that the image is clear and easy to distinguish;
s23, performing image normalization to normalize pixel values to a [0,1] range;
the step S3 further comprises the following steps:
s31, extracting image features by adopting a convolutional neural network based on a residual error module;
s32, adding an attention module in the decoding part;
s33, respectively generating a defect existence prediction graph Pd, a classification confidence graph Pc and a defect segmentation graph Ps, and respectively corresponding to positioning, classification and segmentation results;
s4, post-processing comprises the following steps:
s41, judging whether the image has defects according to the confidence coefficient graph Pd;
s42, judging the defect type according to the classification confidence coefficient map Pc;
s43, obtaining a boundary box of the defect area B according to the segmentation confidence map Ps;
s44, calculating a confidence coefficient mean value of pixels in the region B, and finally determining defect types;
s5, outputting to obtain a defect area B and a corresponding category in the original image Io of the PCB.
5. The pcb defect detection method based on convolutional neural network of claim 4, wherein: and step S2, setting two indexes of the number of blemishes and the edge blurring degree, wherein if the number of blemishes of the image can be controlled at the rated number and the edge blurring degree is smaller than 0.9, the image is clear and easy to distinguish, 0 in the edge blurring degree indicates complete definition, and 1 indicates the highest blurring degree.
6. The pcb defect detection method based on convolutional neural network of claim 5, wherein: detecting edges in an image by using a Scharr filter, counting the number of edge pixel points to serve as an index of the edge blurring degree, detecting centroid points in the image by using probability Hough transformation, counting the number of centroid points to serve as an index of the number of blemishes, and judging whether the definition of the image meets the standard or not according to the two indexes.
7. The pcb defect detection method based on convolutional neural network of claim 6, wherein: the construction of the neural model based on the residual network comprises the following steps:
selecting the number of input channels of the model according to the size of the PCB image;
constructing a first convolution layer, selecting a filter and setting a stride;
alleviating the overfitting, and activating by adopting Batchnormal and ReLU;
constructing a second convolution layer, a corresponding filter, a stride, and then a Batchnormalization and a ReLU;
constructing a third convolution layer, selecting a corresponding filter and stride, and then, carrying out Batchnormalization and ReLU;
using a maximum pooling layer, step 2, downsampling;
constructing a residual block comprising 2-3 convolution layers;
repeatedly constructing a plurality of residual blocks, and downsampling for 2 times each time;
initializing model parameters: he initialization or Xavier initialization is used;
training a model: adopting an SGD optimizer, the learning rate is 0.01, and the learning rate is reduced by 64 batches;
inputting a PCB image, and extracting basic features through a convolution and pooling layer;
extracting higher level features by a plurality of residual blocks;
acquiring the output of the last convolution layer as the overall characteristic of the image;
l2 regularization is carried out on the model to avoid overfitting;
the special requirements of the PCB image are complemented by fine-tuning pre-training weights.
8. The pcb defect detection method based on convolutional neural network of claim 7, wherein: the specific steps of the attention module are as follows:
the convolutional neural network completes feature extraction to obtain feature graphs with various scales;
tiling and reordering, namely tiling all feature images into a feature vector through full connected;
constructing a query vector q as a key vector to be searched for association;
key-value construction, namely decomposing the feature vector into key-value pairs (k, v);
calculating an inner product, namely calculating the inner product of q and all key vectors, and measuring the association degree;
normalization and weighting, namely calculating attention weight by using a Softmax function, and carrying out weighted summation with a value vector;
feature fusion, namely fusing attention to an output layer to realize semantic matching;
and decoding and predicting to realize final classification and segmentation.
9. The pcb defect detection method based on convolutional neural network of claim 8, wherein: the system also comprises a rewarding module; the method specifically comprises the following steps:
the method comprises the steps of obtaining a PCB image, preprocessing the PCB image, inputting the preprocessed PCB image into a detection module, and outputting a defect detection result by the detection module by adopting a convolutional neural network and an attention mechanism;
optimizing and adjusting the defect detection result, and obtaining a defect area in the PCB image by adopting the optimized detection result;
in the online reasoning stage, a state space and an action space are defined;
estimating the value in each state by using the DQN network, continuously exploring different actions, and obtaining the maximum cumulative rewards;
and updating the DQN network in real time, and continuously optimizing the rewarding function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310793897.7A CN116740460A (en) | 2023-06-30 | 2023-06-30 | Pcb defect detection system and detection method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310793897.7A CN116740460A (en) | 2023-06-30 | 2023-06-30 | Pcb defect detection system and detection method based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116740460A true CN116740460A (en) | 2023-09-12 |
Family
ID=87918428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310793897.7A Pending CN116740460A (en) | 2023-06-30 | 2023-06-30 | Pcb defect detection system and detection method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116740460A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117456168A (en) * | 2023-11-08 | 2024-01-26 | 珠海瑞杰电子科技有限公司 | PCBA intelligent detection system and method based on data analysis |
-
2023
- 2023-06-30 CN CN202310793897.7A patent/CN116740460A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117456168A (en) * | 2023-11-08 | 2024-01-26 | 珠海瑞杰电子科技有限公司 | PCBA intelligent detection system and method based on data analysis |
| CN117456168B (en) * | 2023-11-08 | 2024-04-16 | 珠海瑞杰电子科技有限公司 | PCBA intelligent detection system and method based on data analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110363182B (en) | Deep learning-based lane line detection method | |
| CN111723748B (en) | Infrared remote sensing image ship detection method | |
| CN108133188B (en) | Behavior identification method based on motion history image and convolutional neural network | |
| CN110472675B (en) | Image classification method, image classification device, storage medium and electronic equipment | |
| CN111626993A (en) | Image automatic detection counting method and system based on embedded FEFnet network | |
| CN112734696A (en) | Face changing video tampering detection method and system based on multi-domain feature fusion | |
| CN110245620B (en) | Non-maximization inhibition method based on attention | |
| CN116740728B (en) | Dynamic acquisition method and system for wafer code reader | |
| CN116740460A (en) | Pcb defect detection system and detection method based on convolutional neural network | |
| CN113537211A (en) | Deep learning license plate frame positioning method based on asymmetric IOU | |
| CN110991374B (en) | Fingerprint singular point detection method based on RCNN | |
| CN116894985A (en) | Semi-supervised image classification method and semi-supervised image classification system | |
| CN112884721B (en) | Abnormality detection method, abnormality detection system and computer-readable storage medium | |
| CN113989256A (en) | Detection model optimization method, detection method and detection device for remote sensing image building | |
| CN113191352A (en) | Water meter pointer reading identification method based on target detection and binary image detection | |
| CN116311212B (en) | Ship number identification method and device based on high-speed camera and in motion state | |
| CN113421223A (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN116524317A (en) | Image cutting and prediction frequency statistics method, device, equipment and medium | |
| CN111582057B (en) | Face verification method based on local receptive field | |
| CN117392440B (en) | Textile fabric retrieval method and system based on tissue structure and color classification | |
| CN117011730A (en) | Unmanned aerial vehicle image change detection method, electronic terminal and storage medium | |
| CN113792630B (en) | Method and system for identifying extraterrestrial detection image based on contrast learning | |
| CN118038122A (en) | Biological classification method, device, equipment and medium based on image fusion | |
| CN113569835A (en) | Water meter numerical value reading method based on target detection and segmentation identification | |
| CN118053036A (en) | Transmission line image analysis method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |