WO2024087640A1

WO2024087640A1 - Printed circuit board welding spot defect detection method based on digital image processing

Info

Publication number: WO2024087640A1
Application number: PCT/CN2023/098804
Authority: WO
Inventors: 左健存; 常远培; 薛颖; 张宇; 孙晶国; 季张源; 李和威
Original assignee: 上海第二工业大学
Priority date: 2022-10-26
Filing date: 2023-06-07
Publication date: 2024-05-02
Also published as: CN115546182A

Abstract

Disclosed in the present invention is a printed circuit board welding spot defect detection method based on digital image processing. The method comprises the following steps: step 1, collecting a template picture and an original picture of a printed circuit board to be subjected to detection; step 2, preprocessing the template picture and a picture to be subjected to detection; step 3, performing particle-swarm-optimization(PSO)-based maximum inter-class variance (i.e. OTSU) thresholding on the preprocessed template picture and the preprocessed picture to be subjected to detection; step 4, according to the template picture and by using a FLANN optimization SURF algorithm, performing image registration on the picture to be subjected to detection, which has been subjected to thresholding; step 5, by using an image subtraction method, performing subtraction on a registered picture to be subjected to detection and the template picture, so as to obtain a difference image; and step 6, performing binarization on the difference image, and performing morphological processing such as an opening operation of corrosion followed by dilation, so as to finally locate a position with a defect. The detection method in the present invention is simple, is high in practicability, and can significantly improve the accuracy of detection.

Description

A printed circuit board solder joint defect detection method based on digital image processing

Technical Field

The invention belongs to the technical field of printed circuit board defect detection, and in particular relates to a printed circuit board solder joint defect detection method based on digital image processing.

Background technique

The continuous improvement of industrial production and manufacturing levels has accelerated the replacement of electronic products, thus putting forward higher requirements for the hardware facilities of the underlying infrastructure. PCB is an important basic component of integrated circuits, and its quality determines the overall performance of electronic products. However, it is difficult to avoid defects in the produced PCB due to various factors during the production process of PCB. Conventional methods for detecting solder joint defects, mainly including manual visual inspection, electrical inspection, X-ray inspection, etc., all have their own defects. Among them, manual visual inspection is costly, slow, and subjective; electrical inspection has low measurement accuracy, takes a long time, and can only detect a limited number of defects; X-ray inspection is slow, costly, and requires a long software development cycle. At the same time, the demand for PCB defect detection by individuals and small and medium-sized enterprises is also increasing. Whether the defect detection of solder joints can achieve low cost and high accuracy is the most important consideration. Therefore, it is important to study how to improve the accuracy of PCB defect detection through low-cost image processing technology.

Summary of the invention

The technical problem to be solved by the present invention is to provide a printed circuit board solder joint defect detection method based on digital image processing, which has good defect positioning performance, low cost, high accuracy and good practical performance.

The present invention adopts the following technical scheme.

A printed circuit board solder joint defect detection method based on digital image processing comprises the following steps:

Step 1: Capture the template image and the original image of the printed circuit board to be inspected through an image acquisition device.

Step 2: preprocessing the template image and the image to be tested; the preprocessing step includes grayscale transformation and median filtering;

Step 3: Perform particle swarm optimization (PSO) optimized maximum between-class variance (OTSU) threshold segmentation on the preprocessed template image and the image to be tested.

① Initialize PSO, set the population size to N, the number of thresholds to 2, the inertia weight to W, and the maximum number of iterations to G.

②According to the functional relationship of fitness, the fitness value is obtained and judged, and the individual optimal value and global optimal value of the particle are found. According to the relationship between formula (1) and formula (2), the corresponding values of its speed and position are calculated and updated.

_vi,j (t+1)＝wvi _,j (t ₎ + _c1r1 [pi _,j - _xi,j (t ₎ ]+ _c2r2 [pg _,j - _xi,j (t)] (1)

x _i,j (t+1)＝x _i,j (t)+v _i,j (t+1)j＝1,2,3,...,d, (2)

Where: t is the number of cycles; w is the inertia factor; vi _,j is the velocity of the i-th particle in the j-dimensional solution space;

x _i,j is the position of the i-th particle in the j-dimensional solution space; p _g,j is the global extreme value; p _i,j is the individual extreme value; c ₁ and c ₂ are learning factors, c ₁ represents the particle's self-summary learning ability, and c ₂ represents the ability to learn from the best particle in the population; r ₁ and r ₂ are random numbers randomly distributed in [0,1].

③ Calculate the fitness value again based on the inertia weight and make a judgment to find the optimal position as the current position.

④ Termination condition judgment: if the number of iterations is less than the maximum number of iterations, return to ②; if it is greater than the maximum number of iterations, terminate the algorithm.

⑤The global optimal value at this time is the threshold for segmentation. Use this threshold to perform threshold segmentation on the image.

Step 4: Perform image registration of the threshold segmented image with the template image that has also been threshold segmented using the FLANN optimized SURF algorithm. Extract basic feature points through the Hessian matrix, and then use the FLANN algorithm and RANSAC algorithm to remove mismatched points to improve the accuracy of feature point matching, and use the coordinates of the extracted optimal matching points to generate a perspective transformation matrix. Perform geometric transformation on the image to be tested to generate a registered image.

Step 5: Perform the difference method on the image, and subtract the image to be tested after registration from the template image after threshold segmentation to obtain a difference image.

Step 6: Binarize the difference image, and then perform an opening operation of first corrosion and then expansion. Corrosion reduces the gray value of the noise, and expansion increases the gray value of the defect. Therefore, the opening operation can effectively filter out the small noise in the difference image and highlight the defect information. Ultimately, the defects including leaking solder, notch, open circuit, short circuit, burr, and excess copper can be accurately located.

Compared with the existing technology, the beneficial effects of the present invention are:

The subjectivity of manual visual inspection is avoided. Compared with electrical detection, the present invention belongs to non-contact detection, which avoids product damage. The detection equipment of the present invention is simple and low in cost, suitable for individuals and small and medium-sized enterprises. The detection accuracy is as high as 99.3%, the false detection rate and missed detection rate are low, and it is suitable for PCB boards of different specifications. It has practical significance for improving product quality and saving costs.

The present invention reduces and eliminates interference information caused by internal and external factors during image acquisition and transmission through the preprocessing process, and uses image segmentation to make the image have sufficiently clear information of interest before recognition, so as to facilitate subsequent detection work. The above basic features of PCB such as solder joints, circuits, edges, etc. are better preserved and highlighted through image preprocessing and image segmentation.

The detection method of the present invention is simple and practical; the detection accuracy is significantly improved by optimizing the OTSU threshold segmentation by PSO and optimizing the SURF image registration algorithm by FLANN in the above steps 3 and 4.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example diagram of an image to be inspected after preprocessing in Example 1. FIG.

FIG. 2 is an example diagram of the image to be inspected after image segmentation and registration in Example 1. FIG.

FIG. 3 is an example diagram of the difference between the template image and the image to be inspected in Example 1. FIG.

FIG. 4 is an example diagram of the image to be inspected in Example 1 after morphological processing.

FIG. 5 is a diagram showing a final defect detection example of Example 1. FIG.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, but the protection scope of the present invention shall not be limited thereto.

Example 1

First, the template image and the original image of the printed circuit board to be inspected are collected by the image acquisition device. Then, the image preprocessing is performed according to step 2, and the collected original image is grayed, grayscale linear transformation and median filtering are performed: ① Grayscale transformation of the image: Using piecewise linear grayscale transformation to enhance the image contrast can highlight the target or grayscale range of interest and relatively suppress those grayscale areas of no interest.

Among them: x ₁ and x ₂ are the grayscale ranges that need to be converted, and y ₁ and y ₂ determine the slope of the linear transformation.

② Perform median filtering on the image to remove salt and pepper noise:

y _ij : The pixel value of the point after median filtering.

{x _ij (i,j)∈I ² }: is the grayscale value of each pixel in the image.

It corresponds to FIG. 1 .

Then, according to step 3, use the PSO algorithm to optimize the OTSU threshold segmentation algorithm, set the group size to 150, the number of thresholds to 2, the inertia weight to 0.8, the maximum number of iterations to 25, the self-learning factor to 0.5, and the group learning factor to 0.5, find a suitable threshold, and perform image segmentation on the image.

Then, according to step 4, the template image is used as a reference. The image to be tested is optimized by the FLANN algorithm for the SURF registration algorithm. After the RANSAC algorithm removes the mismatched points, a perspective transformation is performed to achieve image registration.

Then, according to step 5, the registered image is subtracted from the template image to find the difference between the image to be tested and the template image, and an image with both noise and defects is obtained.

Finally, after binarizing the difference image according to step 6, morphological processing is performed to remove isolated bright spots in the image, delete objects with an area less than 40 in the binary value, eliminate noise interference, and then perform an opening operation of first erosion and then dilation. The present invention detects 693 PCB images of different specifications and different defect types, with an accuracy rate of 99.1%, a false detection rate of 0.58%, and a missed detection rate of 0.28%. The detection effect is good.

Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

A method for detecting solder joint defects of a printed circuit board based on digital image processing, characterized in that it comprises the following steps:

Step 1: Collect the template image and the original image of the printed circuit board to be tested;

Step 2: Preprocess the template image and the image to be tested by grayscale transformation and median filtering to reduce or eliminate image interference information;

Step 3: Perform particle swarm optimization (PSO)-optimized maximum inter-class variance OTSU threshold segmentation on the preprocessed template image and the image to be tested;

Step 4: Perform image registration using the FLANN optimized SURF algorithm on the image to be tested after threshold segmentation and the template image after the same threshold segmentation;

Step 5: Using the difference method, the registered image to be tested is subtracted from the template image after threshold segmentation to obtain a difference image;

Step 6: Binarize the difference image, and then perform an opening operation of first corrosion and then expansion. The grayscale value at the noise is reduced by corrosion, and the grayscale value at the defect is increased by expansion to filter out the fine noise in the difference image and highlight the defect information, ultimately achieving accurate positioning of defects including solder leaks, gaps, open circuits, short circuits, burrs, and excess copper.
The printed circuit board solder joint defect detection method according to claim 1 is characterized in that in step 3, the specific method of performing the maximum inter-class variance OTSU threshold segmentation optimized by particle swarm PSO on the preprocessed template image and the image to be tested includes:

① Initialize PSO, set the group size to N, the threshold number to 2, the inertia weight to W, and the maximum number of iterations to G;

②According to the functional relationship of fitness, the fitness value is obtained and judged, and the individual optimal value and global optimal value of the particle are found at this time; according to formula (1) and formula (2), the corresponding values of the updated particle speed and position are calculated;
vi,j (t+1)＝wvi ,j (t ) + c1r1 [pi ,j - xi,j (t ) ]+ c2r2 [pg ,j - xi,j (t)] (1)
x i,j (t+1)＝x i,j (t)+v i,j (t+1)j＝1,2,3,…,d, (2)

Where: t is the number of cycles; w is the inertia factor; vi ,j is the speed of the ith particle in the j-dimensional solution space; xi,j is the position of the ith particle in the j-dimensional solution space; pg ,j is the global extreme value; pi ,j is the individual extreme value; c1 and c2 are learning factors, c1 represents the learning ability of the particle itself, and c2 is the learning ability from the best particle in the population; r1 and r2 are random numbers randomly distributed in [0,1];

③ Calculate the fitness value again according to the inertia weight and make a judgment to find the optimal position as the current position;

④Termination condition judgment: if the number of iterations is less than the maximum number of iterations, return to ②; if it is greater than the maximum number of iterations, terminate the algorithm;

⑤The global optimal value at this time is the threshold for segmentation, and this threshold is used to perform threshold segmentation on the image.
The printed circuit board solder joint defect detection method according to claim 1 is characterized in that, in step 4, basic feature points are extracted by the Hessian matrix, and then the FLANN algorithm and the RANSAC algorithm are used to eliminate the mismatched points to improve the accuracy of feature point matching, and the coordinates of the extracted optimal pairing points are used to generate a perspective transformation matrix, and the image to be tested is geometrically transformed to generate a registration image.