CN115829942A - Electronic circuit defect detection method based on non-negative constraint sparse self-encoder - Google Patents

Abstract

The invention discloses an electronic circuit defect detection method based on a non-negativity constrained sparse self-encoder, which includes: collecting and cutting electronic circuit images, image data enhancement and feature extraction, and determining defect data sets and defect types. The image denoising method based on the group graph model and the kernel norm denoises the electronic circuit image, obtains the Laplacian graph matrix of the image by optimizing the learning strategy, and utilizes its encapsulation graph structure. A deep learning model of non-negativity constrained sparse autoencoder is proposed and used in the extraction of defect regions. The circuit image is predicted by using the successfully trained autoencoder model, and the original defect input image is subtracted from the predicted image to generate a defect detection map, and an appropriate threshold is set to obtain the correctly classified defect type. Compared with the prior art, the present invention can effectively detect the true defects of the electronic circuit with large defect color changes and the false defects with small color changes, and improve the reliability and accuracy of electronic circuit defect identification.

Description

Electronic Circuit Defect Detection Method Based on Non-negativity Constrained Sparse Autoencoder

technical field

The invention relates to the technical field of electronic circuit defect classification, in particular to an electronic circuit defect detection method based on a non-negativity constrained sparse autoencoder.

Background technique

Electronic circuit boards are known to mechanically support the connection of electronic components through conductive traces, pads and solder joints. With the advancement of science and technology and the rapid development of the mobile electronic product market, electronic circuit boards are more diverse and complex, more electronic devices are integrated into electronic circuit boards, and the layout of electronic circuit boards is also increasing, resulting in problems are becoming increasingly apparent. It is typical that the signal transmission track defect in the circuit board has a huge impact on the signal of the entire system, resulting in the degradation of the performance of the connected electronic components, and these electronic components are the key to the function of the entire system, so it will eventually cause circuit failure and defects in circuit system performance. At present, it is difficult to distinguish the tiny defects of electronic circuit boards by conventional methods of human eye observation. Therefore, there is an urgent need for research on automatic detection methods for electronic circuit board defects, which is also one of the most important ways to control circuit quality.

Defect detection methods for electronic circuit boards can generally be divided into two categories: direct detection methods and camera-based machine vision methods. Direct inspection is a manual inspection that allows the operator to perform it easily using visual inspection. However, operators are easily fatigued from repetitive tasks, and individual test results are inconsistent, which is a fundamental limitation of human judgment. To overcome these limitations, researchers have studied defect detection based on machine vision, which generally includes a camera, light source, and operating system. Its main purpose is to use an automatic optical inspection system to improve inspection quality. Generally, industrial cameras such as charge-coupled devices (CCD) are used to obtain high-quality circuit defect images. This method is intuitive and easy to understand, but requires high shooting alignment accuracy and Sensitive light environment. In addition to the above methods, developers also use various machine vision and image processing algorithms to complete the defect detection of electronic circuits. In general, these methods require all defect types to be reported in advance. However, we cannot guarantee that the inspection system will only encounter defects that have been identified in advance. In the actual production environment, various sudden defects are often encountered, which cannot be detected correctly by conventional machine vision-based inspection methods. In this case, the defect inspection system must be able to be recalibrated with new sample data in the face of changing circuit manufacturing conditions. This is a major shortcoming of traditional machine vision inspection systems.

Contents of the invention

Purpose of the invention: In view of the fact that electronic circuit boards are increasingly diverse and complex, and it is difficult to identify tiny defects by conventional human eye observation, the present invention provides an electronic circuit defect detection method based on non-negativity constrained sparse autoencoder, By using an image sensor (industrial camera) to capture images of electronic circuits, use these images to train a deep autoencoder model, and decode the original non-defective image from the defective circuit image, and then compare the decoded circuit image with the input circuit image , so as to determine the defect location and achieve effective detection of circuit defects without prior knowledge of the defect type or the normal/defect evaluation criteria of the expert system, which can overcome the problem of small and unbalanced data sets in the early manufacturing stage.

Technical solution: The present invention discloses a method for detecting defects in electronic circuits based on non-negativity constrained sparse autoencoders, which includes the following steps:

(1) Preprocessing the electronic circuit defect data set; collecting and tailoring the electronic circuit image, determining the electronic circuit defect data set and defect type, and completing image data enhancement and feature selection;

(2) Complete the electronic circuit image denoising based on the group-based graph model and kernel norm method GNN, and use the Laplacian graph matrix and kernel norm of the image to encapsulate the graph structure;

(3) Construct a deep learning model based on non-negativity constrained sparse autoencoder FFSAE, and use it in the extraction of defect regions;

(4) Generate a defect detection map, use the successfully trained self-encoder FFSAE model to predict a high-quality circuit image, and subtract the original defect input image from the predicted image to generate a defect detection map, and finally set the defect detection map appropriately The method of threshold value is used to highlight the defect location, so as to complete the correct classification of electronic circuit defect types.

Further, the specific steps of preprocessing the electronic circuit defect data set in the step (1) are as follows:

Step (1)a: collecting and clipping the electronic circuit image, and determining the electronic circuit defect data set;

Step (1)b: Determine the type of defect in the electronic circuit, and set two kinds of defects, one is true defect and the other is false defect; among them, the true defect is really caused by the change of lead wire shape, specifically set as Disconnection defects, connection defects, protruding defects, and crack defects; false defects are characterized by only color changes, while the shape characteristics of leads and basic components remain unchanged, and are specifically divided into oxidation defects and dust defects;

Step (1)c: Complete image data enhancement and feature selection

Data enhancement: apply geometric transformation and adding noise to complete data enhancement: apply random rotation to overcome the position deviation of image data; multiply a random matrix with noise distribution with the original data;

Feature selection: Determine the feature parameters as color information and shape information respectively. Among them, there are 30 kinds of color information, and the following features are extracted from RGB and HSV color models respectively, which are 1) maximum value, 2) minimum value, and 3) average value. value, 4) scale high value, 5) ratio of lead area to candidate area and lead, 6) ratio of base part to candidate part, 7) position difference between numerical center of gravity and maximum value, 8) variance, 9) standard Bias, 10) kurtosis, 11) skewness, 12) entropy, 13) difference between maximum and minimum, 14) median, 15) correlation between test image and reference image; shape information A total of 8 types are included, including 1) area, 2) perimeter, 3) size in x direction, 4) size in y direction, 5) aspect ratio, 6) diagonal length, 7) complexity, and 8) roundness.

Further, in the step (2), the circuit image denoising based on the graphical model of the group and the kernel specification method GNN is completed, and the specific steps are as follows:

Step (2)a: Use the Laplacian matrix L to characterize the collected patch image

(1) Obtain the weighted adjacency matrix W according to the image data

The undirected weighted weighted adjacency matrix W graph is non-negative and has equal diagonal elements, that is, W _ij = w _ji , W _ij ≥ 0, and the threshold Gaussian kernel is used to form the weight matrix W of the edge, as follows:

是在图像顶点v_i和v_j之间的欧几里得距离，σ是控制权重随距离增加而衰减的速度控制参数，ε是阈值参数，代表ε-邻域图；in,

is the Euclidean distance between image vertices v _i and v _j , σ is a speed control parameter that controls weight decay with increasing distance, ε is a threshold parameter, representing the ε-neighborhood graph;

(2) Obtain the image L represented by the Laplacian matrix

L=Δ-W

Among them, Δ is a diagonal matrix, which satisfies the equation Δ _ii = ∑ _j W _ij ;

Step (2)b: Establish a grouping-based graphical model and a combined optimization formula for kernel norms

(1) Construct the basic optimization formula

是与拉普拉斯矩阵关联的正则化项，则基于图像去噪的基本优化公式如下：set up

is the regularization term associated with the Laplacian matrix, then the basic optimization formula based on image denoising is as follows:

Among them, x and y are both n×1 vectors representing image blocks, L is an n×n Laplacian matrix, and θ is a regularization parameter;

(2) Construct an optimization formula based on grouped dual graph

Considering that each group is a matrix, that is, to construct a dual graph T ^m×n including row graphs and column graphs, the optimal expression for defining the dual graph model is as follows:

和列图

Among them, X and Y are m×n image row and column data matrices, θ _r and θ _c are regularization control parameters, which are used to determine the degree of influence of the regularization term, that is, the row map

and column diagram

是基于组的行图正则化项，利用位于所有相似补丁图像的同一位置的像素强度的相似性来定义，具体为：

is a group-based line graph regularization term, defined by the similarity of pixel intensities located at the same location in all similar patch images, specifically:

是基于组的列图正则化项，利用位于每一个补丁图像对应所有位置的像素强度的相似性来定义，具体为：

is a group-based column map regularization term, defined by the similarity of pixel intensities at all locations corresponding to each patch image, specifically:

L _r and L _c are row Laplacian matrices and column Laplacian matrices, respectively;

(3) The optimization formula for constructing the core specification

Introducing low-order optimization processing, the conventional replacement of low-rank data matrix X is called kernel norm or tracking norm ||X|| _* , which is defined as follows:

||X|| _* ＝tr((XX ^T ) ^1/2 )＝∑ _k σ _k

Among them, σ _k is the singular value of X;

(4) Construct a group-based graphical model and a combined optimization formula for core norms

The specific definition is as follows:

Among them, θ _n , θ _r and θ _c are the control parameters of the kernel norm, row map and column map. It can be seen that the regularization term reflects the non-local self-similarity, and the kernel norm reflects the low-rank characteristics of images that can use a large amount of information .

Further, the KNN algorithm is used to optimize the combined optimization formula based on the grouping graphical model and the core specification in step (2), and the specific steps are as follows:

(1) Calculate the value of the optimization formula between the current patch image and all patch images;

(2) Arranged in ascending order according to the optimized value;

(3) Select the K patch images closest to the optimized value;

(4) Count the occurrence frequencies of the categories of the K patch images, and use the category with the highest frequency among the K patch images as the denoised result image.

Further, the step (3) is based on the deep learning model of non-negativity constrained sparse autoencoder (FFSAE) specifically:

Step (3)a: Encode the input data with the encoder

First, use a set of encoders θ _E = {W _E , b _E } to convert the input data into "compressed" image features; transform the input signal X _m ∈ R ^d into a hidden layer feature vector h _m ∈ by the following formula R ^s :

h _m =E(X _m ,θ _E )=sigm(W _E X _E +b _E )

Among them, θ _E represents the encoder parameters composed of weight matrix W _E and bias vector b _E , and the encoder is a nonlinear transformation function: E(): R ^d → R ^s (d>s);

Step (3)b: Define the cost function η _AE (W,b)

The average reconstruction error of all training samples is defined as the cost function η _AE (W, b), and the weight attenuation penalty term α is added at the same time, which is specifically defined as follows:

Among them, W={W _E , W _D }, b={b _E , b _D }, M is the number of training samples, and α is the regularization penalty item that controls the weight reduction;

Step (3)c: Build a sparse autoencoder

(1) Solve the average activation value of the hidden layer unit

Sparse autoencoders can be constructed by imposing sparsity on the hidden layer units of the autoencoder, and the sparse autoencoder expects the average activation value of each hidden layer unit to be close to zero; let [h _m ] _j be related to X _m The activation value of the jth hidden unit, the average activation value of the jth hidden layer unit on the entire training set is calculated as follows:

(2) Define the penalty item

强制执行，其中，ε为预定义的稀疏性参数，增加一个额外的惩罚项，用以惩罚明显偏离ε的

的情况，惩罚项定义为Kullback-Leibler(KL)散度，如下式所示：The sparsity constraint of sparse autoencoders is given by

Mandatory execution, where ε is a predefined sparsity parameter, and an additional penalty is added to punish those that deviate significantly from ε

In the case of , the penalty term is defined as the Kullback-Leibler (KL) divergence, as shown in the following formula:

是隐单元的平均激活向量，s是隐藏单元数，

是用于测量两个分布之间差异的标准函数；in,

is the average activation vector of hidden units, s is the number of hidden units,

is the standard function used to measure the difference between two distributions;

时，

可以达到最小的0值，且当

向上偏离ε时，会导致其无效，因此，最小化此惩罚项可以使得

接近于ε；Visible, when

hour,

can reach the smallest value of 0, and when

Deviates upward from ε, making it invalid, so minimizing this penalty makes

close to ε;

(3) Define the sparse cost function η _SAE (W, b)

由此，稀疏成本函数η_SAE(W，b)定义为：The training objective optimization function of the sparse autoencoder is to minimize the average reconstruction error η _AE (W, b) and the sparsity penalty term

Thus, the sparse cost function η _SAE (W, b) is defined as:

Among them, β is the weight used to control the sparsity penalty term; since ε represents the average activation of all hidden units, and the activation of hidden units depends on the parameters {W, b}, therefore, the ε term also depends on {W, b} ;

Step (3)d: Establish the cost function η _FFSAE (W, b) of the non-negative constrained standard autoencoder

A non-negative constrained self-encoder FFSAE is proposed, and the cost function is modified as η _FFSAE (W, b), as follows:

in,

和

Step (3)e: Utilize the gradient descent method to update

and

和

为接近零的随机值，然后应用梯度下降优化算法进行训练，在每次迭代中更新参数

和

具体如下式：Initialize the parameters first

and

is a random value close to zero, and then applies the gradient descent optimization algorithm for training, updating the parameters in each iteration

and

The specific formula is as follows:

Among them, λ>0 is the learning rate;

Step (3)f: Use the decoder to reconstruct the input signal

实现公式如下：Use the decoder θ _D ={W _D , b _D } to reversely restore the hidden features, thereby reconstructing the input signal. In specific implementation, the hidden feature vector h _m is reversed through the decoder D(): R ^s →R ^d is restored to a reconstruction vector with a similar structure

The implementation formula is as follows:

where _θD denotes the decoder parameters consisting of weight matrix _WD and bias vector _bD .

Further, in the step (4), when using the defect detection map to determine the defect type, it is necessary to define a performance evaluation index, and use the structural similarity measurement index SSI as the evaluation index to measure the structural information of a certain image relative to the structural information of another image. The degree of degradation is calculated as follows:

In the formula, μ and σ respectively represent the average value and variance of each pixel parameter; σ _xy is the covariance; c ₁ and c ₂ are constants to prevent the divisor from being zero.

Beneficial effect:

1. The present invention utilizes a non-negativity constrained sparse autoencoder (FFSAE) and a group-based graphical model and kernel norm image denoising method (GNN) to realize automatic and effective detection of electronic circuit defect types. By using an image sensor (industrial camera) to capture images of electronic circuits, use these images to train a deep autoencoder model, and decode the original non-defective image from the defective circuit image, and then compare the decoded circuit image with the input circuit image , so as to determine the defect location and realize the effective detection of circuit defects. Therefore, the method does not require prior knowledge of the defect type or the normal/defective evaluation criteria of the expert system, which can overcome the problem of small and unbalanced datasets in the early manufacturing stage.

2. The present invention can design a data set suitable for training through an appropriate preprocessing process, thereby improving the performance of the classification model. Complete the preprocessing techniques such as cropping and enhancement of the circuit image, and use the group-based graph model and the kernel norm method (GNN) to denoise the image. This method obtains the Laplacian graph matrix of the image by optimizing the learning strategy. The Laplacian matrix encapsulates the graph structure of the data matrix, so it can reflect the topological structure of the image, thereby ensuring its good performance in image smoothing and denoising. The application effect can further effectively improve the quality of the circuit image and ensure the establishment of a more complete defect data set. The invention uses a non-negativity constrained sparse autoencoder (FFSAE) to extract defect regions. The FFSAE algorithm emphasizes that the weight of neurons is non-negative, converts the input code into a coded form of low-dimensional space, and reconstructs the original data through decoding. This method allows to extract useful features from unlabeled data, and can reproduce the original input data through latent vector expansion. Based on the non-negative value characteristics of neuron weights, the FFSAE algorithm can improve the interpretability of the recognition network operation and enhance the learned features. Further, the reliability and accuracy of electronic circuit defect identification can be improved.

Description of drawings

Fig. 1 is the processing process of the electronic circuit defect detection method based on non-negativity constrained sparse autoencoder (FFSAE);

Figure 2 is a true defect image;

Figure 3 is a pseudo-defect image;

Fig. 4 is the electronic circuit image denoising process based on the graphical model of the group and the kernel specification method (GNN);

Fig. 5 is the image denoising process based on the graphical model of the group and the kernel specification;

Fig. 6 is the structure of automatic encoder in image denoising restoration application;

Figure 7 is a standard autoencoder structure;

Fig. 8 is an image of the experimental result of circuit defect detection based on FFSAE.

Detailed ways

In order to better explain the present invention and facilitate understanding, the technical solutions of the present invention are described in detail below. The following examples are for explanation of the present invention, but the present invention is not limited to the following examples.

The invention proposes an electronic circuit defect detection method based on a non-negativity constrained sparse autoencoder (FFSAE). The electronic circuit defect detection process has two main purposes, one is to preprocess the electronic circuit defect dataset. Since the amount of training data determines the performance of the defect detection classification model, through an appropriate preprocessing process, a data set suitable for training can be designed to improve the performance of the classification model. In response to this problem, this patent completes preprocessing technologies such as cropping and enhancement of circuit images, and uses a group-based graphical model and kernel norm method (GNN) to denoise the image. This method obtains the Laplacian graph matrix of the image by optimizing the learning strategy. The Laplacian matrix encapsulates the graph structure of the data matrix, so it can reflect the topological structure of the image, thereby ensuring its good performance in image smoothing and denoising. The application effect can further effectively improve the quality of the circuit image and ensure the establishment of a more complete defect data set. The second is to extract defect regions and correctly identify them. This patent uses a non-negativity constrained sparse autoencoder (FFSAE) to extract defect regions. The FFSAE algorithm emphasizes that the weight of neurons is non-negative, converts the input code into a coded form of low-dimensional space, and reconstructs the original data through decoding. This method allows to extract useful features from unlabeled data, and can reproduce the original input data through latent vector expansion. Based on the non-negative value characteristics of neuron weights, the FFSAE algorithm can improve the interpretability of the recognition network operation and enhance the learned features. The discriminability, further, can improve the reliability and accuracy of electronic circuit defect identification.

The entire electronic circuit defect detection process mainly includes the model training stage and the defect identification stage. In the model training stage, the image is firstly preprocessed, and the original electronic circuit image is divided into patch images with a size of 500×500 pixels through the image clipping operation, and then the noise suppression processing is performed on each image, and the group-based graph is adopted. Model and kernel norm method (GNN) to complete the image noise suppression to improve the quality of the original electronic circuit image dataset. Next, data augmentation is performed on the denoised electronic circuit image. In the data enhancement module, the enhancement of the image data set is completed by randomly rotating, flipping, and adding noise to the defect patch image to effectively overcome the problem of data class imbalance due to insufficient data. Finally, the enhanced patch image is used to train a non-negative constrained sparse autoencoder model (FFSAE) to predict the input non-defective patch image. After the model is trained successfully, it can enter the identification stage of electronic circuit defects. Send the defective electronic circuit image for detection into the model that has been successfully trained. Once a high-quality image is predicted from the defective electronic circuit image from the trained model, the defect input image can be subtracted from the predicted image to generate a defect Detection map. Finally, by properly setting the threshold of the defect detection map, the position of the defect is highlighted, and the effective and intuitive identification of the defect is completed.

Fig. 1 shows the processing process of the proposed electronic circuit defect detection method based on non-negativity constrained sparse autoencoder (FFSAE) in the embodiment of the present invention. In conjunction with Fig. 1, a non-negativity constraint-based The electronic circuit defect detection method of the sparse self-encoder (FFSAE) specifically includes the following steps:

Step 1, realize the preprocessing of the electronic circuit defect image. Specific steps are as follows:

Step (1)a: Collect and crop the electronic circuit image, and determine the electronic circuit defect data set

In order to verify the effectiveness of the electronic circuit defect detection method, it is necessary to create an electronic circuit defect image dataset. In the implementation, ten reference electronic circuit image data sets were collected, each data set was captured by a megapixel high-definition industrial camera equipped with a CMOS sensor. The original image is 4608*3456 pixels, and the data set contains 600 electronic circuit board defect images, which can be divided into color images and grayscale images. Since the electronic circuit images collected by industrial-grade cameras are high-resolution images, the amount of data to process these images is large, and the amount of calculation for processing is very large, which will result in a long processing time for training the classification model. In order to improve its computing power, in the preprocessing, the size of each electronic circuit is adjusted, and the electronic circuit image is cropped, and the defect area is cropped into a patch image with a size of 500×500.

Step (1)b: Determine the type of electronic circuit defect

After the electronic circuit image is collected, it is necessary to manually process the defect image to determine the type of defect existing in the electronic circuit board. In implementation, two kinds of defects are set, one is true defect and the other is false defect. Among them, the true defect is really a defect caused by the change of the shape of the lead wire. The image of the true defect is shown in Figure 3, and it is specifically set to be a disconnection defect (see Figure 2(a)), a connection defect (see Figure 2(b)), and a protruding defect. defects (see Figure 2(c)) and crack defects (see Figure 2(d)). Pseudo-defects are characterized by only color changes, while the shape characteristics of leads and basic components remain unchanged. The definition image of pseudo-defects is shown in Figure 3, which is specifically divided into oxidation defects (see Figure 3(a)) and dust defects (see Figure 3(b) )).

Step (1)c: Complete image data enhancement and feature selection

Typically, training deep classification algorithms requires large-scale training data. However, in the manufacturing process of electronic circuits, the probability of circuit defects is usually very small, and, in the mass production of electronic circuits, the types of defects will also change. This data imbalance is a basic problem that limits the application of electronic circuit defect detection systems. If these imbalanced data are applied to deep defect detection models, it will lead to problems such as overfitting and performance degradation. In order to avoid these problems, we apply Data augmentation to supplement small amounts of defective data to improve model performance. Considering the redundancy of the enhanced image, the data enhancement is completed by applying geometric transformation and adding noise. Geometric transformations are methods that effectively use shape, orientation, or position of part features, applying random rotations to overcome positional deviations in image data. Adding noise is adding noise to the original image, multiplying a random matrix with noise distribution with the original data. Through the above data enhancement techniques, it can help the classification model to learn more robust features.

After preprocessing, image features need to be extracted from defect candidate regions, and sent to non-negative constrained autoencoder (FFSAE) for learning and classification. In implementation, the determined feature parameters are color information and shape information respectively. Among them, the first one is color information, a total of 30 kinds, the following features are extracted from the RGB and HSV color models, respectively (1) maximum value, (2) minimum value, (3) average value, (4) ratio High value, (5) ratio of lead area to candidate area and lead, (6) ratio of base part to candidate part, (7) position difference between numerical center of gravity and maximum value, (8) variance, (9) standard Bias, (10) kurtosis, (11) skewness, (12) entropy, (13) difference between maximum and minimum, (14) median, (15) between test image and reference image relevance. The second is shape information, including 8 types in total, including (1) area, (2) perimeter, (3) x-direction size, (4) y-direction size, (5) aspect ratio, (6) diagonal Line length, (7) complexity, (8) roundness.

In step (2), the electronic circuit image denoising based on the group graph model and the kernel norm method (GNN) is completed.

In image denoising, an image denoising method based on group graph model and kernel specification (GNN) is proposed. First, the original noisy image is converted into a patch image vector by group sampling, and then processed by GNN. Construct a group-based dual graph and kernel norm optimization formula, use the KNN algorithm to search for similar patches, that is, search for m patch images similar to the original image by performing block matching on each noisy patch image, and finally combine all similar patches Patch images are stacked to build a group-based graph model. In constructing the grouped graph model, a Laplacian matrix graph strategy is proposed. Firstly, the learning strategy is established by optimizing the formula, and then the Laplacian graph matrix is obtained by using the optimized learning strategy. The Laplacian weighted matrix graph regards each pixel in the data matrix as a node, and explores the similarity between pixels in the patch image. Therefore, it can reflect the topological structure of the image, achieve effective smoothing of the image, and further enhance the electronic circuit image denoising smoothing effect.

As shown in Figure 4, the electronic circuit image denoising process based on the group graph model and the kernel norm method (GNN), the specific steps are as follows:

Step (2)a: Use the Laplacian matrix L to characterize the collected patch image

(1) Obtain the weighted adjacency matrix W according to the image data

An undirected weighted graph can be represented by G=(V, E, W). Among them, V is a vertex set composed of N vertices, E is an edge set composed of multiple V×N weighted edges, and W is a weighted adjacency matrix. The weighted adjacency matrix W reflects the similarity between vertices v _i and v _j , in general, the undirected weighted weighted adjacency matrix W graph is non-negative and has equal diagonal elements, i.e. W _ij = w _ji , W _ij ≥0.

Based on this, the threshold Gaussian kernel is used to form the weight matrix W of the edge, as follows:

是在图像顶点v_i和v_j之间的欧几里得距离，σ是控制权重随距离增加而衰减的速度控制参数，ε是阈值参数，代表ε-邻域图。in,

is the Euclidean distance between image vertices v _i and v _j , σ is a speed control parameter that controls weight decay with increasing distance, and ε is a threshold parameter, representing the ε-neighborhood graph.

(2) Obtain the image L represented by the Laplacian matrix

The Laplacian matrix L plays a crucial role in describing the characteristics of graph data. L encapsulates the graph structure of the data matrix. In an undirected weighted graph, the calculation method of L is determined by W, as follows:

L=Δ-W (2)

Wherein, Δ is a diagonal matrix, which satisfies the equation Δ _ii =∑ _j W _ij .

Step (2)b: Establish a grouping-based graphical model and a combined optimization formula for kernel norms

(1) Construct the basic optimization formula

是与拉普拉斯矩阵关联的正则化项。则基于图像去噪的基本优化公式如下：set up

is the regularization term associated with the Laplacian matrix. The basic optimization formula based on image denoising is as follows:

Among them, both x and y are n×1 vectors representing image blocks, L is an n×n Laplacian matrix, and θ is a regularization parameter.

(2) Construct an optimization formula based on grouped dual graph

列的欧几里得距离函数为

因此，利用W^m×m和W^n×n可以获得行拉普拉斯矩阵L_r和列拉普拉斯矩阵L_c。In order to construct a dual graph, the traditional method is to regard each data matrix X ^m×n as an n-dimensional column vector X=(x ₁ ,…,x _n ) or an m-dimensional row vector X=((x′ ₁ ) ^T ,...,(x′ _m ) ^T ) ^T . Therefore, each vector (each column or each row) in the matrix is regarded as a node for computing the weighted adjacency matrix W. Then the Euclidean distance function of the rows of the weighted adjacency matrix W ^m×m is

The Euclidean distance function for a column is

Therefore, a row Laplacian matrix L _r and a column Laplacian matrix L _c can be obtained using W ^m×m and W ^n×n .

Considering that each group is a matrix, that is, to construct a dual graph T ^m×n including row graphs and column graphs, the optimal expression for defining the dual graph model is as follows:

和列图

Among them, X and Y are m×n image row and column data matrices, θ _r and θ _c are regularization control parameters, which are used to determine the degree of influence of the regularization term, that is, the row map

and column diagram

是基于组的行图正则化项，利用位于所有相似补丁图像的同一位置的像素强度的相似性来定义，具体为：

is a group-based line graph regularization term, defined by the similarity of pixel intensities located at the same location in all similar patch images, specifically:

是基于组的列图正则化项，利用位于每一个补丁图像对应所有位置的像素强度的相似性来定义，具体为：

is a group-based column map regularization term, defined by the similarity of pixel intensities at all locations corresponding to each patch image, specifically:

L _r and L _c are row Laplacian matrices and column Laplacian matrices, respectively.

(3) The optimization formula for constructing the core specification

Considering the strong low-rank nature of grouped images, a low-order optimization process is further introduced. The conventional replacement for low-rank data matrix X is called kernel norm or trace norm ||X|| _* , which is defined as follows:

||X|| _* ＝tr((XX ^T ) ^1/2 )＝∑ _k σ _k (7)

Among them, σ _k is the singular value of X.

(4) Construct a group-based graphical model and a combined optimization formula for core norms

The specific definition is as follows:

Among them, θ _n , θ _r and θ _c are the control parameters of kernel norm, row plot and column plot. In (7), the regularization term reflects the non-local self-similarity, and the kernel norm reflects the low-rank properties of images that can use a lot of information.

Step (2)c: use the KNN algorithm to optimize the solution, the specific steps are as follows:

(1) Calculate the value of the optimization formula between the current patch image and all patch images;

(2) Arranged in ascending order according to the optimized value;

(3) Select the K patch images closest to the optimized value;

(4) Count the occurrence frequencies of the categories of the K patch images, and use the category with the highest frequency among the K patch images as the denoised result image.

Figure 5 is the image denoising process based on the group graph model and kernel specification.

In step (3), implement a deep learning model based on non-negativity constrained sparse autoencoder (FFSAE), and use it to extract defect regions.

An autoencoder is a form of encoding that aims to transform an input encoding into a low-dimensional space that can be reconstructed from the original data by a decoder. The method allows extracting useful features from unlabeled data, which reproduces the original input data through latent vector expansion. It is usually used in data compression, denoising, anomaly detection, image restoration, etc.

Figure 6 is the structure of the autoencoder in the application of image denoising and restoration.

在具体实现时，自动编码器通过强制隐藏单元数量小于输入维度，从而可以保证网络学习输入数据的压缩特征，利用该压缩特征即可发现输入数据中的特征结构。标准自编码器由编码器和解码器两部分组成。Autoencoder networks are an unsupervised learning algorithm. Essentially, the autoencoder learns the function M _{W, b(X)} ≈ X, in other words, by learning the estimated value of the feature function, it is possible to obtain a function similar to X

In the specific implementation, the autoencoder can ensure that the network learns the compressed features of the input data by forcing the number of hidden units to be smaller than the input dimension, and the feature structure in the input data can be discovered by using the compressed features. A standard autoencoder consists of two parts: an encoder and a decoder.

Figure 7 is a standard autoencoder structure. In the implementation of this patent, a non-negativity constrained sparse autoencoder (FFSAE) is proposed to extract defect regions and correctly identify them. The FFSAE algorithm emphasizes that the weight of neurons is non-negative, converts the input code into a coded form of low-dimensional space, and reconstructs the original data through decoding. This method allows to extract useful features from unlabeled data, and can reproduce the original input data through latent vector expansion. Based on the non-negative value characteristics of neuron weights, the FFSAE algorithm can improve the interpretability of the recognition network operation and enhance the learned features. The discriminability, further, can improve the reliability and accuracy of electronic circuit defect identification. The specific implementation process of the FFSAE algorithm is as follows:

Step (3)a: Encode the input data with the encoder

First, the input data is converted into a "compressed" representation of image features using a set of encoders θ _E = {W _E , b _E }. The encoder can be understood as a nonlinear transformation function: E(): R ^d → R ^s (d>s), transforming the input signal X _m ∈ R ^d into the hidden layer feature vector h _m ∈ R ^s by formula (9):

h _m =E(X _m , θ _E )=sigm(W _E X _E +b _E ) (9)

Among them, θ _E represents the encoder parameters composed of weight matrix W _E and bias vector b _E.

Step (3)b: Define the cost function η _AE (W,b)

The essence of the autoencoder is to learn the image compression characteristics in the hidden layer, and use the minimum average error in all training samples to reconstruct the input. Therefore, the average reconstruction error of all training samples is defined as the cost function η _AE (W, b), and at the same time, the weight decay penalty term α is added to minimize the risk of overfitting and improve the generalization ability of the algorithm. The specific definition is as formula (10):

Among them, W={W _E , W _D }, b={b _E , b _D }, M is the number of training samples, and α is the regularization penalty item that controls the weight reduction.

Step (3)c: Build a sparse autoencoder

(1) Solve the average activation value of the hidden layer unit

Sparse autoencoders can be constructed by imposing sparsity on the hidden layer units of the autoencoder, and the sparse autoencoder expects the average activation value of each hidden layer unit to be close to zero. Let [h _m ] _j be the activation value of the jth hidden unit related to X _m , then the average activation value of the jth hidden layer unit on the entire training set is calculated as formula (11):

(2) Define the penalty item

强制执行。其中，ε为预定义的稀疏性参数，通常为接近0的较小值(例如0.05)。为了满足稀疏性约束，隐层单位的激活必须大多接近于零。为实现这一点，增加一个额外的惩罚项，用以惩罚明显偏离ε的

的情况，惩罚项定义为Kullback-Leibler(KL)散度，如(12)式所示：The sparsity constraint of sparse autoencoders is given by

enforced. Wherein, ε is a predefined sparsity parameter, usually a small value close to 0 (for example, 0.05). To satisfy the sparsity constraint, the activations of hidden units must be mostly close to zero. To achieve this, an additional penalty term is added to penalize significant deviations from ε

In the case of , the penalty term is defined as the Kullback-Leibler (KL) divergence, as shown in formula (12):

是隐单元的平均激活向量，s是隐藏单元数。

是用于测量两个分布之间差异的标准函数。in,

is the average activation vector of hidden units, and s is the number of hidden units.

is the standard function used to measure the difference between two distributions.

时，

可以达到最小的0值，且当

向上偏离ε时，会导致其无效，因此，最小化此惩罚项可以使得

接近于ε。Visible, when

hour,

can reach the smallest value of 0, and when

Deviates upward from ε, making it invalid, so minimizing this penalty makes

close to ε.

(3) Define the sparse cost function η _SAE (W, b)

由此，稀疏成本函数η_SAE(W，b)定义为(13)式：The training objective optimization function for sparse autoencoders is to minimize the average reconstruction error η _AE (W,b) in Equation (10) and the sparsity penalty in Equation (12)

Thus, the sparse cost function η _SAE (W, b) is defined as (13) formula:

where β is the weight used to control the sparsity penalty. It can be seen that since ε represents the average activation of all hidden units, and the activation of hidden units depends on the parameters {W, b}, therefore, the ε term also depends on {W, b}.

Step (3)d: Establish the cost function η _FFSAE (W,b) of the non-negative constrained standard autoencoder (FFSAE)

When the weights of reinforced neurons are non-negative, the interpretability of network operation can be improved, and the discriminability of learned features can also be enhanced. Therefore, we propose a non-negative constrained autoencoder (FFSAE). In order to realize the non-negativity of the weights, the cost function in the modified formula (13) is η _FFSAE (W, b), as shown in the formula (14):

in,

The goal of FFSAE training is to minimize η _FFSAE (W,b) as a function of W and b. According to formula (15), the penalty value assigned to negative weights is the square value of the corresponding item, while the weights assigned to non-negative values are all 0. Therefore, minimizing the cost function η _FFSAE (W,b) can better reduce the number of negative weights. In addition, as the regularization term η _FFSAE (W,b) of the non-negative constrained autoencoder (FFSAE), Equation (14) will simultaneously reduce the reconstruction error, encourage the learning of sparse features, and reduce the non-negative weight. quantity.

和

Step (3)e: Utilize the gradient descent method to update

and

和

为接近零的随机值，然后应用梯度下降优化算法进行训练，在每次迭代中更新参数

和

具体如(16)、(17)式：In order to achieve the above optimization goals, first initialize the parameters

and

is a random value close to zero, and then applies the gradient descent optimization algorithm for training, updating the parameters in each iteration

and

Specifically, such as formulas (16) and (17):

Among them, λ>0 is the learning rate.

Step (3)f: Use the decoder to reconstruct the input signal

实现公式如下：Utilize the decoder θ _D ={W _D , b _D } to inversely recover the hidden features, thereby reconstructing the input signal. In specific implementation, through the decoder D(): R ^s → R ^d , the hidden feature vector h _m is reversely restored to a reconstruction vector with a similar structure

The implementation formula is as follows:

where _θD denotes the decoder parameters consisting of weight matrix _WD and bias vector _bD .

Step (4), generating a defect detection map.

In the proposed method, defect detection maps need to be generated at the end to obtain the correct defect types. In the specific implementation, the classification model is used to generate a defect-free output image, and then the input image is subtracted from it to obtain the defect detection map. This type of image subtraction is a numerical calculation that subtracts the value of another image from the value of the entire image. With this approach, we can detect changes between the two images and use them in the identification of circuit defects. Specific steps are as follows:

Step (4)a: Predict the circuit image

Send the defective electronic circuit image for detection into the self-encoder (FFSAE) model that has been trained successfully, and predict a high-quality circuit image from the defective circuit image through the training model.

Step (4)b: Generate defect detection map

This predicted image is subtracted from the original defect input image to generate a defect detection map. Finally, the defect position is highlighted by setting an appropriate threshold value on the defect detection map, so as to complete the correct classification of electronic circuit defect types.

When using the defect detection map to determine the defect type, it is necessary to define the performance evaluation index. This performance metric is used to measure how similar the predicted image is to the target image. This patent uses the Structural Similarity Measurement Index (SSI) as an evaluation index to measure the degree of degradation of an image's structural information relative to another image's structural information. SSI calculates the similarity of two images by comparing brightness, contrast, and structure. . The specific calculation is as follows:

In the formula, μ and σ respectively represent the average value and variance of each pixel parameter; σ _xy is the covariance; c ₁ and c ₂ are constants to prevent the divisor from being zero.

Step 5. Based on the above technologies, build an experimental platform to complete the specific implementation of motor fault classification, and mainly complete the following test experiments:

(1) Circuit defect detection experiment based on FFSAE

Randomly sample and generate subsets in the electronic circuit image data set, randomly select color features and shape features, and send them to the non-negative constrained autoencoder (FFSAE), and finally obtain the detection effect as shown in Figure 8. It can be seen that the method can detect samples with large defect color changes with high precision, such as broken wires and oxidation defects. In addition, the proposed method can also detect dust pseudo-defects with small color changes.

(2) Comparison experiment between real defect and false defect detection

To verify the correctness of the proposed method, FFSAE is compared with conventional algorithms (SVM, BP, RBF). The electronic circuit dataset used includes a total of 500 defect images, of which 300 are true defects and 200 are false defects. Each method uses gray and color images. Take the 15 features corresponding to the brightness value of the grayscale image and RGB of the color image. Finally, the test results of real defects and false defects of each method are shown in Table 1. During the testing process, due to the randomness of each subset and feature selection, this method is executed 10 times, and the average value is taken as the final result.

Table 1 Classification results of different methods

The results in Table 1 show that, compared with the existing methods, this method gives better discrimination results of real defects and false defects.

(3) Comparison experiment of defect detection between color image and gray image

In the experiment, 200 color defect images and 200 gray defect images were selected, and different methods were used to measure the defects, and the experimental results in Table 2 were obtained.

Table 2 Different image classification results

Table 2 shows that when the color image is used alone, the measurement results are better than those in Table 1, and when the gray image is used alone, the measurement results are worse than those in Table 1. On the other hand, overall, the correct ratios of all methods are improved when using color images compared to grayscale images. That is to say, the effectiveness of color images is reflected in the combination of features represented by multiple colors, such as ratios, entropy in RGB, correlation between test images and reference images, etc. The combination of colors is better for defect classification, which further explains The important role of color images in defect measurement and classification.

The above-mentioned embodiments are only for illustrating the technical concept and characteristics of the present invention, and its purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and not to limit the scope of protection of the present invention. All equivalent changes or modifications made according to the spirit of the present invention shall fall within the protection scope of the present invention.

Claims (6)

1. An electronic circuit defect detection method based on non-negativity constraint sparse autoencoder, is characterized in that, comprises the following steps: (1) Preprocessing the electronic circuit defect data set; collecting and tailoring the electronic circuit image, determining the electronic circuit defect data set and defect type, and completing image data enhancement and feature selection; (2) Complete the electronic circuit image denoising based on the group-based graph model and kernel norm method GNN, and use the Laplacian graph matrix and kernel norm of the image to encapsulate the graph structure; (3) Construct a deep learning model based on non-negativity constrained sparse autoencoder FFSAE, and use it in the extraction of defect regions; (4) Generate a defect detection map, use the successfully trained self-encoder FFSAE model to predict a high-quality circuit image, and subtract the original defect input image from the predicted image to generate a defect detection map, and finally set the defect detection map appropriately The method of threshold value is used to highlight the defect location, so as to complete the correct classification of electronic circuit defect types. 2. The electronic circuit defect detection method of the non-negativity constrained sparse autoencoder according to claim 1, characterized in that, in the step (1), the specific steps of preprocessing the electronic circuit defect data set are as follows: Step (1)a: collecting and clipping the electronic circuit image, and determining the electronic circuit defect data set; Step (1)b: Determine the type of defect in the electronic circuit, and set two kinds of defects, one is true defect and the other is false defect; among them, the true defect is really caused by the change of lead wire shape, specifically set as Disconnection defects, connection defects, protruding defects, and crack defects; false defects are characterized by only color changes, while the shape characteristics of leads and basic components remain unchanged, and are specifically divided into oxidation defects and dust defects; Step (1)c: Complete image data enhancement and feature selection Data enhancement: apply geometric transformation and adding noise to complete data enhancement: apply random rotation to overcome the position deviation of image data; multiply a random matrix with noise distribution with the original data; Feature selection: Determine the feature parameters as color information and shape information respectively. Among them, there are 30 kinds of color information, and the following features are extracted from RGB and HSV color models respectively, which are 1) maximum value, 2) minimum value, and 3) average value. value, 4) scale high value, 5) ratio of lead area to candidate area and lead, 6) ratio of base part to candidate part, 7) position difference between numerical center of gravity and maximum value, 8) variance, 9) standard Bias, 10) kurtosis, 11) skewness, 12) entropy, 13) difference between maximum and minimum, 14) median, 15) correlation between test image and reference image; shape information A total of 8 types are included, including 1) area, 2) perimeter, 3) size in x direction, 4) size in y direction, 5) aspect ratio, 6) diagonal length, 7) complexity, and 8) roundness. 3. the electronic circuit defect detection method based on non-negativity constrained sparse self-encoder according to claim 1, is characterized in that, in described step (2), finish the circuit image based on group graph model and kernel specification method GNN Denoising, the specific steps are as follows: Step (2)a: Use the Laplacian matrix L to characterize the collected patch image (1) Obtain the weighted adjacency matrix W according to the image data The undirected weighted weighted adjacency matrix W graph is non-negative and has equal diagonal elements, that is, W _ij = w _ji , W _ij ≥ 0, and the threshold Gaussian kernel is used to form the weight matrix W of the edge, as follows:

in,

is the Euclidean distance between image vertices v _i and v _j , σ is a speed control parameter that controls weight decay with increasing distance, ε is a threshold parameter, representing the ε-neighborhood graph; (2) Obtain the image L represented by the Laplacian matrix L=Δ-W Among them, Δ is a diagonal matrix, which satisfies the equation Δ _ii = ∑ _j W _ij ; Step (2)b: Establish a grouping-based graphical model and a combined optimization formula for kernel norms (1) Construct the basic optimization formula set up

is the regularization term associated with the Laplacian matrix, then the basic optimization formula based on image denoising is as follows:

Among them, x and y are both n×1 vectors representing image blocks, L is an n×n Laplacian matrix, and θ is a regularization parameter; (2) Construct an optimization formula based on grouped dual graph Considering that each group is a matrix, that is, to construct a dual graph T ^m×n including row graphs and column graphs, the optimal expression for defining the dual graph model is as follows:

Among them, X and Y are m×n image row and column data matrices, θ _r and θ _c are regularization control parameters, which are used to determine the degree of influence of the regularization term, that is, the row map

and column diagram

is a group-based line graph regularization term, defined by the similarity of pixel intensities located at the same location in all similar patch images, specifically:

is a group-based column map regularization term, defined by the similarity of pixel intensities at all locations corresponding to each patch image, specifically:

L _r and L _c are row Laplacian matrices and column Laplacian matrices, respectively; (3) The optimization formula for constructing the core specification Introducing low-order optimization processing, the conventional replacement of low-rank data matrix X is called kernel norm or tracking norm ||X|| _* , which is defined as follows: ||X|| _* ＝tr((XX ^T ) ^1/2 )＝∑ _k σ _k Among them, σ _k is the singular value of X; (4) Construct a group-based graphical model and a combined optimization formula for core norms The specific definition is as follows:

Among them, θ _n , θ _r and θ _c are the control parameters of the kernel norm, row map and column map. It can be seen that the regularization term reflects the non-local self-similarity, and the kernel norm reflects the low-rank characteristics of images that can use a large amount of information . 4. the electronic circuit defect detection method based on non-negativity constrained sparse self-encoder according to claim 3, is characterized in that, utilizes KNN algorithm to the combined optimization formula based on the graphical model of grouping and core specification in step (2) Perform optimization processing to solve the problem, the specific steps are as follows: (1) Calculate the value of the optimization formula between the current patch image and all patch images; (2) Arranged in ascending order according to the optimized value; (3) Select the K patch images closest to the optimized value; (4) Count the occurrence frequencies of the categories of the K patch images, and use the category with the highest frequency among the K patch images as the denoised result image. 5. the electronic circuit defect detection method based on non-negativity constrained sparse autoencoder according to claim 1, is characterized in that, described step (3) is based on the deep learning of non-negativity constrained sparse autoencoder (FFSAE) The model is specifically: Step (3)a: Encode the input data with the encoder First, use a set of encoders θ _E = {W _E , b _E } to convert the input data into "compressed" image features; transform the input signal X _m ∈ R ^d into a hidden layer feature vector h _m ∈ by the following formula R ^s : h _m =E(X _m ,θ _E )=sigm(W _E X _E +b _E ) Among them, θ _E represents the encoder parameters composed of weight matrix W _E and bias vector b _E , and the encoder is a nonlinear transformation function: E(): R ^d → R ^s (d>s); Step (3)b: Define the cost function η _AE (W,b) The average reconstruction error of all training samples is defined as the cost function η _AE (W, b), and the weight attenuation penalty term α is added at the same time, which is specifically defined as follows:

Among them, W={W _E , W _D }, b={b _E , b _D }, M is the number of training samples, and α is the regularization penalty item that controls the weight reduction; Step (3)c: Build a sparse autoencoder (1) Solve the average activation value of the hidden layer unit

Sparse autoencoders can be constructed by imposing sparsity on the hidden layer units of the autoencoder. The sparse autoencoder expects the average activation value of each hidden layer unit to be close to _zero ; let [h _m ] _j be the The activation value of the jth hidden unit, the average activation value of the jth hidden layer unit on the entire training set is calculated as follows:

(2) Define the penalty term

The sparsity constraint of sparse autoencoders is given by

Mandatory execution, where ε is a predefined sparsity parameter, and an additional penalty is added to punish those that deviate significantly from ε

In the case of , the penalty term is defined as the Kullback-Leibler (KL) divergence, as shown in the following formula:

in,

is the average activation vector of hidden units, s is the number of hidden units,

is the standard function used to measure the difference between two distributions; Visible, when

hour,

can reach the smallest value of 0, and when

Deviates upward from ε, making it invalid, so minimizing this penalty makes

close to ε; (3) Define the sparse cost function η _SAE (W, b) The training objective optimization function of the sparse autoencoder is to minimize the average reconstruction error η _AE (W, b) and the sparsity penalty term

Thus, the sparse cost function η _SAE (W, b) is defined as:

Among them, β is the weight used to control the sparsity penalty term; since ε represents the average activation of all hidden units, and the activation of hidden units depends on the parameters {W, b}, therefore, the ε term also depends on {W, b} ; Step (3)d: Establish the cost function η _FFSAE (W, b) of the non-negative constrained standard autoencoder A non-negative constrained self-encoder FFSAE is proposed, and the cost function is modified as η _FFSAE (W, b), as follows:

in,

Step (3)e: Utilize the gradient descent method to update

and

Initialize the parameters first

and

is a random value close to zero, and then applies the gradient descent optimization algorithm for training, updating the parameters in each iteration

and

The specific formula is as follows:

Among them, λ>0 is the learning rate; Step (3)f: Use the decoder to reconstruct the input signal Use the decoder θ _D ={W _D , b _D } to restore the hidden features in reverse, so as to reconstruct the input signal. In the specific implementation, the hidden feature vector h _m is reversed through the decoder D(): R ^s →R ^d is restored to a reconstruction vector with a similar structure

The implementation formula is as follows:

where _θD denotes the decoder parameters consisting of weight matrix _WD and bias vector _bD . 6. The electronic circuit defect detection method based on non-negativity constrained sparse autoencoder according to claim 1, characterized in that, the step (4) needs to define the performance evaluation when using the defect detection map to determine the defect type Index, using the Structural Similarity Measurement Index (SSI) as an evaluation index to measure the degree of degradation of a certain image structure information relative to another image structure information, the specific calculation is as follows:

In the formula, μ and σ respectively represent the average value and variance of each pixel parameter; σ _xy is the covariance; c ₁ and c ₂ are constants to prevent the divisor from being zero.

Images