CN108961236B - Circuit board defect detection method and device - Google Patents

Circuit board defect detection method and device Download PDF

Info

Publication number
CN108961236B
CN108961236B CN201810701246.XA CN201810701246A CN108961236B CN 108961236 B CN108961236 B CN 108961236B CN 201810701246 A CN201810701246 A CN 201810701246A CN 108961236 B CN108961236 B CN 108961236B
Authority
CN
China
Prior art keywords
circuit board
feature
training sample
sample pair
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810701246.XA
Other languages
Chinese (zh)
Other versions
CN108961236A (en
Inventor
王健
杜家鸣
李长升
陈进宝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guoxin Youe Data Co Ltd
Original Assignee
Guoxin Youe Data Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guoxin Youe Data Co Ltd filed Critical Guoxin Youe Data Co Ltd
Priority to CN201810701246.XA priority Critical patent/CN108961236B/en
Publication of CN108961236A publication Critical patent/CN108961236A/en
Application granted granted Critical
Publication of CN108961236B publication Critical patent/CN108961236B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer

Abstract

The application provides a training method and a device, a detection method and a device of a circuit board defect detection model, wherein the training method comprises the following steps: obtaining at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images; inputting each training sample pair in at least one acquired training sample pair into an inlet corresponding to each branch of a neural network with two branches, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network; comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with the labeled value of the corresponding training sample pair; performing the training of the neural network according to the matching result; and obtaining a circuit board defect detection model through multiple rounds of training of the neural network. This application is through the circuit board defect detection model who trains out, can automated inspection circuit board whether have the defect, and the fault-tolerant rate and the rate of accuracy that detect are all higher.

Description

Circuit board defect detection method and device
Technical Field
The application relates to the technical field of circuit board detection, in particular to a training method and device and a detection method and device of a circuit board defect detection model.
Background
A Printed Circuit Board (PCB) serves as a carrier for electrical connection of electronic components, and implements a specific Circuit function by mounting a plurality of components thereon and a predetermined logic Circuit connection relationship between the components. Considering that various conditions may cause neglected mounting, multiple mounting and the like of components in the production process or the assembly process of the PCB, the PCB cannot work or operate normally. The PCB is used as a carrier of the electronic product, and the PCB which can not work normally or run can only ensure the performance of the electronic product if detected.
The earliest methods of defect detection of PCBs relied on manual visual inspection, which was performed by an operator visually inspecting the PCBs for defects with the aid of a magnifying glass or a calibrated microscope. It can be known that the method completely depends on the detection result of naked eyes, and is time-consuming, labor-consuming and poor in automation degree. In order to solve the above technical problems, the related art provides a method for detecting defects of a PCB based on an automatic optical detection means, which processes an acquired image of the PCB by various image processing methods (such as graying, binary processing, feature extraction, feature detection, etc.), and automatically detects whether the PCB has defects according to a matching result between the processed image and a template.
However, in the method for detecting the defects of the PCB based on the automatic optical detection method, different matching templates need to be set for different error types, and the error types without the matching templates cannot be effectively detected, so that the fault tolerance of the detection is low.
Disclosure of Invention
In view of this, an object of the embodiments of the present application is to provide a method and an apparatus for training a circuit board defect detection model, and a method and an apparatus for detecting a circuit board defect, so as to improve the accuracy and the fault tolerance of defect detection.
In a first aspect, an embodiment of the present application provides a training method for a circuit board defect detection model, including:
obtaining at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images;
inputting each training sample pair in at least one acquired training sample pair into inlets corresponding to branches of a neural network with two branches respectively, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with a labeled value of a corresponding training sample pair; and are
Performing the training of the neural network according to the matching result;
and obtaining a circuit board defect detection model through multi-round training of the neural network.
With reference to the first aspect, the present application provides a first possible implementation of the first aspect, wherein the neural network comprises a twin siense network;
inputting each training sample pair in at least one acquired training sample pair into an inlet corresponding to each branch of a neural network with two branches, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively, wherein the method comprises the following steps:
inputting each training sample pair in at least one obtained training sample pair into two inlets of a Siamese network respectively, and determining a first feature vector and a second feature vector for the input training sample pairs by using the Siamese network respectively;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with a labeled value of a corresponding training sample pair; and performing the training of the neural network according to the matching result, including:
performing the following matching operation until the comparison result of the first characteristic vector and the second characteristic vector is consistent with the labeled value of the corresponding training sample pair, and finishing the training of the current round; wherein the labeled values represent similarity or dissimilarity of corresponding training sample pairs;
the matching operation comprises:
comparing the first feature vector with the second feature vector to obtain a comparison result;
matching the comparison result with the labeled value of the corresponding training sample pair;
generating first feedback information aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result; and are
Performing parameter adjustment on the Siamese network based on the first feedback information;
based on the adjusted parameters, new first and second feature vectors are determined for corresponding pairs of training samples using the siemese network, and the matching operation is performed again.
With reference to the first aspect, the present application provides a second possible implementation of the first aspect, wherein the neural network comprises a two-channel neural network;
inputting each training sample pair in at least one acquired training sample pair into an inlet corresponding to each branch of a neural network with two branches, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively, wherein the method comprises the following steps:
respectively taking each training sample pair in at least one acquired training sample pair as source data and target data, and respectively inputting the source data and the target data into different channels of the two-channel neural network;
determining a first feature and a second feature for an input training sample pair using the two-channel neural network, respectively;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with a labeled value of a corresponding training sample pair; and performing the training of the neural network according to the matching result, including:
performing the following distance determination operation until the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the labeled value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the labeled value representation of the corresponding source data is not similar to the corresponding training sample pair is met, and finishing the training of the current round;
the distance determining operation comprising:
determining a distance between the first feature and the second feature;
generating second feedback information aiming at the condition that the distance between the first feature and the second feature is larger than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is not larger than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair; and are
Performing parameter adjustment on the two-channel neural network based on the second feedback information;
based on the adjusted parameters, new first and second features are determined for corresponding pairs of training samples using the two-channel neural network, and the distance determination operation is performed again.
In a second aspect, the present application further provides a method for detecting defects of a circuit board, including:
inputting a circuit board image to be detected and a corresponding standard circuit board image into a circuit board defect detection model obtained by training by adopting the training method in any one of the first aspect, the first possible implementation manner of the first aspect and the second possible implementation manner of the first aspect;
extracting a third feature and a fourth feature for the circuit board image to be detected and the corresponding standard circuit board image respectively by using the circuit board defect detection model; and are
Comparing the third feature to the fourth feature;
and determining whether the circuit board to be detected has defects according to the obtained comparison result.
With reference to the second aspect, the present application provides a first possible implementation manner of the second aspect, wherein the following method is adopted to determine a standard circuit board image corresponding to the circuit board image to be detected:
determining first identification information of components included in the circuit board image to be detected aiming at each circuit board image to be detected;
comparing the first identification information with second identification information of each component included in the corresponding standard circuit board full graph, and determining the region of the matched second identification information corresponding to the component in the standard circuit board full graph;
performing traversal operation on the area determined in the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
With reference to the second aspect, the present application provides a second possible implementation manner of the second aspect, wherein the following method is adopted to determine a standard circuit board image corresponding to the circuit board image to be detected:
performing traversal operation on the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
In a third aspect, the present application further provides a training apparatus for a circuit board defect detection model, including:
the sample acquisition module is used for acquiring at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images;
the characteristic determining module is used for inputting each training sample pair in the obtained at least one training sample pair into inlets corresponding to branches of a neural network with two branches respectively, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively;
the matching module is used for matching a comparison result obtained by comparing the first characteristic with the second characteristic with a labeling value corresponding to the training sample pair; performing the training of the neural network according to the matching result;
and the model training module is used for obtaining a circuit board defect detection model through multi-round training of the neural network.
In combination with the third aspect, the present application provides a first possible implementation of the third aspect, wherein the neural network comprises a twin siemese network;
the feature determination module is specifically configured to:
inputting each training sample pair in at least one obtained training sample pair into two inlets of a Siamese network respectively, and determining a first feature vector and a second feature vector for the input training sample pairs by using the Siamese network respectively;
the matching module is specifically configured to:
performing the following matching operation until the comparison result of the first characteristic vector and the second characteristic vector is consistent with the labeled value of the corresponding training sample pair, and finishing the training of the current round; wherein the labeled values represent similarity or dissimilarity of corresponding training sample pairs;
the matching operation comprises:
comparing the first feature vector with the second feature vector to obtain a comparison result;
matching the comparison result with the labeled value of the corresponding training sample pair;
generating first feedback information aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result; and are
Performing parameter adjustment on the Siamese network based on the first feedback information;
based on the adjusted parameters, new first and second feature vectors are determined for corresponding pairs of training samples using the siemese network, and the matching operation is performed again.
With reference to the third aspect, the present application provides a second possible implementation of the third aspect, wherein the neural network comprises a two-channel neural network;
the feature determination module is specifically configured to:
respectively taking each training sample pair in at least one acquired training sample pair as source data and target data, and respectively inputting the source data and the target data into different channels of the two-channel neural network;
determining a first feature and a second feature for an input training sample pair using the two-channel neural network, respectively;
the matching module is specifically configured to:
performing the following distance determination operation until the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the labeled value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the labeled value representation of the corresponding source data is not similar to the corresponding training sample pair is met, and finishing the training of the current round;
the distance determining operation comprising:
determining a distance between the first feature and the second feature;
generating second feedback information aiming at the condition that the distance between the first feature and the second feature is larger than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is not larger than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair; and are
Performing parameter adjustment on the two-channel neural network based on the second feedback information;
based on the adjusted parameters, new first and second features are determined for corresponding pairs of training samples using the two-channel neural network, and the distance determination operation is performed again.
In a fourth aspect, an embodiment of the present application further provides a circuit board defect detecting apparatus, including:
the model input module is used for inputting the circuit board image to be detected and the corresponding standard circuit board image into a circuit board defect detection model obtained by training the training device in any one of the third aspect, the first possible implementation mode and the second possible implementation mode of the third aspect;
the characteristic determining module is used for extracting a third characteristic and a fourth characteristic for the circuit board image to be detected and the corresponding standard circuit board image respectively by using the circuit board defect detection model;
a feature comparison module for comparing the third feature with the fourth feature;
and the defect detection module is used for determining whether the circuit board to be detected has defects according to the obtained comparison result.
In the above scheme provided in the embodiment of the present application, two identical and/or different circuit board images in a training sample pair are correspondingly input into two branches in a neural network, and multiple rounds of training are performed on the neural network through a matching result obtained by matching a comparison result of a first feature and a second feature determined by the neural network with a labeled value of the training sample pair, so as to obtain a circuit board defect detection model. The circuit board defect detection model trained by the scheme can automatically detect whether the circuit board has defects, the problem that the fault tolerance rate and the accuracy rate are low due to an automatic optical detection method is solved, the fault tolerance rate and the accuracy rate of detection are high, and the applicability is good.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 is a flow chart illustrating a training method of a circuit board defect detection model according to an embodiment of the present application;
FIG. 2 is a flow chart illustrating another training method for a circuit board defect inspection model provided by an embodiment of the present application;
FIG. 3 is a flow chart illustrating another training method for a circuit board defect inspection model provided by an embodiment of the present application;
FIG. 4 is a flow chart illustrating a method for detecting defects of a circuit board according to an embodiment of the present disclosure;
FIG. 5 is a flow chart illustrating another method for detecting defects of a circuit board according to an embodiment of the present application;
FIG. 6 is a flow chart of another method for detecting defects of a circuit board according to an embodiment of the present application;
FIG. 7 is a functional block diagram of a training apparatus for a circuit board defect inspection model according to an embodiment of the present application;
FIG. 8 is a functional block diagram of a circuit board defect detecting apparatus according to an embodiment of the present disclosure;
FIG. 9 is a schematic structural diagram of a computer device provided in an embodiment of the present application;
fig. 10 shows a schematic structural diagram of another computer device provided in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Considering that in the related art, the method for detecting the defects of the PCB based on the automatic optical detection means needs to set different matching templates for different error types, and the error types without the matching templates cannot be effectively detected, so that the fault tolerance of the detection is low, and in order to prevent the omission detection, the method often adjusts the threshold range of the detection, which causes more correct PCBs to be considered as errors and has low accuracy. In view of this, an embodiment of the present application provides a method for training a defect detection model of a circuit board to improve accuracy and fault tolerance of defect detection, which is described in the following embodiments.
Referring to fig. 1, a flowchart of a training method for a circuit board defect detection model provided in the embodiment of the present application is applied to a computer device, and the training method includes the following steps:
s101, obtaining at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images.
Here, training sample pairs need to be acquired in advance. The two circuit board images in the training sample pair appear in pairs, and the two circuit board images appearing in pairs may be the same or different. In the embodiment of the present application, the same two circuit board images may both be correct circuit board images, and the different two circuit board images may be one correct circuit board image and the other wrong circuit board image. The correct circuit board image may be a circuit board image corresponding to an incorrect circuit board image found from a full standard circuit board diagram, and the incorrect circuit board image may be a circuit board image having a defect determined by an automatic optical detection means in the related art, or a circuit board image having a defect manually collected, which is not limited in this embodiment of the present application.
S102, inputting each training sample pair in at least one acquired training sample pair into inlets corresponding to branches of a neural network with two branches respectively, and determining a first feature and a second feature for the input training sample pairs by using the neural network respectively.
Here, based on the above-mentioned at least one training sample pair, the present embodiment performs training of the neural network with each training sample pair as training data of the neural network having two branches.
In the training process, for a training sample pair input to the neural network, corresponding first and second features can be determined for two circuit board images included in the training sample pair. The first characteristic is related to an input circuit board image and a training parameter of one branch of the neural network corresponding to the circuit board image, and the second characteristic is related to an input other circuit board image and a training parameter of the other branch of the neural network corresponding to the circuit board image.
S103, comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with the labeled value of the corresponding training sample pair; and performing the training of the neural network according to the matching result.
Here, the embodiment of the present application may perform multiple rounds of training on the neural network, and in each round of training, may perform adjustment of training parameters on one or two branches of the neural network according to a matching result between a comparison result of the first feature and the second feature and a labeled value of a corresponding training sample pair, and update the first feature and the second feature based on the adjusted training parameters until the matching result reaches a preset condition, the round of training is completed.
The preset condition may be that the comparison result of the first feature and the second feature is consistent with the label value of the corresponding training sample pair, or that the distance between the first feature and the second feature is not greater than a preset distance threshold, and the label value of the corresponding source data (such as one circuit board image in the training sample pair) represents that the corresponding training sample pair is similar, and/or that the distance between the first feature and the second feature is greater than the preset distance threshold, and the label value of the corresponding source data represents that the corresponding training sample pair is not similar.
In addition, the labeled value of the training sample pair in the embodiment of the present application may be labeled in advance, for example, the training sample pair corresponding to the same two circuit board images may be labeled as 0, and the training sample pair corresponding to different two circuit board images may be labeled as 1. It should be noted that the above labeling manner is only a specific example, and when the training method of the circuit board defect detection model provided in the embodiment of the present application is specifically applied, other labeling manners may be adopted, which is not specifically limited in the embodiment of the present application.
And S104, obtaining a circuit board defect detection model through multi-round training of the neural network.
Here, in order to further ensure the training effect of the training method for the circuit board defect detection model provided in the embodiment of the present application, the embodiment of the present application may train the neural network in a multi-round training manner to obtain the circuit board defect detection model.
In a specific implementation, the circuit board defect detection model maps two circuit board images (i.e., a to-be-detected circuit board image and a standard circuit board image) into one detection result (i.e., a labeled value of the to-be-detected circuit board image relative to the standard circuit board image). The method and the device can adopt the twin Siemese network to carry out model training and can also adopt the dual-channel neural network to carry out model training. That is, the twin siense network or the dual-channel neural network is adopted in the method, the training parameters are gradually determined through repeated iterative learning, and finally, how to generate a marking value according to the two circuit board images is learned.
The following describes the process of circuit board defect detection model training using twin siemese network and dual channel neural network, respectively.
For the method for training the circuit board defect detection model by applying the twin siemsee network, each training sample pair in at least one obtained training sample pair can be firstly and respectively input into two inlets of the siemsee network, so that the siemsee network can respectively map two inputs to a new space to form a representation input into the new space, namely the siemsee network can be used for determining a corresponding first feature vector and a corresponding second feature vector for the input training sample pair. After the first feature vector and the second feature vector are determined, the embodiment of the application may execute the matching operation in a loop until the new first feature vector and the new second feature vector determined after the matching operation are executed can satisfy that the comparison result of the first feature vector and the second feature vector is consistent with the labeled value of the corresponding training sample pair, and then end a round of training.
It should be noted that, the fact that the comparison result of the first feature vector and the second feature vector is consistent with the labeled value of the corresponding training sample pair may mean that when the two circuit board images in the training sample pair are similar images, the comparison result indicates a smaller value, and when the two circuit board images in the training sample pair are dissimilar images, the comparison result indicates a larger value. The judgment of the larger value and the smaller value can be based on a preset threshold, wherein the larger value is higher than the preset threshold, and the smaller value is lower than the preset threshold.
As shown in fig. 2, the embodiment of the present application may perform the matching operation according to the following steps:
s201, comparing the first characteristic vector with the second characteristic vector to obtain a comparison result;
s202, matching the comparison result with the labeled value of the corresponding training sample pair;
s203, aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result, generating first feedback information;
s204, adjusting parameters of the Siamese network based on the first feedback information;
and S205, based on the adjusted parameters, determining new first feature vectors and second feature vectors for the corresponding training sample pairs by using the Siemese network, and executing the matching operation again.
Here, the matching operation in the embodiment of the present application may be executed multiple times, and in each execution, the first feature vector and the second feature vector may be compared first, then the comparison result obtained by the comparison is matched with the labeled value of the training sample pair, then the matching condition of the corresponding training sample pair is characterized by the labeled value and the matching condition characterized by the comparison result, and first feedback information is generated, and finally the parameter of the Siamese network is adjusted based on the first feedback information, and the updated first feature vector and second feature vector are determined based on the adjusted parameter.
The Siamese network used in the embodiment of the present application uses two networks on the upper and lower sides, which are completely the same network structure, and they share the same weight parameter W, and if the input training sample pair and its labeled value are (X1, X2, Y), where X1 and X2 may represent the training sample pair, Y may represent the labeled values corresponding to X1 and X2, and Y may include two values. Y-0 indicates that X1 and X2 belong to similar circuit board images, and Y-1 indicates that the two images are not similar. Namely, the similarity pair is (X1, X2, 0), and the spoof pair is (X1, X2', 1). For two different inputs X1 and X2, the low-dimensional spatial result, i.e., the first eigenvector and the second eigenvector, respectively, is output.
In the embodiment of the present application, a comparison result obtained by comparing the first feature vector and the second feature vector may be used to represent a feature similarity between the two feature vectors. Here, the feature similarity between two feature vectors may be determined based on a cosine distance calculation method, may also be determined based on an euclidean distance calculation method, and may also be determined based on other methods, which is not specifically limited in the embodiment of the present application.
In addition, as set forth above, the first feature and the second feature may be affected by the training parameters of the two corresponding scores of the neural network, and thus, the first feature vector and the second feature vector may also be affected by the training parameters of the Siamese network, that is, when it is determined that the matching condition of the corresponding training sample pair represented by the labeled value is inconsistent with the matching condition represented by the comparison result, the training parameters may be adjusted according to the generated first feedback information, and after the parameters are adjusted, the first feature vector and the second feature vector may also be updated accordingly.
Furthermore, the goal of training with the Siamese network is to make the feature similarity corresponding to two similar circuit board images as small as possible, and the feature similarity corresponding to two dissimilar circuit board images as large as possible. And then, after the first characteristic vector and the second characteristic vector are updated, carrying out network cyclic training by taking the target as a convergence condition until a circuit board defect detection model is obtained.
For the defect detection model of the circuit board trained by using the dual-channel neural network, each training sample pair in at least one acquired training sample pair can be used as source data and target data respectively and input into different channels of the dual-channel neural network respectively, so that the dual-channel neural network can map two inputs to a new space respectively to form a representation input into the new space, namely the dual-channel neural network can be used for determining corresponding first characteristics and second characteristics for the input training sample pairs. After the first feature and the second feature are determined, the embodiment of the application may perform the distance determining operation in a loop until the new first feature and the new second feature determined after the distance determining operation is performed can satisfy that the distance between the first feature and the second feature is not greater than the preset distance threshold, and the labeled value of the corresponding source data represents that the corresponding training sample pairs are similar, and/or satisfy that the distance between the first feature and the second feature is greater than the preset distance threshold, and the labeled value of the corresponding source data represents that the corresponding training sample pairs are not similar, and then end a round of training.
As shown in fig. 3, the embodiment of the present application may perform the distance determining operation according to the following steps:
s301, determining the distance between the first feature and the second feature;
s302, generating second feedback information aiming at the situation that the distance between the first feature and the second feature is larger than a preset distance threshold value and the corresponding labeled value representation of the source data is similar to the corresponding training sample pair and/or aiming at the situation that the distance between the first feature and the second feature is not larger than the preset distance threshold value and the corresponding labeled value representation of the source data is not similar to the corresponding training sample pair;
s303, adjusting parameters of the dual-channel neural network based on the second feedback information;
s304, based on the adjusted parameters, determining new first features and second features for the corresponding training sample pairs by using the two-channel neural network, and executing distance determination operation again.
Here, the distance determining operation according to the embodiment of the present application may be performed multiple times, and in each time of the performing, the distance between the first feature and the second feature may be determined first, then, for the case that the distance between the first feature and the second feature is greater than the preset distance threshold, and the labeled value of the corresponding source data represents that the corresponding training sample pair is similar, and/or for the case that the distance between the first feature and the second feature is not greater than the preset distance threshold, and the labeled value of the corresponding source data represents that the corresponding training sample pair is not similar, the second feedback information is generated, and finally, the parameter of the dual-channel neural network is adjusted based on the second feedback information, and the updated first feature and the updated second feature are determined based on the adjusted parameter.
Here, the distance between the two features may be determined based on not only a cosine distance calculation method, but also an euclidean distance calculation method, and may also be determined based on other methods, which is not specifically limited in the embodiment of the present application.
In addition, as explained above, the first feature and the second feature are affected by the training parameters of two corresponding branches of the neural network, so that the first feature and the second feature are also affected by the training parameters of the two-channel neural network, that is, the embodiment of the present application adjusts the training parameters according to the second feedback information, and after the parameters are adjusted, the first feature and the second feature are updated accordingly. Then, after the first feature and the second feature are updated, the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the labeled value of the corresponding source data represents that the corresponding training sample pairs are similar, and/or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the labeled value of the corresponding source data represents that the corresponding training sample pairs are dissimilar is taken as a convergence condition to perform network circulation training until a circuit board defect detection model is obtained.
Based on the circuit board defect detection model obtained by training in the above embodiment, an embodiment of the present application further provides a circuit board defect detection method, as shown in fig. 4, which is a flowchart of the circuit board defect detection method provided in the embodiment of the present application, and is applied to a computer device, and the circuit board defect detection method includes the following steps:
s401, inputting a circuit board image to be detected and a corresponding standard circuit board image into a circuit board defect detection model;
s402, extracting a third feature and a fourth feature for the circuit board image to be detected and the corresponding standard circuit board image respectively by using a circuit board defect detection model;
s403, comparing the third characteristic with the fourth characteristic;
s404, determining whether the circuit board to be detected has defects according to the obtained comparison result.
Here, in the embodiment of the present application, the to-be-detected circuit board image and the corresponding standard circuit board image are input into the trained circuit board defect detection model, so that the third feature and the fourth feature corresponding to the input can be obtained, the labeled value of the to-be-detected circuit board image relative to the standard circuit board image can be determined according to the comparison result of the third feature and the fourth feature, and whether the to-be-detected circuit board has a defect can be determined according to the labeled value.
The defect detection method and the defect detection device can add more training samples of defect types in a model training stage to ensure that a trained circuit board defect detection model has stronger detection capability, so that automatic detection can be performed on various defects (such as cold joint, adhesion, copper foil falling and the like) in a subsequent model application stage, time and labor are saved, and the fault tolerance rate and the accuracy rate of detection are higher.
In the embodiment of the application, the standard circuit board image corresponding to the circuit board image to be detected can be searched from the whole standard circuit board image based on the circuit board image to be detected. For the search operation, the search operation can be directly performed based on the traversal operation of the to-be-detected circuit board image on the full graph of the standard circuit board, the full graph of the standard circuit board can be roughly searched based on the first identification information of the components included in the to-be-detected circuit board image, and then the area determined by the rough search can be finely searched. Next, two search methods will be described.
For the direct search mode, as shown in fig. 5, the embodiment of the present application searches for a corresponding standard circuit board image by the following steps:
s501, performing traversal operation on the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
s502, calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
s503, determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
Here, the circuit board image to be detected slides on the standard circuit board full map (such as a PCB full map), for example, the circuit board image to be detected slides from left to right and from top to bottom, so that at each sliding position, similarity calculation can be performed once to indicate the matching degree between the candidate standard circuit board image in the area where the sliding position is located and the circuit board image to be detected, and the candidate standard circuit board image with the highest matching degree with the circuit board image to be detected is selected from the candidate standard circuit board images corresponding to each sliding position as the standard circuit board image.
The similarity calculation may be an image similarity between the circuit board image to be detected and the candidate standard circuit board image. In the embodiment of the present application, a correlation coefficient matching method may be adopted for evaluating the similarity, as shown in the following formula:
Figure GDA0002627293550000151
wherein, R (i, j) is a standard correlation coefficient, the (i, j) is each pixel coordinate in the overall graph of the standard circuit board, T is the circuit board image to be detected, and T (m, n) is the pixel value with the coordinate (m, n) in the circuit board image to be detected. And S, a full graph of the standard circuit board, Sij is a candidate standard circuit board image intercepted by the full graph coordinate of the standard circuit board as the position (i, j), and Sij (m, n) is a pixel value with the coordinate (m, n) in the candidate standard circuit board image.
For the way of coarse search and fine search, as shown in fig. 6, in the embodiment of the present application, the following steps are adopted to search for a corresponding standard circuit board image:
s601, determining first identification information of components included in each circuit board image to be detected;
s602, comparing the first identification information with second identification information of each component included in the full graph of the corresponding standard circuit board, and determining the region of the matched second identification information corresponding to the component in the full graph of the standard circuit board;
s603, performing traversal operation on the area determined in the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
s604, calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and S605, determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
Here, it is considered that one or more components may be included in the circuit board image to be detected, which may increase the difficulty of obtaining the standard circuit board image. In order to further ensure the accuracy of the acquired standard circuit board image, in the embodiment of the application, first, based on comparison between first identification information of components included in the circuit board image to be detected and second identification information of each component included in the corresponding standard circuit board full map, an area of the component corresponding to the matched second identification information in the standard circuit board full map is determined, that is, an area corresponding to the circuit board image to be detected is determined in the standard circuit board full map, and then, the corresponding standard circuit board image is scrutinized and found by sliding the circuit board image to be detected in the determined area.
The method for searching the standard circuit board image from the determined area is similar to the method for directly searching the standard circuit board image from the standard circuit board full image, or the standard circuit board image can be searched by calculating the image similarity after the traversal operation is executed, and the specific traversal process and the similarity calculation process are referred to above, and are not described herein again.
Based on the same inventive concept, the embodiment of the present application further provides a training device of a circuit board defect detection model corresponding to the training method of the circuit board defect detection model, and as the principle of solving the problem of the device in the embodiment of the present application is similar to the training method of the circuit board defect detection model in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and repeated parts are not repeated.
As shown in fig. 7, a schematic structural diagram of a training apparatus for a circuit board defect inspection model provided in an embodiment of the present application is shown, where the training apparatus for the circuit board defect inspection model includes:
a sample obtaining module 701, configured to obtain at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images;
a feature determining module 702, configured to input each training sample pair of the obtained at least one training sample pair into an entry corresponding to each branch of a neural network having two branches, and determine a first feature and a second feature for the input training sample pair by using the neural network;
a matching module 703, configured to match a comparison result obtained by comparing the first feature with the second feature with a labeled value of a corresponding training sample pair; performing the training of the neural network according to the matching result;
and the model training module 704 is used for obtaining a circuit board defect detection model through multi-round training of the neural network.
In one embodiment, the neural network comprises a twin siense network;
the feature determining module 702 is specifically configured to:
inputting each training sample pair in at least one obtained training sample pair into two inlets of a Siamese network respectively, and determining a first feature vector and a second feature vector for the input training sample pairs respectively by using the Siamese network;
the matching module 703 is specifically configured to:
performing the following matching operation until the comparison result of the first characteristic vector and the second characteristic vector is consistent with the labeled value of the corresponding training sample pair, and finishing the training of the current round; wherein, the marked value represents that the corresponding training sample pairs are similar or dissimilar;
the matching operation comprises the following steps:
comparing the first feature vector with the second feature vector to obtain a comparison result;
matching the comparison result with the labeled value of the corresponding training sample pair;
generating first feedback information aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result; and are
Adjusting parameters of the Siamese network based on the first feedback information;
based on the adjusted parameters, new first and second feature vectors are determined for the corresponding training sample pairs using the siemese network, and the matching operation is performed again.
In another embodiment, the neural network comprises a two-channel neural network;
the feature determining module 702 is specifically configured to:
respectively taking each training sample pair in the obtained at least one training sample pair as source data and target data, and respectively inputting the source data and the target data into different channels of the two-channel neural network;
determining a first feature and a second feature for an input training sample pair using a two-channel neural network, respectively;
the matching module 703 is specifically configured to:
performing the following distance determination operation until the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the labeled value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the labeled value representation of the corresponding source data is not similar to the corresponding training sample pair is met, and finishing the training of the current round;
a distance determination operation comprising:
determining a distance between the first feature and the second feature;
generating second feedback information aiming at the condition that the distance between the first feature and the second feature is larger than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is not larger than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair; and are
Adjusting parameters of the dual-channel neural network based on the second feedback information;
based on the adjusted parameters, new first and second features are determined for corresponding pairs of training samples using the two-channel neural network, and the distance determination operation is performed again.
Based on the same application concept, the embodiment of the application also provides a circuit board defect detection device corresponding to the circuit board defect detection method, and as the principle of solving the problems of the device in the embodiment of the application is similar to the circuit board defect detection method in the embodiment of the application, the implementation of the device can refer to the implementation of the method, and repeated parts are not repeated.
As shown in fig. 8, which is a schematic structural diagram of a circuit board defect detecting apparatus provided in the embodiment of the present application, the circuit board defect detecting apparatus includes:
the model input module 801 is used for inputting the circuit board image to be detected and the corresponding standard circuit board image into a circuit board defect detection model;
the feature determination module 802 is configured to extract a third feature and a fourth feature for the circuit board image to be detected and the corresponding standard circuit board image respectively by using a circuit board defect detection model;
a feature comparison module 803, configured to compare the third feature with the fourth feature;
and the defect detection module 804 is used for determining whether the circuit board to be detected has defects according to the obtained comparison result.
In one embodiment, the circuit board defect detecting apparatus further includes:
the first standard image determining module 805 is configured to determine, for each circuit board image to be detected, first identification information of a component included in the circuit board image to be detected;
comparing the first identification information with second identification information of each component included in the corresponding standard circuit board full graph, and determining the region of the matched second identification information corresponding to the component in the standard circuit board full graph;
performing traversal operation on the area determined in the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
In one embodiment, the circuit board defect detecting apparatus further includes:
a second standard image determining module 806, configured to perform traversal operation on the full standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
As shown in fig. 9, a schematic structural diagram of a computer device provided in an embodiment of the present application is shown, where the computer device includes: a processor 901, a memory 902 and a bus 903, the memory 902 storing machine readable instructions executable by the processor 901, the processor 901 and the memory 902 communicating via the bus 903 when the computer device is operating, the machine readable instructions when executed by the processor 901 performing the following:
obtaining at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images;
inputting each training sample pair in at least one acquired training sample pair into an inlet corresponding to each branch of a neural network with two branches, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with the labeled value of the corresponding training sample pair; and are
Performing the training of the neural network according to the matching result;
and obtaining a circuit board defect detection model through multiple rounds of training of the neural network.
In one embodiment, the neural network comprises a twin siense network; the above-mentioned processor 901 performs a process of inputting each training sample pair of at least one acquired training sample pair into an entry corresponding to each branch of a neural network having two branches, and determining a first feature and a second feature for the input training sample pair by using the neural network, respectively, including:
inputting each training sample pair in at least one obtained training sample pair into two inlets of a Siamese network respectively, and determining a first feature vector and a second feature vector for the input training sample pairs respectively by using the Siamese network;
in the processing executed by the processor 901, a comparison result obtained by comparing the first feature with the second feature is matched with the labeled value of the corresponding training sample pair; and according to the matching result, the neural network is trained in the current round, which comprises the following steps:
performing the following matching operation until the comparison result of the first characteristic vector and the second characteristic vector is consistent with the labeled value of the corresponding training sample pair, and finishing the training of the current round; wherein, the marked value represents that the corresponding training sample pairs are similar or dissimilar;
the matching operation comprises the following steps:
comparing the first feature vector with the second feature vector to obtain a comparison result;
matching the comparison result with the labeled value of the corresponding training sample pair;
generating first feedback information aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result; and are
Adjusting parameters of the Siamese network based on the first feedback information;
based on the adjusted parameters, new first and second feature vectors are determined for the corresponding training sample pairs using the siemese network, and the matching operation is performed again.
In another embodiment, the neural network comprises a two-channel neural network; the above-mentioned processor 901 performs a process of inputting each training sample pair of at least one acquired training sample pair into an entry corresponding to each branch of a neural network having two branches, and determining a first feature and a second feature for the input training sample pair by using the neural network, respectively, including:
respectively taking each training sample pair in the obtained at least one training sample pair as source data and target data, and respectively inputting the source data and the target data into different channels of the two-channel neural network;
determining a first feature and a second feature for an input training sample pair using a two-channel neural network, respectively;
in the processing executed by the processor 901, a comparison result obtained by comparing the first feature with the second feature is matched with the labeled value of the corresponding training sample pair; and according to the matching result, the neural network is trained in the current round, which comprises the following steps:
performing the following distance determination operation until the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the labeled value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the labeled value representation of the corresponding source data is not similar to the corresponding training sample pair is met, and finishing the training of the current round;
a distance determination operation comprising:
determining a distance between the first feature and the second feature;
generating second feedback information aiming at the condition that the distance between the first feature and the second feature is larger than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair and/or the condition that the distance between the first feature and the second feature is not larger than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair; and are
Adjusting parameters of the dual-channel neural network based on the second feedback information;
based on the adjusted parameters, new first and second features are determined for corresponding pairs of training samples using the two-channel neural network, and the distance determination operation is performed again.
As shown in fig. 10, a schematic structural diagram of a computer device provided in an embodiment of the present application is shown, where the computer device includes: a processor 1001, a memory 1002 and a bus 1003, the memory 1002 storing machine readable instructions executable by the processor 1001, the processor 1001 and the memory 1002 communicating via the bus 1003 when the computer device is operating, the machine readable instructions when executed by the processor 1001 perform the following:
inputting a circuit board image to be detected and a corresponding standard circuit board image into a circuit board defect detection model;
extracting a third characteristic and a fourth characteristic for the circuit board image to be detected and the corresponding standard circuit board image respectively by using a circuit board defect detection model; and are
Comparing the third feature with the fourth feature;
and determining whether the circuit board to be detected has defects according to the obtained comparison result.
In one embodiment, the processing performed by the processor 1001 further includes: determining a standard circuit board image corresponding to the circuit board image to be detected by adopting the following method:
determining first identification information of components included in each circuit board image to be detected;
comparing the first identification information with second identification information of each component included in the corresponding standard circuit board full graph, and determining the region of the matched second identification information corresponding to the component in the standard circuit board full graph;
performing traversal operation on the area determined in the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
In another embodiment, the processing performed by the processor 1001 further includes: determining a standard circuit board image corresponding to the circuit board image to be detected by adopting the following method:
performing traversal operation on the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor 901, the steps of the training method for the circuit board defect detection model are executed.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, and when a computer program on the storage medium is run, the training method of the circuit board defect detection model can be executed, so that the problems of low fault tolerance rate and low accuracy rate caused by the existing automatic optical detection method are solved, and the effects of improving the accuracy rate and the fault tolerance rate of defect detection are achieved.
The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor 1001, the steps of the circuit board defect detection method are performed.
Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, and when a computer program on the storage medium is run, the method for detecting the circuit board defect can be executed, so that the problems of low fault tolerance rate and low accuracy rate caused by the existing automatic optical detection method are solved, and the effects of improving the accuracy rate and the fault tolerance rate of defect detection are achieved.
The computer program product of the network traffic monitoring method provided in the embodiment of the present application includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the method in the foregoing method embodiment, and specific implementation may refer to the method embodiment, which is not described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (8)

1. A method for detecting defects of a circuit board is characterized by comprising the following steps:
obtaining at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images;
inputting each training sample pair in at least one acquired training sample pair into inlets corresponding to branches of a neural network with two branches respectively, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with a labeled value of a corresponding training sample pair; the comparison result represents the feature similarity between the first feature and the second feature, and the label value represents the similarity or the dissimilarity of the training sample pairs; and are
Performing the training of the neural network according to the matching result;
obtaining a circuit board defect detection model through multi-round training of the neural network;
inputting a circuit board image to be detected and a corresponding standard circuit board image into the circuit board defect detection model;
extracting a third feature and a fourth feature for the circuit board image to be detected and the corresponding standard circuit board image respectively by using the circuit board defect detection model;
comparing the third feature to the fourth feature;
and determining whether the circuit board to be detected has defects according to the obtained comparison result.
2. The method of claim 1, wherein the neural network comprises a twin siense network;
inputting each training sample pair in at least one acquired training sample pair into an inlet corresponding to each branch of a neural network with two branches, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively, wherein the method comprises the following steps:
inputting each training sample pair in at least one obtained training sample pair into two inlets of a Siamese network respectively, and determining a first feature vector and a second feature vector for the input training sample pairs by using the Siamese network respectively;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with a labeled value of a corresponding training sample pair; and performing the training of the neural network according to the matching result, including:
performing the following matching operation until the comparison result of the first characteristic vector and the second characteristic vector is consistent with the labeled value of the corresponding training sample pair, and finishing the training of the current round; wherein the labeled values represent similarity or dissimilarity of corresponding training sample pairs;
the matching operation comprises:
comparing the first feature vector with the second feature vector to obtain a comparison result;
matching the comparison result with the labeled value of the corresponding training sample pair;
generating first feedback information aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result; and are
Performing parameter adjustment on the Siamese network based on the first feedback information;
based on the adjusted parameters, new first and second feature vectors are determined for corresponding pairs of training samples using the siemese network, and the matching operation is performed again.
3. The method of claim 1, wherein the neural network comprises a two-channel neural network;
inputting each training sample pair in at least one acquired training sample pair into an inlet corresponding to each branch of a neural network with two branches, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively, wherein the method comprises the following steps:
respectively taking each training sample pair in at least one acquired training sample pair as source data and target data, and respectively inputting the source data and the target data into different channels of the two-channel neural network;
determining a first feature and a second feature for an input training sample pair using the two-channel neural network, respectively;
comparing the first characteristic with the second characteristic to obtain a comparison result, and matching the comparison result with a labeled value of a corresponding training sample pair; and performing the training of the neural network according to the matching result, including:
performing the following distance determination operation until the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair is met, or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair is met, and finishing the training of the current round;
the distance determining operation comprising:
determining a distance between the first feature and the second feature;
generating second feedback information aiming at the condition that the distance between the first feature and the second feature is greater than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair, or aiming at the condition that the distance between the first feature and the second feature is not greater than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair; and are
Performing parameter adjustment on the two-channel neural network based on the second feedback information;
based on the adjusted parameters, new first and second features are determined for corresponding pairs of training samples using the two-channel neural network, and the distance determination operation is performed again.
4. The method according to claim 1, characterized in that the standard circuit board image corresponding to the circuit board image to be detected is determined by the following method:
determining first identification information of components included in the circuit board image to be detected aiming at each circuit board image to be detected;
comparing the first identification information with second identification information of each component included in the corresponding standard circuit board full graph, and determining the region of the matched second identification information corresponding to the component in the standard circuit board full graph;
performing traversal operation on the area determined in the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
5. The method according to claim 1, characterized in that the standard circuit board image corresponding to the circuit board image to be detected is determined by the following method:
performing traversal operation on the whole standard circuit board image based on the size of the circuit board image to be detected and a preset traversal step length to obtain at least one candidate standard circuit board image;
calculating the image similarity of the circuit board image to be detected and each candidate standard circuit board image;
and determining the candidate standard circuit board image with the maximum image similarity as the standard circuit board image.
6. A circuit board defect detecting apparatus, comprising:
the sample acquisition module is used for acquiring at least one training sample pair used in the current round of training; wherein the training sample pair comprises two identical and/or different circuit board images;
the characteristic determining module is used for inputting each training sample pair in the obtained at least one training sample pair into inlets corresponding to branches of a neural network with two branches respectively, and determining a first characteristic and a second characteristic for the input training sample pair by using the neural network respectively;
the matching module is used for matching a comparison result obtained by comparing the first characteristic with the second characteristic with a labeling value corresponding to the training sample pair; performing the training of the neural network according to the matching result; the comparison result represents the feature similarity between the first feature and the second feature, and the label value represents the similarity or the dissimilarity of the training sample pairs;
the model training module is used for obtaining a circuit board defect detection model through multi-round training of the neural network;
the model input module is used for inputting the circuit board image to be detected and the corresponding standard circuit board image into the circuit board defect detection model;
the characteristic determining module is used for extracting a third characteristic and a fourth characteristic for the circuit board image to be detected and the corresponding standard circuit board image respectively by using the circuit board defect detection model;
a feature comparison module for comparing the third feature with the fourth feature;
and the defect detection module is used for determining whether the circuit board to be detected has defects according to the obtained comparison result.
7. The apparatus of claim 6, wherein the neural network comprises a twin siense network;
the feature determination module is specifically configured to:
inputting each training sample pair in at least one obtained training sample pair into two inlets of a Siamese network respectively, and determining a first feature vector and a second feature vector for the input training sample pairs by using the Siamese network respectively;
the matching module is specifically configured to:
performing the following matching operation until the comparison result of the first characteristic vector and the second characteristic vector is consistent with the labeled value of the corresponding training sample pair, and finishing the training of the current round; wherein the labeled values represent similarity or dissimilarity of corresponding training sample pairs;
the matching operation comprises:
comparing the first feature vector with the second feature vector to obtain a comparison result;
matching the comparison result with the labeled value of the corresponding training sample pair;
generating first feedback information aiming at the condition that the matching condition of the corresponding training sample pair represented by the mark value is inconsistent with the matching condition represented by the comparison result; and are
Performing parameter adjustment on the Siamese network based on the first feedback information;
based on the adjusted parameters, new first and second feature vectors are determined for corresponding pairs of training samples using the siemese network, and the matching operation is performed again.
8. The apparatus of claim 6, wherein the neural network comprises a two-channel neural network;
the feature determination module is specifically configured to:
respectively taking each training sample pair in at least one acquired training sample pair as source data and target data, and respectively inputting the source data and the target data into different channels of the two-channel neural network;
determining a first feature and a second feature for an input training sample pair using the two-channel neural network, respectively;
the matching module is specifically configured to:
performing the following distance determination operation until the condition that the distance between the first feature and the second feature is not larger than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair is met, or the condition that the distance between the first feature and the second feature is larger than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair is met, and finishing the training of the current round;
the distance determining operation comprising:
determining a distance between the first feature and the second feature;
generating second feedback information aiming at the condition that the distance between the first feature and the second feature is greater than a preset distance threshold value and the marked value representation of the corresponding source data is similar to the corresponding training sample pair, or aiming at the condition that the distance between the first feature and the second feature is not greater than the preset distance threshold value and the marked value representation of the corresponding source data is not similar to the corresponding training sample pair; and are
Performing parameter adjustment on the two-channel neural network based on the second feedback information;
based on the adjusted parameters, new first and second features are determined for corresponding pairs of training samples using the two-channel neural network, and the distance determination operation is performed again.
CN201810701246.XA 2018-06-29 2018-06-29 Circuit board defect detection method and device Active CN108961236B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810701246.XA CN108961236B (en) 2018-06-29 2018-06-29 Circuit board defect detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810701246.XA CN108961236B (en) 2018-06-29 2018-06-29 Circuit board defect detection method and device

Publications (2)

Publication Number Publication Date
CN108961236A CN108961236A (en) 2018-12-07
CN108961236B true CN108961236B (en) 2021-02-26

Family

ID=64484087

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810701246.XA Active CN108961236B (en) 2018-06-29 2018-06-29 Circuit board defect detection method and device

Country Status (1)

Country Link
CN (1) CN108961236B (en)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109829883B (en) * 2018-12-19 2020-11-17 歌尔光学科技有限公司 Product quality detection method and device
CN109685142A (en) * 2018-12-25 2019-04-26 国信优易数据有限公司 A kind of image matching method and device
WO2020181458A1 (en) * 2019-03-11 2020-09-17 西门子（中国）有限公司 Data processing method, apparatus, and system
CN110533654A (en) * 2019-08-30 2019-12-03 北京明略软件系统有限公司 The method for detecting abnormality and device of components
CN110570490B (en) * 2019-09-06 2021-07-30 北京航空航天大学 Saliency image generation method and equipment
CN112634190A (en) * 2019-09-24 2021-04-09 上海汽车集团股份有限公司 Gluing detection method and device
CN113012088A (en) * 2019-12-03 2021-06-22 浙江大搜车软件技术有限公司 Circuit board fault detection and twin network training method, device and equipment
TWI783200B (en) * 2019-12-30 2022-11-11 緯創資通股份有限公司 Method and device for determining whether object has defect
CN111179251B (en) * 2019-12-30 2021-04-02 上海交通大学 Defect detection system and method based on twin neural network and by utilizing template comparison
CN111537532A (en) * 2020-06-11 2020-08-14 全球能源互联网研究院有限公司 Membrane electrode defect detection method and device
CN111768387A (en) * 2020-06-30 2020-10-13 创新奇智(青岛)科技有限公司 Flaw detection method, twin neural network training method and device and electronic equipment
CN112183224A (en) * 2020-09-07 2021-01-05 北京达佳互联信息技术有限公司 Model training method for image recognition, image recognition method and device
CN114519326B (en) * 2020-11-19 2023-03-24 Tcl科技集团股份有限公司 Photoelectric characteristic obtaining method and device, storage medium and terminal equipment
CN112559869A (en) * 2020-12-18 2021-03-26 上海众源网络有限公司 Comment information display method and device, electronic equipment and storage medium
CN112967224A (en) * 2021-01-29 2021-06-15 绍兴隆芙力智能科技发展有限公司 Electronic circuit board detection system, method and medium based on artificial intelligence
CN113905606B (en) * 2021-09-13 2022-09-30 中国地质大学（武汉） Chip mounter surface mounting scheduling model training method based on deep reinforcement learning
CN115855950A (en) * 2022-11-23 2023-03-28 环维电子(上海)有限公司 Image detection method and system for tiny flaws and wrong parts
CN115953403B (en) * 2023-03-14 2023-05-30 常州微亿智造科技有限公司 Defect detection method and device
CN116168040B (en) * 2023-04-26 2023-07-07 四川元智谷科技有限公司 Component direction detection method and device, electronic equipment and readable storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106686472A (en) * 2016-12-29 2017-05-17 华中科技大学 High-frame-rate video generation method and system based on depth learning
CN106897681A (en) * 2017-02-15 2017-06-27 武汉喜恩卓科技有限责任公司 A kind of remote sensing images comparative analysis method and system
CN107562805A (en) * 2017-08-08 2018-01-09 浙江大华技术股份有限公司 It is a kind of to scheme to search the method and device of figure
WO2018084948A1 (en) * 2016-11-07 2018-05-11 Qualcomm Incorporated Enhanced siamese trackers
CN108133238A (en) * 2017-12-29 2018-06-08 国信优易数据有限公司 A kind of human face recognition model training method and device and face identification method and device
CN108171209A (en) * 2018-01-18 2018-06-15 中科视拓（北京）科技有限公司 A kind of face age estimation method that metric learning is carried out based on convolutional neural networks

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10740596B2 (en) * 2016-11-08 2020-08-11 Nec Corporation Video security system using a Siamese reconstruction convolutional neural network for pose-invariant face recognition

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018084948A1 (en) * 2016-11-07 2018-05-11 Qualcomm Incorporated Enhanced siamese trackers
CN106686472A (en) * 2016-12-29 2017-05-17 华中科技大学 High-frame-rate video generation method and system based on depth learning
CN106897681A (en) * 2017-02-15 2017-06-27 武汉喜恩卓科技有限责任公司 A kind of remote sensing images comparative analysis method and system
CN107562805A (en) * 2017-08-08 2018-01-09 浙江大华技术股份有限公司 It is a kind of to scheme to search the method and device of figure
CN108133238A (en) * 2017-12-29 2018-06-08 国信优易数据有限公司 A kind of human face recognition model training method and device and face identification method and device
CN108171209A (en) * 2018-01-18 2018-06-15 中科视拓（北京）科技有限公司 A kind of face age estimation method that metric learning is carried out based on convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network";Haokui Zhang等;《Remote Sensing Letters》;20170118;第8卷(第5期);第438-447页 *
"基于改进深度孪生网络的分类器及其应用";沈雁等;《计算机工程与应用》;20180531;第54卷(第10期);第19-25页 *

Also Published As

Publication number Publication date
CN108961236A (en) 2018-12-07

Similar Documents

Publication Publication Date Title
CN108961236B (en) Circuit board defect detection method and device
WO2017032311A1 (en) Detection method and apparatus
CN111983935B (en) Performance evaluation method and device
TWI748242B (en) Method and system for scanning wafer
CN104615986B (en) The method that pedestrian detection is carried out to the video image of scene changes using multi-detector
CN112508846B (en) Defect detection method and device, electronic equipment and storage medium
CN108520514B (en) Consistency detection method for electronic elements of printed circuit board based on computer vision
CN109752392B (en) PCB defect type detection system and method
CN111862057B (en) Picture labeling method and device, sensor quality detection method and electronic equipment
CN109002784B (en) Street view identification method and system
CN109447062A (en) Pointer-type gauges recognition methods based on crusing robot
CN111758117B (en) Inspection system, recognition system, and learning data generation device
CN110084830A (en) A kind of detection of video frequency motion target and tracking
CN105354816B (en) A kind of electronic units fix method and device
CN111325265B (en) Detection method and device for tampered image
CN107423738B (en) Test paper subject positioning method and device based on template matching
CN115775246A (en) Method for detecting defects of PCB (printed circuit board) components
CN109509165B (en) Image positioning area selection method and device
CN108992033B (en) Grading device, equipment and storage medium for vision test
CN113505789A (en) Electrical equipment nameplate text recognition method based on CRNN
CN114782431B (en) Printed circuit board defect detection model training method and defect detection method
CN115082713A (en) Method, system and equipment for extracting target detection frame by introducing space contrast information
CN103020601B (en) Hi-line visible detection method and device
JP2008185395A (en) Mounting substrate visual inspection method
SE516239C2 (en) Method and apparatus for determining nominal data for electronic circuits, by taking a digital image and comparing it with stored nominal data.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 100070, No. 101-8, building 1, 31, zone 188, South Fourth Ring Road, Beijing, Fengtai District

Applicant after: Guoxin Youyi Data Co., Ltd

Address before: 100070, No. 188, building 31, headquarters square, South Fourth Ring Road West, Fengtai District, Beijing

Applicant before: SIC YOUE DATA Co.,Ltd.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant