TW202402117A - Training method for defect detection model and defect detection method for printed circuit board - Google Patents

Abstract

The present disclosure provides a training method for defect detection model and defect detection method for printed circuit board, wherein the training method comprises: acquiring design document information of a printed circuit board, wherein the design document information comprises region information of the printed circuit board; scanning the printed circuit board with a scanning camera device to obtain a scanned image of the printed circuit board; generating learning samples based on the scanned image and region information of the printed circuit board, wherein each learning sample comprises a sample image and a corresponding label, comprising capturing a partial image in the scanned image to obtain a sample image and manually labeling the sample image, wherein the label obtained by manual labeling comprises region information and defect information; establishing a sample library, comprising collecting and storing learning samples generated based on multiple printed circuit boards according to above steps; using the learning samples of the sample library to train a basic model and obtain the defect detection model of printed circuit board.

Description

Printed circuit board defect detection model training method and defect detection method

The invention relates to the field of PCB defect detection, and in particular to a printed circuit board defect detection model training method and a defect detection method.

Automated Optical Inspection (AOI) equipment has now become an important inspection tool and process quality control tool for the electronics manufacturing industry to ensure product quality. The detection principle of AOI equipment is: when automatically inspecting, the AOI equipment automatically Scan the PCB product to collect images, compare the test detection points with the qualified parameters in the database, and detect defects on the tested product after image processing.

Usually after the defect is detected by the AOI equipment, the defect information will be sent to the maintenance worker for maintenance. However, the detection accuracy of AOI equipment is low. For example, dust or stains on the circuit board will be misjudged as defects by the AOI equipment. Therefore, it is necessary to improve the precision detection accuracy of PCB defects in the existing technology.

The disclosure of the above background technology content is only used to assist in understanding the inventive concepts and technical solutions of the present invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical guidance; in the absence of clear evidence that the above content is If the patent application has been published before the filing date, the above background technology should not be used to evaluate the novelty and inventiveness of the application.

The purpose of the present invention is to provide a training method for a printed circuit board defect detection model so as to train an improved AI model that can accurately and quickly identify circuit board defects.

In order to achieve the above objects, the technical solutions adopted by the present invention are as follows:

A printed circuit board defect detection model training method includes the following steps:

Obtain the design document information of the printed circuit board, the design document information includes the layer information and/or the area block position information of the printed circuit board; and use a scanning camera device to scan the printed circuit board to obtain the printed circuit board Scanned image of the plate;

Based on the scanned image of the printed circuit board and design document information, one or more learning samples are generated. Each learning sample includes a sample image and a corresponding label, including: intercepting a partial image in the scanned image, Obtain a sample image, and perform manual marking on the sample image, where the labels obtained by manual marking include one or two types of layer information, area block position information, and defect information;

Establish a sample library, including following the above steps to collect and store learning samples generated based on multiple printed circuit boards;

The basic model is trained using the learning samples of the sample library to obtain a printed circuit board defect detection model.

Further, the basic model is configured with a first learning sub-module and a second learning sub-module, wherein,

The first learning sub-module performs learning and training based on the sample images in the learning sample and the defect information in the labels to obtain an intermediate model;

The second learning sub-module learns feature information between one or both of the layer information, region block position information and defect information based on the labels in the learning sample; and,

The intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample to obtain the printed circuit board defect detection model.

Further, the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including:

If the layer information corresponding to the sample image is the power layer or the ground layer, the learning goal of the intermediate model is to learn the features in the sample image to be non-short-circuit features.

Further, the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including:

If the layer information corresponding to the sample image is the line layer or the corresponding area block position information is the copper surface area, the intermediate model will focus the learning attention on the specific learning sample, and the defect information in the label of the specific learning sample is defective, and the defect type is other than short circuit or open circuit, or the defect information in the label of the specific learning sample is no defect.

Further, the method for the intermediate model to learn the specific learning sample is:

If there are two separate copper features or a line feature connecting two cables in the sample image of the specific learning sample, the recognition of the two separate copper features or the line feature connecting two cables in the sample image will be weakened. The ability to identify the line features of the wiring.

Further, the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including:

If the layer information corresponding to the sample image is the circuit layer or the corresponding area block position information is the copper surface area, then the method for the intermediate model to learn the sample image corresponding to the defect information of the defect type is short circuit is to strengthen the sample image. The identification of two separate copper features in the image, or, the enhancement of the identification of the line feature connecting the two cables in the sample image.

Further, the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including:

If the layer information corresponding to the sample image is the line layer or the corresponding area block position information is the copper surface area, then the method for the intermediate model to learn the sample image corresponding to the defect information whose defect type is a circuit break is to strengthen the sample image. The ability to identify features with gaps in the alignment line in the image.

Further, preprocessing the sample library includes: traversing the sample images in the sample library. If the layer information corresponding to the sample image is the line layer or the corresponding area block position information is the copper surface area, use the image The processor marks the electronic devices and copper wires in the sample image with different colors;

The basic model is trained using the learning samples of the preprocessed sample library to obtain the printed circuit board defect detection model.

Further, the scanning camera device is integrated on an AOI device, the design document information of the printed circuit board is input to the AOI device, and the basic model is the detection model of the AOI device or its back-end AI model.

Further, the sample library is used to construct a training set and a test set, and the training set is used to perform multiple rounds of training on the basic model;

Using the test set to verify the trained model includes: using the mean square error loss function or the average absolute value error loss function to calculate the loss value of the trained model; and based on the number of predictions and predictions that the prediction results are consistent with the label The total number of times is used to calculate the accuracy of the trained model;

Verify whether the loss value and accuracy meet the preset training goals, then use the currently trained model as the printed circuit board defect detection model; otherwise, use the training set for iterative training until the iterative training results The loss value and accuracy of the model have been verified.

According to another aspect of the present invention, a printed circuit board defect detection method is provided, including the following steps:

Obtain the image of the printed circuit board to be inspected and its layer information and/or area block position information;

Input the image of the printed circuit board and its layer information and/or area block position information into a printed circuit board defect detection model that has been trained in advance;

The printed circuit board defect detection model outputs detection results;

Among them, the printed circuit board defect detection model is trained through the following steps:

Obtain the design document information of the printed circuit board, the design document information includes the layer information and/or the area block position information of the printed circuit board; and use a scanning camera device to scan the printed circuit board to obtain the printed circuit board Scanned image of the plate;

Based on the scanned image of the printed circuit board and design document information, one or more learning samples are generated. Each learning sample includes a sample image and a corresponding label, including: intercepting a partial image in the scanned image, Obtain a sample image, and perform manual marking on the sample image, where the labels obtained by manual marking include one or two types of layer information, area block position information, and defect information;

Establish a sample library, including following the above steps to collect and store learning samples generated based on multiple printed circuit boards;

The basic model is trained using the learning samples of the sample library. The basic model is configured with a first learning sub-module and a second learning sub-module. The first learning sub-module is based on the sample images and labels in the learning samples. Learn and train the defect information in the sample to obtain an intermediate model; the second learning sub-module learns the characteristics between one or two of the layer information, region block position information and defect information based on the labels in the learning sample Information; the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample to obtain the printed circuit board defect detection model.

Further, the printed circuit board defect detection model is trained by the printed circuit board defect detection model training method as described above.

The beneficial effects brought by the technical solution provided by the present invention are as follows:

a. Make full use of the regional information of the circuit board to learn the rules between regional information and defects, and train an improved AI model. The improved AI model combines the regional information of the circuit board to accurately identify the circuit board image;

b. Mastering the correlation characteristics between circuit board areas and circuit board defects can quickly eliminate defect types that are impossible to exist in certain areas, improving defect detection efficiency and accuracy of detection results.

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of the present invention.

It should be noted that the terms "first", "second", etc. in the description of the invention, the patent scope of the invention, and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. . It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein are capable of being practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, processes, methods, devices, products or equipment that include a series of steps or units and need not be limited to those explicitly listed. Those steps or units may instead include other steps or units not expressly listed or inherent to the processes, methods, products or devices.

In printed circuit board (PCB) defect detection, the scanned image of the circuit board is usually input into the AI model, and the features in the image are extracted through image analysis technology to analyze whether they match the characteristics of the defect. In other words, the current In defect detection technology, only the circuit board image is analyzed and detected. The present invention proposes a technology that combines circuit board images with corresponding area information to perform defect detection. The whole layer or local area information of the circuit board is used to assist the AI model in obtaining PCB defect detection faster and more accurately. result. As shown in Figure 1, each circuit board/each type of circuit board has a corresponding design document, which records in detail the information on each layer of the circuit board, as well as the distributed design information on each layer, which is called the area block location. Information; before using the AOI device to scan the circuit board, the design document is input into the AOI device, and the AOI device scans the corresponding PCB layers one by one according to the design document. In the embodiment of the present invention, the AOI device combines the scanned PCB image with The corresponding PCB area information (layer information and/or area block position information) is sent to the back-end AI model. The AI model obtains PCB defect detection results based on these two input information. In one embodiment of the present invention, the AOI has defects. The preliminary identification function, accordingly, intercepts the defective areas identified by the AOI device, obtains the PCB sub-image, and then passes it to the back-end AI model.

A slightly abnormal local area on the PCB scan image is identified by the AOI device. According to experience, there will be misjudgements, such as some dust or stains, and the intercepted sub-image is sent to the AI model, which combines the various The regional information of PCB sub-images accurately identifies each PCB sub-image, effectively eliminating false defects misidentified by AOI equipment. Different from other PCB defect detection models, the AI model in the present invention is a model that combines regional information to perform AI recognition of PCB images. To this end, the present invention proposes a method for training it, as shown in Figure 2, Printing The circuit board defect detection model training method includes the following steps:

Obtain the design document information of the printed circuit board, the design document information includes the layer information and/or the area block position information of the printed circuit board; and use a scanning camera device to scan the printed circuit board to obtain the printed circuit board. Scanned image of the plate;

Based on the scanned image of the printed circuit board and design document information, one or more learning samples are generated. Each learning sample includes a sample image and a corresponding label, including: intercepting a partial image in the scanned image, Obtain a sample image, and perform manual marking on the sample image, where the labels obtained by manual marking include one or two types of layer information, area block position information, and defect information;

Establish a sample library, including following the above steps to collect and store learning samples generated based on multiple printed circuit boards;

The basic model is trained using the learning samples of the sample library to obtain a printed circuit board defect detection model.

Using learning samples with printed circuit board layer information and/or area block position information (referred to as area information in Figure 3), the specific method of training the AI basic model is as follows: See Figure 3, the basic model is configured with The first learning sub-module and the second learning sub-module, where,

The first learning sub-module performs learning and training based on the sample images in the learning sample and the defect information in the labels to obtain an intermediate model;

The second learning sub-module learns feature information between one or both of the layer information, region block position information and defect information based on the labels in the learning sample; and,

The intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample to obtain the printed circuit board defect detection model.

Wherein, the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample, including the following aspects:

In the first aspect, the second learning sub-module learned that the PCB image of the power layer or ground layer does not have the characteristic information of short circuit defects. Based on this learning result, the intermediate model uses the sample image of the power layer or ground layer when the learning layer information is the sample image of the power layer or ground layer. , the learning goal of the intermediate model is to learn the features in the sample image into non-short-circuit features.

In the second aspect, the second learning sub-module learned that in images where the layer information is the line layer or the corresponding area block position information is the copper surface area, the defect types are likely to be concentrated in the two types of short circuit and open circuit. Based on this learning result, When the intermediate model learns the line layer or the corresponding area block position information to be a sample image of the copper surface area, the intermediate model focuses the learning attention on the specific learning sample, and the defect information in the label of the specific learning sample is defective. , and the defect type is other than short circuit and open circuit, or the defect information in the label of the specific learning sample is no defect. That is, it learns low-probability defect types, reduces the probability of mistakenly identifying defects in images of the circuit layer or copper surface area as short circuit or open circuit types, and improves the accuracy of identification.

Specifically, the method for the intermediate model to learn the specific learning sample is: if there are two separate copper features or line features connecting two cables in the sample image of the specific learning sample, then weaken the sample image. Discrimination of two separate copper features or line features connecting two flat cables.

For third parties, in addition to the second aspect of focusing learning attention on specific learning samples, it is also necessary to learn non-specific learning samples (that is, the defect information in the label is short circuit or open circuit type). For short circuit type sample graphs For example, the intermediate model enhances the recognition of the two separate copper features in the sample image; or the intermediate model enhances the recognition of the line feature connecting the two cables in the sample image. For a sample image of the circuit break type, the intermediate model combines the feature information learned by the second learning sub-module to enhance the ability to identify features of gaps on the wiring line in the sample image.

In one embodiment of the present invention, the area information of the circuit board can be used to preprocess the sample library, including: traversing the sample images in the sample library. If the layer information corresponding to the sample image is the circuit layer or the corresponding The area block position information is the copper surface area, and the image processor is used to mark the electronic devices and copper wires in the sample image with different colors; the basic model is trained using the learning samples of the preprocessed sample library , obtain the printed circuit board defect detection model. By knowing which areas are where electronic devices are located and which areas are where copper wires are located, electronic devices and copper wires can be marked with different colors, making it easier for the model to distinguish electronic devices and copper wires, further improving the efficiency and efficiency of defect identification. Accuracy.

The training process of the printed circuit board defect detection model also involves the convergence and verification of the model: using the sample library to construct a training set and a test set, and using the training set to perform multiple rounds of training on the basic model;

Using the test set to verify the trained model includes: using the mean square error loss function or the average absolute value error loss function to calculate the loss value of the trained model; and based on the number of predictions and predictions that the prediction results are consistent with the label The total number of times to calculate the accuracy of the trained model: ;

Verify whether the loss value and accuracy meet the preset training goals, then use the currently trained model as the printed circuit board defect detection model; otherwise, use the training set for iterative training until the iterative training results The loss value and accuracy of the model have been verified.

In one embodiment of the present invention, a printed circuit board defect detection method is provided. As shown in Figure 4, the detection method includes the following steps:

Obtain the image of the printed circuit board to be inspected and its layer information and/or area block position information;

Input the image of the printed circuit board and its layer information and/or area block position information into a printed circuit board defect detection model that has been trained in advance;

The printed circuit board defect detection model outputs detection results;

Among them, the printed circuit board defect detection model is trained through the following steps:

Obtain the design document information of the printed circuit board, the design document information includes the layer information and/or the area block position information of the printed circuit board; and use a scanning camera device to scan the printed circuit board to obtain the printed circuit board Scanned image of the plate;

Based on the scanned image of the printed circuit board and design document information, one or more learning samples are generated. Each learning sample includes a sample image and a corresponding label, including: intercepting a partial image in the scanned image, Obtain a sample image, and perform manual marking on the sample image, where the labels obtained by manual marking include one or two types of layer information, area block position information, and defect information;

Establish a sample library, including following the above steps to collect and store learning samples generated based on multiple printed circuit boards;

The basic model is trained using the learning samples of the sample library. The basic model is configured with a first learning sub-module and a second learning sub-module. The first learning sub-module is based on the sample images and labels in the learning samples. Learn and train the defect information in the sample to obtain an intermediate model; the second learning sub-module learns the characteristics between one or two of the layer information, region block position information and defect information based on the labels in the learning sample Information; the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample to obtain the printed circuit board defect detection model.

This embodiment of the defect detection method uses the printed circuit board defect detection model trained in the above training method embodiment to perform AI recognition on the input image of the printed circuit board to be inspected and its layer information and/or area block position information. Then output the defect detection results. The entire content of the above embodiment of the printed circuit board defect detection model training method is introduced into this embodiment of the printed circuit board defect detection method.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

The above are only specific embodiments of the present application. It should be noted that those of ordinary skill in the technical field can also make several improvements and modifications without departing from the principles of the present application. These improvements and modifications can also be made. should be regarded as the scope of protection of this application.

without

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the purpose of describing the embodiments or the prior art. For some of the embodiments recorded in the application, those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts. Figure 1 is a conceptual diagram provided by an exemplary embodiment of the present invention; Figure 2 is a schematic flow chart of printed circuit board defect detection model training provided by an exemplary embodiment of the present invention; Figure 3 is a schematic flowchart of training an AI basic model using circuit board area information according to an exemplary embodiment of the present invention; FIG. 4 is an information flow diagram of printed circuit board defect detection provided by an exemplary embodiment of the present invention.

Claims (12)

A printed circuit board defect detection model training method, which includes the following steps: Obtain the design document information of the printed circuit board, the design document information includes the layer information and/or the area block position information of the printed circuit board; and use a scanning camera device to scan the printed circuit board to obtain the printed circuit board Scanned image of the plate; Based on the scanned image of the printed circuit board and design document information, one or more learning samples are generated. Each learning sample includes a sample image and a corresponding label, including: intercepting a partial image in the scanned image, Obtain a sample image, and perform manual marking on the sample image, where the labels obtained by manual marking include one or two types of layer information, area block position information, and defect information; Establish a sample library, including following the above steps to collect and store learning samples generated based on multiple printed circuit boards; The basic model is trained using the learning samples of the sample library to obtain a printed circuit board defect detection model. The printed circuit board defect detection model training method according to claim 1, wherein the basic model is configured with a first learning sub-module and a second learning sub-module, The first learning sub-module performs learning and training based on the sample images in the learning sample and the defect information in the labels to obtain an intermediate model; The second learning sub-module learns feature information between one or both of the layer information, region block position information and defect information based on the labels in the learning sample; and, The intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample to obtain the printed circuit board defect detection model. The printed circuit board defect detection model training method according to claim 2, wherein the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including : If the layer information corresponding to the sample image is the power layer or the ground layer, the learning goal of the intermediate model is to learn the features in the sample image to be non-short-circuit features. The printed circuit board defect detection model training method according to claim 2, wherein the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including : If the layer information corresponding to the sample image is the line layer or the corresponding area block position information is the copper surface area, the intermediate model will focus the learning attention on the specific learning sample, and the defect information in the label of the specific learning sample is defective, and the defect type is other than short circuit or open circuit, or the defect information in the label of the specific learning sample is no defect. The printed circuit board defect detection model training method as described in claim 4, wherein the method for the intermediate model to learn the specific learning sample is: If there are two separate copper features or a line feature connecting two cables in the sample image of the specific learning sample, the recognition of the two separate copper features or the line feature connecting two cables in the sample image will be weakened. The ability to identify the line features of the wiring. The printed circuit board defect detection model training method according to claim 2, wherein the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including : If the layer information corresponding to the sample image is the circuit layer or the corresponding area block position information is the copper surface area, then the method for the intermediate model to learn the sample image corresponding to the defect information of the defect type is short circuit is to strengthen the sample image. The identification of two separate copper features in the image, or, the enhancement of the identification of the line feature connecting the two cables in the sample image. The printed circuit board defect detection model training method according to claim 2, wherein the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample including : If the layer information corresponding to the sample image is the line layer or the corresponding area block position information is the copper surface area, then the method for the intermediate model to learn the sample image corresponding to the defect information whose defect type is a circuit break is to strengthen the sample image. The ability to identify features with gaps in the alignment line in the image. The printed circuit board defect detection model training method as described in any one of claims 1 to 7, wherein preprocessing the sample library includes: Traverse the sample images in the sample library. If the layer information corresponding to the sample image is the line layer or the corresponding area block position information is the copper surface area, the image processor is used to separately analyze the electronic devices and copper wires in the sample image. Mark different colors; The basic model is trained using the learning samples of the preprocessed sample library to obtain the printed circuit board defect detection model. The printed circuit board defect detection model training method according to any one of claims 1 to 7, wherein the scanning camera device is integrated on an AOI device, and the design document information of the printed circuit board is input to the AOI equipment, and the basic model is the detection model of the AOI equipment or its back-end AI model. The printed circuit board defect detection model training method according to any one of claims 1 to 7, wherein the sample library is used to construct a training set and a test set, and the training set is used to perform multiple rounds on the basic model training; Using the test set to verify the trained model includes: using the mean square error loss function or the average absolute value error loss function to calculate the loss value of the trained model; and based on the number of predictions and predictions that the prediction results are consistent with the label The total number of times is used to calculate the accuracy of the trained model; To verify whether the loss value and accuracy meet the preset training goals, the currently trained model will be used as the printed circuit board defect detection model; otherwise, the training set will be used for iterative training until the iterative training results The loss value and accuracy of the model have been verified. A printed circuit board defect detection method, which includes the following steps: Obtain the image of the printed circuit board to be inspected and its layer information and/or area block position information; Input the image of the printed circuit board and its layer information and/or area block position information into a printed circuit board defect detection model that has been trained in advance; The printed circuit board defect detection model outputs detection results; Among them, the printed circuit board defect detection model is trained through the following steps: Obtain the design document information of the printed circuit board, the design document information includes the layer information and/or the area block position information of the printed circuit board; and use a scanning camera device to scan the printed circuit board to obtain the printed circuit board. Scanned image of the plate; Based on the scanned image of the printed circuit board and design document information, one or more learning samples are generated. Each learning sample includes a sample image and a corresponding label, including: intercepting a partial image in the scanned image, Obtain a sample image, and perform manual marking on the sample image, where the labels obtained by manual marking include one or two types of layer information, area block position information, and defect information; Establish a sample library, including following the above steps to collect and store learning samples generated based on multiple printed circuit boards; The basic model is trained using the learning samples of the sample library. The basic model is configured with a first learning sub-module and a second learning sub-module. The first learning sub-module is based on the sample images and labels in the learning samples. Learn and train the defect information in the sample to obtain an intermediate model; the second learning sub-module learns the characteristics between one or two of the layer information, region block position information and defect information based on the labels in the learning sample Information; the intermediate model combines the feature information learned by the second learning sub-module to re-learn the sample images in the learning sample to obtain the printed circuit board defect detection model. The printed circuit board defect detection method according to claim 11, wherein the printed circuit board defect detection model is trained by the printed circuit board defect detection model training method according to any one of claims 3 to 10.

Images