WO2023236038A1 - Update method and system for circuit board detection model, and electronic device and storage medium - Google Patents

Abstract

An update method and system for a circuit board detection model, and an electronic device and a storage medium. When applied to an edge device, the update method comprises: inputting at least one circuit board image into a first detection model, so as to obtain a feature vector of each circuit board image (101); performing difficult sample mining on the at least one circuit board image (102); respectively acquiring a label of each difficult sample (103); obtaining update information according to the feature vector and the label of each difficult sample (104); sending the update information to a server, such that the server updates the first detection model according to the update information, so as to obtain a second detection model (105); and the edge device performing anomaly detection on a circuit board by means of the second detection model (106). The problem of massive labor costs being consumed due to it being necessary to label all samples when detection is performed on a circuit board is solved; and update information is generated for a feature vector and a label of a difficult sample, such that the accuracy and high efficiency of update of a detection model can be ensured.

Description

Update method, system, electronic device and storage medium for circuit board inspection model Technical field

This application relates to artificial intelligence, and in particular to a method, system, electronic device and storage medium for updating a circuit board detection model.

Background technique

During the automatic circuit board installation process, in order to reduce development costs, it is necessary to deploy a deep learning model on the installation scenario to detect abnormalities on the circuit board. Although there are common requirements in these scenarios, there are actually differences in different installation scenarios, such as different lighting environments, different components, different camera positions, etc. If the basic detection model is directly deployed on different installation scenarios, the effect of anomaly detection will be poor, and it will not be able to well reflect the abnormal conditions of the circuit board in the current installation scenario.

Currently, in order to deal with the above differences, one method is to collect and manually annotate all data samples in the current installation environment, and then update the basic detection model based on the annotations derived from the annotation to obtain a new detection model to improve the accuracy of anomaly detection. Effect. However, annotating all data samples within the installation scenario to update the basic detection model will consume a lot of labor.

Contents of the invention

In order to solve the above technical problems, embodiments of the present invention provide a method, system, electronic device and storage medium for updating a circuit board detection model, so as to at least solve or alleviate the above problems.

According to a first aspect of the embodiment of the present application, a method for updating a circuit board detection model is provided, which is applied to edge devices. The method includes: inputting at least one circuit board image into the first detection model, and obtaining each circuit board image. The feature vector of the image, wherein the first detection model is used to detect abnormality of the circuit board based on the circuit board image; perform difficult sample mining on the at least one circuit board image to obtain at least one difficult sample; obtain each Labels of the difficult samples; obtain update information based on the feature vectors and labels of each difficult sample; send the update information to the server, so that the server performs on the first detection model based on the update information Update to obtain a second detection model; obtain the second detection model issued by the server, and perform anomaly detection on the circuit board through the second detection model.

According to a second aspect of the embodiment of the present application, a method for updating a circuit board detection model is provided, which is applied to a server. The method includes: receiving update information from an edge device, wherein the update information is based on at least one difficult sample The feature vectors and labels of the difficult samples are determined, and the feature vectors of the difficult samples are obtained by inputting the circuit board image into the first detection model; the first detection model is updated according to the update information to obtain the second detection model; the The second detection model is delivered to the edge device, so that the edge device performs anomaly detection on the circuit board through the second detection model.

According to a third aspect of the embodiment of the present application, a system for updating a circuit board detection model is provided, including: an edge device and a server, the edge device being configured to execute the above-mentioned updating method of the circuit board detection model of the first aspect; The server is used to execute the updating method of the circuit board inspection model of the second aspect.

According to a fourth aspect of the embodiment of the present application, an electronic device is provided, including: a processor, a communication interface, a memory and a communication bus. The processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is used to store At least one executable instruction, the executable instruction causes the processor to perform an operation corresponding to the above-mentioned updating method of the circuit board inspection model in the first aspect or the above-mentioned updating method of the circuit board inspection model in the second aspect.

According to a fifth aspect of the embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the updating method of the circuit board inspection model as described in the first aspect or the above-mentioned third aspect is implemented. Two aspects of circuit board inspection model update methods.

According to a sixth aspect of the embodiment of the present application, a computer program product is provided, including computer instructions. The computer instructions instruct the computing device to perform the updating method of the circuit board detection model as described in the first aspect or the circuit board detection model as described in the second aspect. Detect the operations corresponding to the update method of the model.

It can be seen from the above technical solution that after the edge device obtains the first detection model, it can obtain difficult samples by mining at least one input circuit board image for difficult samples. By manually annotating the difficult samples, all input circuits can be Annotating plate images can reduce a large amount of labor costs and man-hours used to annotate samples. By updating the first detection model through the server, the server automatically completes the update action of the first detection model, and the server automatically delivers the updated second detection model to the server, which also reduces labor costs and corresponding working hours.

Obtaining updated information through the feature vectors and labels of difficult samples can improve the accuracy of the updated information. The server uses the updated information to update the first detection model to a second detection model that is more suitable for the current environment, which can improve the effect of anomaly detection.

Description of drawings

In order to explain the embodiments of the present application or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are some of the embodiments recorded in the embodiments of this application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

Figure 1 is a schematic diagram of a method for updating a circuit board detection model according to an embodiment of the present application;

Figure 2 is a schematic diagram of a method for updating a circuit board detection model according to another embodiment of the present application;

Figure 3 is a schematic diagram of a circuit board detection model system according to an embodiment of the present application;

Figure 4 is a schematic diagram of a method for updating a circuit board detection model according to another embodiment of the present application;

Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present application.

List of reference signs:

100: Update method for circuit board detection model 10: Edge device 20: Server

200: Update method for circuit board inspection model 300: Update system for circuit board inspection model 50: Electronic equipment

400: Update method of circuit board inspection model 51: Processor 52: Communication interface

53: Storage 54: Communication bus 55: Program

101: Input at least one circuit board image into the first detection model to obtain the feature vector of each circuit board image

102: Mining difficult samples on at least one circuit board image and obtaining at least one difficult sample

103: Obtain the label of each difficult sample separately

104: Obtain updated information based on the feature vectors and labels of each difficult sample

105: Send update information to the server

106: Obtain the second detection model issued by the server, and perform anomaly detection on the circuit board through the second detection model

201: Receive update information

202: Update the first detection model

203: Deliver the second detection model to the edge device

401: The server obtains the first detection model

402: The server delivers the first detection model to the edge device

403: Edge device obtains at least one circuit board image

404: The edge device inputs at least one circuit board image into the first detection model

405: Edge device performs difficult sample mining on at least one circuit board image

406: The edge device obtains the label of each difficult sample separately

407: The edge device obtains the gradient based on the feature vector and label of each difficult sample, and uses the gradient as update information

408: The edge device sends updated information to the server

409: The server receives the update information sent by the edge device and performs data cleaning on the update information.

410: The server updates the first detection model based on the updated information after data cleaning.

411: The server delivers the second detection model to the edge device

Detailed ways

In order to enable those in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the description The embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the examples in the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art should fall within the scope of protection of the embodiments of this application.

An updated method for circuit board inspection models applied to edge devices

Figure 1 is a circuit board detection model updating method 100 according to an embodiment of the present application, which is applied to edge devices. As shown in Figure 1, the circuit board inspection model updating method 100 includes the following steps:

Step 101: Input at least one circuit board image into the first detection model to obtain the feature vector of each circuit board image.

A first detection model is deployed on the edge device. The first detection model is used to detect abnormality of the circuit board based on the circuit board image. The circuit board image may be an image of the circuit board to be detected during the actual production process of the circuit board. After the edge device obtains the circuit board image, the obtained circuit board image can be input into the deployed first detection model. The first detection model performs anomaly detection on the circuit board based on the input circuit board image. During the detection process, the first detection model A feature vector corresponding to the circuit board image will be generated, and the feature vector may indicate the abnormality detection result of the circuit board by the corresponding first detection model.

Step 102: Perform difficult sample mining on at least one circuit board image to obtain at least one difficult sample.

After inputting each circuit board image into the first detection model respectively, the first detection model performs abnormality detection on the corresponding circuit board based on the input circuit board image. According to the intermediate information obtained during the abnormality detection process of the first detection model, it can be Perform difficult sample mining on each circuit board image to obtain at least one difficult sample. Difficult samples refer to circuit board images where there is a large error between the prediction result and the actual abnormality of the circuit board. The number of difficult samples is less than or equal to the number of circuit board images input to the first detection model.

Step 103: Obtain the label of each difficult sample respectively.

For each difficult sample excavated, each difficult sample is labeled separately, and the label corresponding to each difficult sample is obtained. The label is used to indicate whether there is an abnormality in the corresponding difficult sample (circuit board image). Among them, the labeling of difficult samples can be completed through manual annotation, or it can be obtained by averaging multiple prediction results of the first detection model on the enhanced data.

For the method of using the average of multiple prediction results as the label of a difficult sample, for each difficult sample, multiple new samples are formed by adding random perturbations to the difficult sample, and then the first detection model is used to detect the generated multiple new samples. The samples are predicted separately, and the average of the prediction results of the first detection model for multiple new samples is used as the label of the difficult sample. For example, by adding random perturbation to a difficult sample, 20 new samples are generated. When predicting these 20 new samples through the first detection model, the prediction results of 18 new samples are that the circuit board has no abnormality, and 2 new samples The prediction result is that there is an abnormality in the circuit board, then the label of the difficult sample is determined to be no abnormality.

Step 104: Obtain update information based on the feature vector and label of each difficult sample.

According to each difficult sample mined, the feature vector of each difficult sample is obtained based on the feature vector of each circuit board image that has been obtained. The update information includes the feature vector and label of each difficult sample. The update information is used to A detection model is updated.

Step 105: Send the update information to the server.

After the edge device obtains the update information, it sends the update information to the server, and the server updates the first detection model according to the update information, obtains the second detection model, and sends the second detection model to the edge device 10 .

The first detection model is stored on the server. After the edge device sends the update information to the server, the server can automatically update the first detection model based on the update information to obtain the second detection model. The process of the server updating the first detection model does not require Human involvement. The server can connect to multiple edge devices and update the detection model deployed on each edge device separately. After obtaining the second detection model, the server will deliver the second detection model to the corresponding edge device.

Step 106: Obtain the second detection model issued by the server, and perform anomaly detection on the circuit board through the second detection model.

After the edge device receives the second detection model issued by the server, the edge device uses the second detection model to detect abnormalities on the circuit board, that is, inputs the circuit board image of the circuit board to be detected into the second detection model, and detects it through the second detection model. Is there any abnormality in the corresponding circuit board?

The edge device can input the circuit board image into the first detection model to obtain the feature vector of each circuit board image, and can also perform difficult sample mining on multiple circuit board images input into the first detection model to obtain difficult samples. The samples are manually annotated to obtain labels, and then the feature vectors and labels of the difficult samples can be obtained. Update information is obtained based on the feature vectors and labels of the difficult samples. After the update information is sent to the server, the server updates the first detection model based on the update information. , and delivers the obtained second detection model to the edge device, and then the edge device can use the second detection model to perform anomaly detection on the circuit board. A smaller number of difficult samples are obtained through difficult sample mining, and then only the difficult samples need to be annotated to update the first detection model, without the need to annotate all the circuit board images input to the first detection model, thereby reducing a large amount of Labor costs and hours used to label the samples. In addition, the update information is sent to the server, and the server updates the first detection model according to the update information. The server can automatically complete the update action of the first detection model, and automatically deliver the updated second detection model to the edge device. The process of updating the first detection model based on the updated information does not require manual participation, thereby reducing manual costs.

Obtaining updated information through the feature vectors and labels of difficult samples can improve the accuracy of the updated information. The server uses the updated information to update the first detection model to a second detection model that is more suitable for the current environment, which can improve the effect of anomaly detection.

In one possible implementation, the update information includes the feature vectors and labels of each difficult sample, that is, after obtaining the feature vectors and labels of the difficult samples, the feature vectors and labels of the difficult samples are directly sent to the server as update information, by The server updates the first detection model according to the feature vector and label of the difficult sample to obtain the second detection model.

In the embodiment of the present application, since the feature vector of the difficult sample can indicate the abnormality detection result of the corresponding circuit board by the first detection model, and the label of the difficult sample can indicate whether the corresponding circuit board actually has an abnormality, the feature vector of the difficult sample is and labels as update information, so that the update information can reflect the accuracy of the first detection model in detecting anomalies on the circuit board on the edge device, and then updating the first detection model through the update information can improve the accuracy of the detection model in detecting anomalies on the circuit board. accuracy.

In one possible implementation, after obtaining the feature vector and label of each difficult sample, the gradient corresponding to the difficult sample can be calculated based on the feature vector and label of each difficult sample, and then the gradient of each difficult sample is determined as Update information.

Optionally, for each difficult sample, the edge device can calculate the gradient of the difficult sample through the back propagation algorithm based on the feature vector and label of the difficult sample. Of course, the edge device can also use other methods to calculate gradients based on the feature vectors and labels of difficult samples, which is not limited in the embodiments of the present application.

In this embodiment of the present application, the edge device calculates the gradient based on the feature vector and label of the difficult sample, and transmits the gradient of each difficult sample to the server as update information, so that the server updates the first detection model based on the gradient of each difficult sample. Since the feature vectors and labels of difficult samples cannot be inferred based on the gradient of difficult samples, the edge device sends the gradient of difficult samples to the server as update information, ensuring the data security of the circuit board image and thus protecting the user's privacy. In addition, the edge device calculates the gradient based on the feature vectors and labels of difficult samples, and the server can directly update the first detection model based on the gradient, so that the edge device can share part of the server's calculations, thereby reducing the load on the server.

In a possible implementation manner, when mining difficult samples, the center offset information f _c (x), width offset information f _w (x) and width offset information f w (x) output by the first detection model for each circuit board image can be obtained Height offset information f _h (x), according to the center offset information f _c (x), width offset information f _w (x) and height offset information f _h (x), the corresponding position of each circuit board image can be determined respectively. Uncertainty score U, and then several circuit board images with corresponding larger uncertainty scores U can be determined as difficult samples.

The center offset information f _c (x) can indicate the center offset of the target object in the circuit board image, and the width offset information f _w (x) can indicate the width offset and height offset information of the target object in the circuit board image. f _h (x) can indicate the height offset of the target object in the circuit board image. Among them, the target objects include images of various components on the circuit board.

After inputting the circuit board image into the first detection model, the first detection model can integrate the center offset, width offset and height offset of each circuit board image into center offset information f _c (x) , width offset information f _w (x) and height offset information f _h (x), center offset information f _c (x), width offset information f _w (x) and height offset information f _h (x) The uncertainty score U corresponding to each circuit board image can be calculated as basic information.

The uncertainty score U can indicate the accuracy of the first detection model in abnormal detection of the corresponding circuit board. The higher the uncertainty score U, the lower the accuracy of the first detection model in abnormal detection of the corresponding circuit board, so that the first detection model can detect the abnormality of the corresponding circuit board. Several circuit board images with large uncertainty scores U are determined to be difficult samples. Specifically, at least one circuit board image with a corresponding uncertainty score U greater than a preset threshold may be determined as a difficult sample, or at least one circuit board image with a corresponding uncertainty score U greater than a preset threshold may be determined as a difficult sample.

In the embodiment of the present application, after obtaining the center offset information f _c (x), width offset information f _w (x) and height offset information f _h (x) of each circuit board image, according to the corresponding center offset Shift information f _c (x), width offset information f _w (x) and height offset information f _h (x) are used to determine the uncertainty score U of each circuit board image respectively, ensuring the determined uncertainty The score U can accurately indicate the accuracy of the first detection model in detecting anomalies on the corresponding circuit board, and then when determining difficult samples based on the uncertainty score U, the accuracy of the determined difficult samples can be guaranteed, thereby ensuring that the first detection model Validity of making updates.

In a possible implementation manner, the center offset information f _c (x) of each circuit board image includes the center offset mean μ _c and the center offset variance σ _c , and the width offset information f _w (x) includes the width The offset mean μ _w and the width offset variance σ _w , and the height offset information f _h (x) includes the height offset mean μ _h and the height offset variance σ _h . For each circuit board image, according to the center offset mean μ _c and center offset variance σ _c of the circuit board image, the center uncertainty score U _c of the circuit board image can be determined. According to the width deviation of the circuit board image, The width uncertainty score _{U w of} the circuit board image can be determined based on the moving mean μ _w and the width offset variance σ w . According to the height offset mean μ _h and height offset variance σ _h of the circuit board image, the width uncertainty score U w can be determined. The high uncertainty score U _h of the circuit board image. Then, based on the center uncertainty score U _c , width uncertainty score U _w and height uncertainty score U _h corresponding to each circuit board image, the uncertainty score U of the circuit board image can be determined.

For each circuit board image, through variational reasoning, the center offset mean μ _c , center offset variance σ _c , width offset mean μ _w , width offset variance σ _w , corresponding to the circuit board image can be obtained. The height offset mean μ _h and the height offset variance σ _h . By inputting the center offset mean μ _c and the center offset variance σ _c into the Bayesian active learning by disagreement (BALD) acquisition function, the center uncertainty score U _c can be obtained, and the width offset mean μ _w And the width offset variance σ _w is input into the BALD acquisition function, and the width uncertainty score U _w can be obtained. The height offset mean μ _h and the height offset variance σ _h are input into the BALD acquisition function, and the height uncertainty score U _h can be obtained. .

In the embodiment of the present application, the mean center offset μ _c , center offset variance σ c , width offset mean μ _w , width offset variance σ _w , _and height offset mean of the circuit board image are determined through variational reasoning. μ _h and height offset variance σ _h . Based on these data, the center uncertainty score U c , width uncertainty score U w and height uncertainty score U h of the circuit board image can be determined through the BALD acquisition function, and then the center uncertainty score U _c , the width uncertainty score U _w and the height uncertainty score U _h can be determined. Determine the uncertainty score U corresponding to the circuit board image based on the center uncertainty score U _c , the width uncertainty score U _w and the height uncertainty score U _h , ensuring that the calculated uncertainty score U can accurately indicate the first detection The accuracy of the model's anomaly detection on the corresponding circuit board, thereby ensuring the accuracy of the identified difficult samples.

In one possible implementation method, the uncertainty score corresponding to the circuit board image is determined based on the center uncertainty score U _c , the width uncertainty score U _w and the height uncertainty score U _h corresponding to the circuit board image. When U, the average or maximum value of the center uncertainty score U _c , the width uncertainty score U _w and the height uncertainty score U _h corresponding to the circuit board image can be determined as the uncertainty corresponding to the circuit board image. Sex rating U.

In the embodiment of this application, the average or maximum value of the center uncertainty score U _c , the width uncertainty score U _w and the height uncertainty score U _h is determined as the uncertainty score U of the circuit board image, It is guaranteed that the determined uncertainty score U can accurately indicate the accuracy of abnormality detection of the corresponding circuit board by the first detection model, thereby ensuring the accuracy of the determined difficult sample.

An update method for circuit board inspection models applied to servers

FIG. 2 is a flow chart of a method for updating a circuit board detection model according to another embodiment of the present application. The method 200 for updating a circuit board detection model can be applied to a server. As shown in Figure 2, the circuit board inspection model updating method 200 includes the following steps:

Step 201: Receive update information.

The server can receive updated information from edge devices. The updated information is determined by the edge device based on the feature vector and label of at least one difficult sample, and the feature vector of the difficult sample is obtained by inputting the circuit board image into the first detection model. The update information is used to update the first detection model.

Step 202: Update the first detection model.

The first detection model used by the edge device is stored on the server. After the server obtains the update information, the server can update the first detection model according to the update information to obtain the second detection model.

Step 203: Send the second detection model to the edge device.

After the server completes updating the first detection model and obtains the second detection model, the server can deliver the second detection model to the edge device, so that the edge device can perform anomaly detection on the circuit board through the second detection device.

In the embodiment of the present application, since the update information is determined based on the feature vectors and labels of difficult samples, the difficult samples are only a part of all the circuit board images input into the first detection model, so only the difficult samples need to be annotated to obtain Labeling of difficult samples without the need to label all circuit board images input to the first detection model, thereby reducing a large amount of labor costs and man-hours for labeling samples. In addition, the server updates the first detection model based on the update information. The server can automatically complete the update action of the first detection model and automatically deliver the updated second detection model to the edge device. The model update and delivery process does not require manual labor. Participation can further reduce labor costs and corresponding working hours.

In one possible implementation manner, after the server receives the updated information from the edge device, the server can perform data cleaning on the updated information, and then update the first detection model based on the updated information after data cleaning.

By performing data cleaning on the update information, some invalid data in the update information can be removed. When performing data cleaning on updated information, incorrectly labeled samples can be filtered out based on the prediction results of multiple edge devices for the same sample. Specifically, for the same difficult sample, the prediction results of multiple edge devices for the difficult sample through their respective detection models are obtained. If the consistency of the prediction results of each edge device for the difficult sample meets the preset conditions, the prediction results in the update information are retained. Information corresponding to the difficult sample. If the consistency of the prediction results of the difficult sample by each edge device does not meet the preset conditions, the information corresponding to the difficult sample will be deleted from the updated information. The preset condition may be that the ratio of the number of edge devices with consistent prediction results to the total number of edge devices is greater than a preset threshold, such as greater than 80%.

Since the update information can be the feature vectors and labels of difficult samples, or the gradient of difficult samples, data cleaning of the update information is to clean out the feature vectors and labels of wrongly labeled samples, or clean out the gradients of wrongly labeled samples.

In the embodiment of the present application, when obtaining the labels of difficult samples, there may be label errors for some difficult samples. Through data cleaning, the information of difficult samples with incorrect labels in the update information can be removed, thereby ensuring that the server updates the first detection model. efficiency and accuracy.

In a possible implementation, the update information includes the feature vector and label of at least one difficult sample, or the update information includes the gradient of at least one difficult sample, wherein the gradient of a difficult sample is based on the feature vector and label of the difficult sample Sure.

In this embodiment of the present application, when the update information includes the feature vectors and labels of difficult samples, the server can accurately update the first detection model based on the feature vectors and labels of difficult samples to ensure the model update effect. When the update information includes the gradient of a difficult sample, the server can update the first detection model based on the difficult sample. There is no need to preprocess the difficult sample before updating the model, which can reduce the load on the server.

Update system for circuit board inspection models

Figure 3 is a schematic diagram of a circuit board detection model update system according to an embodiment of the present application. As shown in FIG. 3 , a circuit board detection model update system 300 includes an edge device 10 and a server 20 . The edge device 10 is configured to execute the above-mentioned updating method 100 of a circuit board detection model applied to an edge device. The server 20 is configured to execute the above updating method 200 for the circuit board detection model applied to the server.

The edge device 10 is configured to generate update information for updating the first detection model through the input at least one circuit board image and the deployed first detection model, and send the update information to the server 20 . The server 20 is configured to receive the update information sent by the edge device 10 , update the first detection model according to the update information, obtain the second detection model, and send the updated second detection model to the edge device 10

In the embodiment of the present application, after the edge device 10 obtains the first detection model, the difficult sample can be obtained by mining at least one input circuit board image for difficult samples. By manually annotating the difficult samples, all inputs can be eliminated. Annotating circuit board images can reduce a large amount of labor costs and man-hours used to annotate samples. By updating the first detection model through the server 20, the server automatically completes the update action of the first detection model, and the server 20 automatically delivers the updated second detection model to the server, which also reduces labor costs and corresponding man-hours.

Obtaining updated information through the feature vectors and labels of difficult samples can improve the accuracy of the updated information. The server uses the updated information to update the first detection model to a second detection model that is more suitable for the current environment, which can improve the effect of anomaly detection.

In one possible implementation, the server 20 is a cloud server.

In the embodiment of the present application, by using a cloud server, the cloud server and multiple edge devices 10 can transmit information through the network. The cloud server updates the first detection model by obtaining the update information uploaded by each edge device 10, and updates each edge device 10. The second detection model corresponding to the edge device 10 is delivered to each edge device 10 to complete the update of the first detection model, which reduces the amount of settings on the server 20 and can support the first detection model deployed on large-scale edge devices 10 Make an update.

An update method applied to update system circuit board inspection models

Based on the circuit board detection model update system 300 in the above embodiment, embodiments of the present application provide a circuit board detection model update method. Figure 4 is a circuit board detection model update method 400 according to an embodiment of the present application. The circuit board detection model update method 400 can be executed by the edge device 10 and the server 20 in the previous embodiment. Unless otherwise stated, the following method is implemented. The edge device in the example may be the edge device 10 in the previous embodiment, and the server in the following method embodiment may be the server 20 in the previous embodiment. As shown in Figure 4, the circuit board detection model update method 400 includes the following steps:

Step 401: The server obtains the first detection model.

By performing deep learning on a certain amount of circuit board images, a first detection model is obtained, and the first detection model is used to detect abnormalities on the circuit board based on the circuit board images. By acquiring the first detection model, the server can deliver the first detection model to at least one edge device.

Step 402: The server delivers the first detection model to the edge device.

After the server obtains the first detection model, it delivers the first detection model to at least one edge device respectively, so that the edge device can detect anomalies on the circuit board through the first detection model.

Step 403: The edge device obtains at least one circuit board image.

The circuit board image can be an image of the circuit board to be inspected during the actual production process of the circuit board. The circuit board after installation can be captured by a camera or other image capturing device to obtain at least one circuit board image, and the at least one circuit board image can be Sent to edge device.

Step 404: The edge device inputs at least one circuit board image into the first detection model.

After the edge device obtains at least one circuit board image, the obtained at least one circuit board image is input into the deployed first detection model, the input at least one circuit board image can be processed, and the information of each circuit board is obtained. Feature vector.

Step 405: The edge device performs difficult sample mining on at least one circuit board image.

After each circuit board image is input into the first detection model, each circuit board image is processed through the first detection model. After completing the processing, difficult samples are mined for all samples to obtain at least one difficult sample and the number of difficult samples. Less than or equal to the number of board images entered.

Step 406: The edge device obtains the label of each difficult sample respectively.

For each difficult sample excavated, each difficult sample is labeled separately, and the label corresponding to each difficult sample is obtained. The label is used to indicate whether there is an abnormality in the corresponding difficult sample (circuit board image). Among them, labeling difficult samples can be completed through manual annotation. Difficult samples obtain the label of each labeled difficult sample separately.

Step 407: The edge device obtains the gradient according to the feature vector and label of each difficult sample, and uses the gradient as update information.

According to each difficult sample that is mined, the edge device obtains the feature vector of each difficult sample based on the feature vector of each circuit board image that has been obtained. According to the feature vector and label of each difficult sample, the gradient corresponding to the difficult sample is calculated, and the gradient of each difficult sample is determined as the update information. The calculation of the gradient is implemented in the edge device through the back propagation algorithm (Back propagation).

Step 408: The edge device sends the update information to the server.

After the edge device obtains the update information, it sends the update information to the server, so that the server updates the first detection model through the update information and obtains the second detection model.

Step 409: The server receives the update information sent by the edge device and performs data cleaning on the update information.

After the server receives the update information from the edge device, the server performs data cleaning on the update information and filters out the information gradients of difficult samples with incorrect labels in the update information.

Step 410: The server updates the first detection model according to the updated information after data cleaning.

After the server performs data cleaning on the updated information, the server updates the first detection model based on the updated information after data cleaning to obtain a second detection model.

Step 411: The server delivers the second detection model to the edge device.

After the server completes updating the first detection model and obtains the second detection model, the server delivers the second detection model to the edge device, so that the edge device performs anomaly detection on the circuit board through the second detection device.

In the embodiment of the present application, after the edge device obtains the first detection model, the difficult sample can be obtained by mining at least one input circuit board image for difficult samples. By manually labeling the difficult samples, all input Annotating circuit board images can reduce a large amount of labor costs and man-hours used to annotate samples. By updating the first detection model through the server, the server automatically completes the update action of the first detection model, and the server automatically delivers the updated second detection model to the server, which also reduces labor costs and corresponding working hours.

The gradient is obtained through the feature vector and label of the difficult sample, and the updated information is obtained based on the gradient. By calculating the feature vector and label of each difficult sample as a gradient and transmitting it to the server, it replaces the transmission of each difficult sample to the server. Feature vectors and labels. At this time, the data obtained by the server is the gradient, which ensures the security of the updated information and protects the user's privacy. At the same time, the process of calculating the gradient is implemented in the edge device, which can reduce the workload of the server and improve the operating efficiency of the server. The server updates the first detection model to a second detection model that is more suitable for the current environment by updating the information, which can improve the effect of anomaly detection.

Electronic equipment

FIG. 5 is a schematic diagram of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device. As shown in Figure 5, the electronic device 50 may include: a processor (processor) 51, a communications interface (Communications Interface) 52, a memory (memory) 53, and a communication bus 54. in:

The processor 51, the communication interface 52, and the memory 53 complete communication with each other through the communication bus 54.

Communication interface 52 is used to communicate with other electronic devices or servers.

The processor 51 is configured to execute the program 55. Specifically, the processor 51 can execute the relevant steps in the foregoing embodiment of the method for updating the circuit board detection model.

Specifically, program 55 may include program code including computer operating instructions.

The processor 51 may be a CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors included in the smart device can be the same type of processor, such as one or more CPUs; or they can be different types of processors, such as one or more CPUs and one or more ASICs.

Memory 53 is used to store programs 55. The memory 53 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 55 can be specifically used to cause the processor 51 to execute the updating method of the circuit board detection model in the previous embodiment.

For the specific implementation of each step in the program 55, please refer to the corresponding steps and corresponding descriptions in the units in the foregoing embodiment of the method for updating the circuit board detection model, which will not be described again here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be described again here.

Through the electronic device of the embodiment of the present application, after the edge device obtains the first detection model, the difficult sample can be obtained by mining at least one input circuit board image for difficult samples. By manually annotating the difficult samples, all the difficult samples can be obtained. Labeling the input circuit board images can reduce a lot of labor costs and man-hours for labeling samples. By updating the first detection model through the server, the server automatically completes the update action of the first detection model, and the server automatically delivers the updated second detection model to the server, which also reduces labor costs and corresponding working hours.

computer storage media

The present application also provides a computer-readable storage medium that stores instructions for causing a machine to execute the updating method of a circuit board inspection model as described herein. Specifically, a system or device equipped with a storage medium may be provided, on which the software program code that implements the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device ) reads and executes the program code stored in the storage medium.

In this case, the program code itself read from the storage medium can implement the functions of any one of the above embodiments, and therefore the program code and the storage medium storing the program code form part of this application.

Examples of storage media for providing program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), Tapes, non-volatile memory cards and ROM. Alternatively, the program code can be downloaded from the server computer via the communications network.

computer program product

Embodiments of the present application also provide a computer program product, including computer instructions, which instruct the computing device to perform any corresponding operation in the multiple method embodiments mentioned above.

It should be pointed out that according to the needs of implementation, each component/step described in the embodiments of this application can be split into more components/steps, or two or more components/steps or partial operations of components/steps can be combined into New components/steps to achieve the purpose of the embodiments of this application.

The above-mentioned methods according to the embodiments of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or by The computer code downloaded by the network is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium, so that the method described here can be stored using a general-purpose computer, a special-purpose processor or a programmable computer. or such software processing on a recording medium of dedicated hardware such as ASIC or FPGA. It will be understood that a computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code when the software or computer code is used by the computer, When accessed and executed by a processor or hardware, the methods described herein are implemented. Furthermore, when a general-purpose computer accesses code for implementing the methods illustrated herein, execution of the code converts the general-purpose computer into a special-purpose computer for performing the methods illustrated herein.

It should be noted that not all steps and modules in the above-mentioned processes and system structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by multiple Some components in separate devices are implemented together.

In the above embodiments, the hardware module can be implemented mechanically or electrically. For example, a hardware module may include permanently dedicated circuitry or logic (such as a specialized processor, FPGA, or ASIC) to complete the corresponding operation. Hardware modules may also include programmable logic or circuits (such as general-purpose processors or other programmable processors), which can be temporarily set by software to complete corresponding operations. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily installed circuit) can be determined based on cost and time considerations.

The present invention has been shown and described in detail through the drawings and preferred embodiments above. However, the present invention is not limited to these disclosed embodiments. Based on the above-mentioned multiple embodiments, those skilled in the art will know that the above-mentioned different embodiments can be combined. The code review means in the method can lead to more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims (14)

1. A circuit board detection model updating method (100), applied to edge devices (10), the method includes:

   Input at least one circuit board image into a first detection model to obtain a feature vector of each circuit board image, wherein the first detection model is used to detect abnormalities on the circuit board based on the circuit board image;

   Perform difficult sample mining on the at least one circuit board image to obtain at least one difficult sample;

   Obtain the label of each difficult sample separately;

   Obtain updated information based on the feature vectors and labels of each difficult sample;

   Send the update information to the server (20), so that the server updates the first detection model according to the update information to obtain a second detection model;

   Obtain the second detection model issued by the server (20), and perform anomaly detection on the circuit board through the second detection model.
2. The method according to claim 1, wherein the update information includes feature vectors and labels of each of the difficult samples.
3. The method according to claim 1, wherein said obtaining update information according to the feature vector and label of each difficult sample includes:

   For each difficult sample, calculate the gradient corresponding to the difficult sample based on the feature vector and label of the difficult sample;

   The gradient corresponding to each difficult sample is determined as the update information.
4. The method according to claim 1, wherein said performing difficult sample mining on the at least one circuit board image to obtain at least one difficult sample includes:

   For each circuit board image, obtain the center offset information, width offset information and height offset information output by the first detection model for the circuit board image, where the center offset information is used to indicate the circuit The center offset of the target object in the board image. The width deviation information is used to indicate the width offset of the target object in the circuit board image. The height deviation information is used to indicate the height offset of the target object in the circuit board image. ;

   For each circuit board image, determine the uncertainty score corresponding to the circuit board image based on the center offset information, width offset information and height offset information corresponding to the circuit board image;

   At least one of the circuit board images with a corresponding uncertainty score greater than a preset threshold is determined as a difficult sample, or the circuit board image with a corresponding uncertainty score greater than a preset threshold is determined as a difficult sample.
5. The method according to claim 4, wherein, for each circuit board image, determining the corresponding center offset information, width offset information and height offset information of the circuit board image corresponding to the circuit board image. uncertainty scores, including:

   For each of the board images described, do:

   Based on the center offset mean and center offset variance included in the center offset information corresponding to the circuit board image, determine the center uncertainty score corresponding to the circuit board image;

   Determine the width uncertainty score corresponding to the circuit board image based on the width offset mean and width offset variance included in the width offset information corresponding to the circuit board image;

   Determine the height uncertainty score corresponding to the circuit board image based on the height offset mean value and height offset variance included in the height offset information corresponding to the circuit board image;

   The uncertainty score corresponding to the circuit board image is determined based on the center uncertainty score, width uncertainty score and height uncertainty score corresponding to the circuit board image.
6. The method according to claim 5, wherein the uncertainty score corresponding to the circuit board image is determined based on the center uncertainty score, width uncertainty score and height uncertainty score corresponding to the circuit board image, include:

   For each circuit board image, the average or maximum value of the center uncertainty score, width uncertainty score, and height uncertainty score corresponding to the circuit board image is determined to be the different values corresponding to the circuit board image. Certainty score.
7. A circuit board detection model updating method (200), applied to the server (20), the method includes:

   Receive updated information from the edge device (10), wherein the updated information is determined based on a feature vector and a label of at least one difficult sample, the feature vector of the difficult sample being obtained by inputting the circuit board image into the first detection model;

   Update the first detection model according to the update information to obtain a second detection model;

   The second detection model is delivered to the edge device (10), so that the edge device performs anomaly detection on the circuit board through the second detection model.
8. The method according to claim 7, wherein updating the first detection model according to the update information to obtain a second detection model includes:

   Perform data cleaning on the updated information;

   The first detection model is updated according to the updated information after data cleaning to obtain the second detection model.
9. The method according to claim 7, wherein the update information includes a feature vector and a label of the at least one difficult sample, or the update information includes a gradient of at least one difficult sample, wherein one of the difficult samples The gradient is determined based on the feature vector and label of this difficult sample.
10. A circuit board inspection model updating system (300), including: an edge device (10) and a server (20);

    The edge device (10) is used to execute the updating method of the circuit board detection model described in any one of the above claims 1-6;

    The server (20) is configured to execute the updating method of the circuit board detection model described in any one of claims 7-9.
11. The system according to claim 10, wherein the server (20) is a cloud server.
12. An electronic device, including: a processor (51), a communication interface (52), a memory (53) and a communication bus (54), the processor (51), the memory (53) and the communication interface (53) 52) Complete mutual communication through the communication bus (54);

    The memory (53) is used to store at least one executable instruction. The executable instruction causes the processor (51) to execute the updating method of the circuit board detection model as claimed in any one of claims 1-6 or as claimed in claim 7. -Operations corresponding to the update method of the circuit board detection model described in any one of -9.
13. A computer storage medium on which a computer program is stored. When the program is executed by a processor, the updating method of a circuit board detection model as described in any one of claims 1-6 or any of claims 7-9 is implemented. A method for updating a circuit board inspection model.
14. A computer program product, including computer instructions, the computer instructions instruct a computing device to perform the updating method of a circuit board inspection model as claimed in any one of claims 1-6 or as claimed in any one of claims 7-9. The operations corresponding to the update method of the circuit board inspection model described above.

Images