CN117078620B - PCB welding spot defect detection method and device, electronic equipment and storage medium - Google Patents

PCB welding spot defect detection method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117078620B
CN117078620B CN202311027218.1A CN202311027218A CN117078620B CN 117078620 B CN117078620 B CN 117078620B CN 202311027218 A CN202311027218 A CN 202311027218A CN 117078620 B CN117078620 B CN 117078620B
Authority
CN
China
Prior art keywords
welding spot
detected
image
defect detection
welding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311027218.1A
Other languages
Chinese (zh)
Other versions
CN117078620A (en
Inventor
沈建华
洪乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chint Group R & D Center Shanghai Co ltd
Original Assignee
Chint Group R & D Center Shanghai Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chint Group R & D Center Shanghai Co ltd filed Critical Chint Group R & D Center Shanghai Co ltd
Priority to CN202311027218.1A priority Critical patent/CN117078620B/en
Publication of CN117078620A publication Critical patent/CN117078620A/en
Application granted granted Critical
Publication of CN117078620B publication Critical patent/CN117078620B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30141—Printed circuit board [PCB]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The embodiment of the application provides a PCB welding spot defect detection method, a device, electronic equipment and a storage medium, which relate to the technical field of target detection and are used for acquiring a PCB image to be detected; carrying out welding spot recognition on the PCB image to be detected based on the welding spot detection layer in the trained detection model, and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; performing welding spot defect detection through a defect detection layer in the trained detection model according to the welding spot type and welding spot position data of the welding spot to be detected and the PCB image to be detected, and obtaining a welding spot defect detection result of the PCB image to be detected; according to the method and the device, the defect detection is carried out on the PCB image to be detected based on the trained detection model comprising the welding spot detection layer and the defect detection layer, so that the detection efficiency is improved; and by adapting to the detection of the welding spot defects of different welding spot types, the accuracy of the follow-up defect detection result is improved, so that the misjudgment rate of the welding spot defect detection is reduced.

Description

PCB welding spot defect detection method and device, electronic equipment and storage medium
Technical Field
The invention relates to the technical field of target detection, in particular to a PCB welding spot defect detection method, a device, electronic equipment and a storage medium.
Background
In the production process of the release, the quality of welding spots of components on a PCB of the release needs to be detected in the factory, wherein the main defects of the quality of the welding spots include the defects of welding leakage, virtual welding, multi-tin and the like.
In the prior art, two methods of target detection and image classification are mostly used for intelligent auxiliary calculation of PCB welding spot defect detection, wherein a classification model method based on image processing is more common, morphological processing modes such as binarization, opening and closing operation and the like are used for PCB welding spot images, images are processed through some edge extraction algorithms, welding spot positions are obtained and classified, then characteristics of each classification are learned, and the obtained PCB welding spot images and real images are assumed to have a certain specific relation, and the corresponding classification of each welding spot is judged through a probability training relation model. The detection result has misjudgment and low detection efficiency.
Disclosure of Invention
The embodiment of the application provides a PCB welding spot defect detection method, a device, electronic equipment and a storage medium, so as to solve the problems of high misjudgment probability and low detection efficiency of the existing PCB welding spot defect detection method.
In one aspect, an embodiment of the present application provides a method for detecting a solder joint defect of a PCB, where the method includes:
Acquiring a PCB image to be detected;
performing welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected; the welding spot types include circular welding spots and long welding spots.
In another aspect, an embodiment of the present application provides a device for detecting a solder joint defect of a PCB, the device including:
The acquisition module is used for acquiring the PCB image to be detected;
the detection module is used for carrying out welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected; the welding spot types include circular welding spots and long welding spots.
In another aspect, an embodiment of the present application provides an electronic device including a memory and a processor; the memory stores an application program, and the processor is used for running the application program in the memory so as to execute the operation in the PCB solder joint defect detection method.
In another aspect, embodiments of the present application provide a storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the PCB solder joint defect detection method described above.
The embodiment of the application provides a PCB welding spot defect detection method, a device, electronic equipment and a storage medium, which relate to the technical field of target detection and are used for acquiring a PCB image to be detected; carrying out welding spot recognition on the PCB image to be detected based on the welding spot detection layer in the trained detection model, and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the welding spot type comprises a round welding spot and an elongated welding spot; performing welding spot defect detection through a defect detection layer in a trained detection model according to the welding spot type and welding spot position data of the welding spot to be detected and an image area corresponding to the welding spot to be detected in the PCB image to be detected, and obtaining a welding spot defect detection result of the PCB image to be detected; according to the embodiment of the application, the defect detection is carried out on the PCB image to be detected based on the trained detection model comprising the welding spot detection layer and the defect detection layer, two different detection models are not required to be additionally deployed, and the detection efficiency is improved; and the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected are determined through the welding spot detection layer, and then the defect detection is carried out through the defect detection layer based on the welding spot type and the welding spot position data of the welding spots to be detected and the PCB image to be detected, so that the accuracy of the follow-up defect detection result is improved through adapting the welding spot defect detection of different welding spot types, and the misjudgment rate of the welding spot defect detection is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a method for detecting a solder joint defect of a PCB according to an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of a method for detecting a solder joint defect of a PCB according to an embodiment of the present disclosure;
FIG. 3 is a schematic structural diagram of a trained test model provided in an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a solder joint detection layer according to an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of a defect detection layer according to an embodiment of the present disclosure;
FIG. 5a is a schematic diagram of a feature extractor provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of a solder joint type provided by an embodiment of the present application;
fig. 7 is a schematic diagram of a solder joint to be detected in a PCB image to be detected according to an embodiment of the present application;
FIG. 8 is a flowchart of a method for recognizing a welding spot based on image preprocessing according to an embodiment of the present application;
FIG. 9 is a schematic flow chart of a method for identifying welding spots according to an embodiment of the present application;
FIG. 10 is a schematic structural diagram of another trained test model provided by an embodiment of the present application;
FIG. 11 is a flowchart of a method for detecting a solder joint defect based on a solder joint type according to an embodiment of the present application;
FIG. 12 is a flow chart of a training method of a detection model according to an embodiment of the present application;
fig. 13 is a schematic structural diagram of a PCB solder joint defect detecting apparatus according to an embodiment of the present disclosure;
fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
It should be noted that: references herein to "a plurality" means two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., a and/or B may represent: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
As described in the background art, in the prior art, two methods of target detection and image classification are mostly used for intelligent auxiliary calculation of the detection of the welding spot defect of the PCB welding plate, which are affected by the large difficulty in identifying the type and position of the welding spot, and in the existing scheme for detecting the welding spot defect of the PCB welding plate, the misjudgment probability is large, and in order to ensure the accuracy of the defect detection result, the position of the welding spot to be detected in the PCB plate needs to be found based on the detection scheme before the detection of the welding spot, then the scheme for detecting the welding spot defect is executed to determine whether the welding spot to be detected has the defect, so that the detection in two stages needs to be executed, and the detection efficiency is low. And the defects of the welding spots with different welding spot types are difficult to detect due to the influence of the diversity of the welding spot types, so that the misjudgment probability is further increased.
Based on the method, the device, the electronic equipment and the storage medium for detecting the defects of the welding spots in the PCB, in order to improve the detection accuracy and the detection efficiency of the detection of the defects of the welding spots in the PCB, the embodiment of the application provides the method, the device, the electronic equipment and the storage medium for detecting the defects of the welding spots of the PCB, and the defects of the PCB image to be detected are detected based on the trained detection model comprising the welding spot detection layer and the defect detection layer, so that two different detection models do not need to be additionally deployed, and the detection efficiency is improved; and the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected are determined through the welding spot detection layer, and then the defect detection is carried out through the defect detection layer based on the welding spot type and the welding spot position data of the welding spots to be detected and the PCB image to be detected, so that the accuracy of the follow-up defect detection result is improved through adapting the welding spot defect detection of different welding spot types, and the misjudgment rate of the welding spot defect detection is reduced.
In order to facilitate understanding of the technical scheme of the application, the method for detecting the defects of the welding spots of the PCB provided by the embodiment of the application is described below in combination with an actual application scene.
For an example, taking a PCB to be detected as a PCB in a tripper, an application scenario of a method for detecting a solder joint defect of a PCB is provided, referring to fig. 1, fig. 1 is an application scenario of a method for detecting a solder joint defect of a PCB provided in an embodiment of the present application, where the application scenario includes a transmission belt, a robot, a tripper, and a system for detecting a solder joint defect of a PCB. The PCB welding spot defect detection method is used for detecting welding spot defects of the PCB in the trippers on the conveying belt, and the robot is used for sorting the trippers with welding spot defects, for example, sorting the trippers with welding spot defects to defective conveying belts and sorting the trippers without welding spot defects to defective conveying belts. The welding spot with the welding spot defect may be a welding spot with a welding spot position which does not meet a preset position requirement, a welding spot shape which does not meet a preset shape requirement, and/or a welding spot size which does not meet a preset size requirement, which is not specifically limited in the embodiment of the present application.
Illustratively, as shown in fig. 1, the PCB solder joint defect detection system is disposed on a transmission belt, and performs solder joint defect detection on a PCB board in a release on the transmission belt, and as shown in fig. 1, the PCB solder joint defect detection system includes an image sensor, a detection unit, and a communication unit. The detection unit is deployed with a trained detection model provided by embodiments of the present application.
Specifically, in the application scenario, the PCB solder joint defect detection step includes:
(1) The NXBLE-32 tripper is transmitted to the PCB welding spot defect detection system through a transmission belt, and after the PCB welding spot defect detection system monitors that the tripper is in place, an image sensor in the PCB welding spot defect detection system is called to collect the image of the PCB in the tripper, so that the image of the PCB to be detected is obtained, and the image of the PCB to be detected is transmitted to the detection unit.
(2) The detection unit carries out welding spot identification on the PCB image to be detected based on the welding spot detection layer in the deployed trained detection model, determines the welding spot type and welding spot position data of the welding spot to be detected of the PCB image to be detected, carries out image processing on the PCB image to be detected according to the welding spot position data of the welding spot to be detected, obtains an image area corresponding to the welding spot to be detected in the PCB image to be detected, carries out welding spot defect detection on the image area corresponding to the welding spot to be detected in the PCB image to be detected according to the welding spot type of the welding spot to be detected and the defect detection layer in the trained detection model, obtains a welding spot defect detection result of the PCB image to be detected, and transmits the welding spot defect detection result to the communication unit; the solder joint types include circular solder joints and elongated solder joints.
(3) The communication unit sends the welding spot defect detection result to the robot.
(4) The robot sorts the trippers with welding spot defects to defective product conveying belts based on welding spot defect detection results, and sorts the trippers without welding spot defects to defective product conveying belts.
Alternatively, the image sensor may be a CCD sensor or an industrial camera.
In the application scene provided by the embodiment of the application, the trained detection model comprising the welding spot detection layer and the defect detection layer is used for detecting the defects of the PCB image to be detected, so that two different detection models are not required to be additionally deployed, and the detection efficiency is improved; and the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected are determined through the welding spot detection layer, and then the defect detection is carried out through the defect detection layer based on the welding spot type and the welding spot position data of the welding spots to be detected and the PCB image to be detected, so that the accuracy of the follow-up defect detection result is improved through adapting the welding spot defect detection of different welding spot types, and the misjudgment rate of the welding spot defect detection is reduced.
Based on the application scenario shown in fig. 1, the embodiment of the present application provides a method for detecting a PCB solder joint defect, as shown in fig. 2, fig. 2 is a flow chart of the method for detecting a PCB solder joint defect provided in the embodiment of the present application, where the method for detecting a PCB solder joint defect may be executed by the PCB solder joint defect detecting system shown in fig. 1, or may be executed by an electronic device having a data processing capability, for example, a server cluster, a cloud server, or the like, which is not limited in particular. Specifically, the method for detecting the defects of the solder joints of the PCB shown in fig. 2 at least includes steps 210 to 220, which are described in detail as follows:
Step 210, obtaining an image of the PCB to be detected.
The PCB image to be detected can be an image of the PCB to be detected acquired in real time, or can be an image of the PCB sent by other equipment.
And 220, performing welding spot defect detection on the PCB image to be detected based on the trained detection model, and obtaining a welding spot defect detection result of the PCB image to be detected.
The trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model through a sample image.
In some embodiments, the trained detection model may be a neural network-based detection model or a machine learning-based detection model. Illustratively, taking a detection model in which the trained detection model is based on a neural network as an example, as shown in fig. 3, fig. 3 is a schematic structural diagram of the trained detection model provided in an embodiment of the present application, where the trained detection model includes an input layer, a solder joint detection layer, a defect detection layer, and an output layer.
The welding spot detection layer is used for carrying out welding spot detection and identification on the input PCB image to be detected, and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected. The defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected, so as to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected, so as to obtain a welding spot defect detection result of the PCB image to be detected. The output layer is used for outputting the welding spot defect detection result.
Optionally, the solder joint detection layer may be a detector built based on yolov8 neural network, or a detector built based on a master-cnn neural network, or a detector built based on a resnet neural network. Taking a detector in which the solder joint detection layer is built based on a yolov8 neural network as an example, as shown in fig. 4, fig. 4 is a schematic structural diagram of the solder joint detection layer provided in the embodiment of the present application, where the solder joint detection layer includes a feature extraction unit, a feature fusion unit and an identification unit. The feature extraction unit is used for extracting features of the PCB image to be detected to obtain image features of the PCB image to be detected; the feature fusion unit is used for carrying out fusion processing on the image features extracted by the feature extraction unit to obtain image fusion features; the recognition unit is used for carrying out feature calculation based on the image fusion features obtained by the feature fusion unit and determining the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected.
Alternatively, the defect detection layer may be a detector constructed based on a shufflelenet, or a detector constructed based on a mobileet. Exemplary, as shown in fig. 5, fig. 5 is a schematic structural diagram of a defect detection layer provided in an embodiment of the present application, where the defect detection layer includes a feature extractor, a probability predictor, and a classifier. The feature extractor is used for extracting features based on the welding spot type and the welding spot position data of the welding spots to be detected and the PCB image to be detected to obtain defect image features; the probability predictor is used for carrying out feature calculation based on the defect image features extracted by the feature extractor, and determining the prediction probability of defects of the welding spots to be detected; the classifier is used for obtaining a welding spot defect detection result of the PCB image to be detected based on the prediction probability obtained by the probability predictor, for example, regression processing is performed based on the prediction probability obtained by the probability predictor, and the welding spot defect detection result of the PCB image to be detected is obtained; for example, the prediction probability obtained by the probability predictor is compared with a preset probability threshold value, and a welding spot defect detection result of the PCB image to be detected is obtained based on the comparison result.
Optionally, as shown in fig. 5, the feature extractor provided in the embodiment of the present application includes a first convolution unit 501, a second convolution unit 502, a third convolution unit 503, and a fourth convolution unit 504; the probability predictor comprises three fully connected networks connected in series; the classifier includes a Softmax layer.
The output of the first convolution unit 501 is input to the second convolution unit 502 and the probability predictor, the output of the second convolution unit 502 is input to the third convolution unit 503 and the probability predictor, the output of the third convolution unit 503 is input to the fourth convolution unit 504 and the probability predictor, respectively, and the output of the fourth convolution unit 504 is input to the probability predictor.
As shown in fig. 5a, fig. 5a is a schematic structural diagram of a feature extractor provided in the embodiment of the present application, where a first convolution unit 501 includes three cascaded convolution networks, and a second convolution unit 502, a third convolution unit 503, and a fourth convolution unit 504, which have similar structures, all include a first convolution subunit 51 and a second convolution subunit 52, and the first convolution subunit 51 and the second convolution subunit 52 are connected in parallel. In some embodiments, the first convolution subunit 51 includes 3 concatenated convolution networks and the second convolution subunit 52 includes one convolution network of 3*3. It should be noted that, in the embodiment of the present application, the convolution kernel size of the convolution network in the first convolution unit 501 is not specifically limited, and may be set according to practical application scenarios, for example, the sizes of the convolution networks in the first convolution unit 501 are 3*3, 7*7, 3*3, the sizes of the convolution networks in the first convolution subunit 51 in the second convolution unit 502 are 3*3, 5*5, and 3*3, the parameters of the convolution network in the second convolution subunit 52 in the third convolution unit 503 are the same as those of the second convolution unit 502, and the sizes of the convolution networks in the first convolution subunit 51 in the fourth convolution unit 504 are 3*3, 3*3, and 3*3, respectively.
When the defect detection layer detects defects, based on the type of welding spots and the position data of the welding spots to be detected and the PCB image to be detected, inputting the data of the welding spots to be detected into a first convolution layer for feature extraction, respectively inputting the output of a first convolution unit 501 into a probability predictor, and inputting the output of a first convolution subunit 51 and a second convolution subunit 52 of a second convolution unit 502 into the probability predictor of the second convolution unit 502 after the respective outputs of the first convolution subunit 51 and the second convolution subunit 52 in the second convolution unit 502 are overlapped; the output of the second convolution unit 502 is input to the probability predictor, the first convolution subunit 51 and the second convolution subunit 52 of the third convolution unit 503, and the output of the first convolution subunit 51 and the second convolution subunit 52 of the third convolution unit 503 is superimposed and then used as the output of the third convolution unit 503, and the output of the second convolution subunit 52 of the third convolution unit 503 is input to the probability predictor; the output of the third convolution unit 503 is input to the probability predictor, the first convolution subunit 51 and the second convolution subunit 52 of the fourth convolution unit 504, and the output of each of the first convolution subunit 51 and the second convolution subunit 52 in the fourth convolution unit 504 is superimposed and then used as the output of the fourth convolution unit 504, the output of the fourth convolution unit 504 is input to the probability predictor, and the output of the second convolution subunit 52 of the fourth convolution unit 504 is input to the probability predictor; the probability predictor performs characteristic calculation based on the output of each of the first convolution unit 501, the second convolution unit 502, the third convolution unit 503, the fourth convolution unit 504, the second convolution subunit 52 in the second convolution unit 502, the second convolution subunit 52 in the third convolution unit 503, and the second convolution subunit 52 in the fourth convolution unit 504, and determines the prediction probability that the welding spot to be detected has a defect; the classifier is used for obtaining a welding spot defect detection result of the PCB image to be detected based on the prediction probability obtained by the probability predictor.
Optionally, when it is determined that a welding spot defect exists, in order to achieve rapid positioning of the defective welding spot, the welding spot detection layer and the defect detection layer may be respectively connected with the output layer, as shown in fig. 3, where the output layer receives the welding spot defect detection result output by the defect detection layer, if the welding spot defect detection result represents that the welding spot defect exists, the welding spot position data of the defective welding spot is determined based on the welding spot position data of the welding spot to be detected output by the welding spot detection layer, and the final welding spot defect detection result is output based on the welding spot position data of the defective welding spot. For example, the output layer may mark the PCB image to be detected based on the solder joint position data of the defective solder joint, and output the marked PCB image as a final solder joint defect detection result, and the output layer may generate a detection report based on the solder joint position data of the defective solder joint, and output the detection report as a final solder joint defect detection result. The detection report comprises a to-be-detected welding spot with a welding spot defect and welding spot position data of the to-be-detected welding spot with the welding spot defect.
In some embodiments, the weld pattern includes a circular weld and an elongated weld. As shown in fig. 6, fig. 6 is a schematic view of a solder joint type provided in the embodiment of the present application, fig. 6 (a) is a schematic view of a solder joint type being a circular solder joint, and fig. 6 (b) is a schematic view of a solder joint type being an elongated solder joint.
Alternatively, the pads to be detected may be each pad in the image of the PCB to be detected, as shown in fig. 7 (a); alternatively, the solder joint to be detected may also be a solder joint at a preset target position in the image of the PCB to be detected, as shown in (b) of fig. 7, and the solder joint to be detected may be a solder joint in a mark frame in the image of the PCB to be detected.
In some embodiments, solder joint position data is used to characterize the position coordinates of the solder joint to be inspected in the image of the PCB to be inspected. Optionally, the solder joint position data may be a position coordinate of a center point of the solder joint to be detected in the PCB image to be detected; optionally, the welding spot position data may also be the position coordinates of the center point of the detection frame of the welding spot to be detected in the image of the PCB to be detected; alternatively, the solder joint position data may also be the position coordinates of the vertex of the detection frame of the solder joint to be detected in the image of the PCB to be detected. The center point may be a geometric center point or a center of gravity point.
In some embodiments, the solder joint defect detection result is used for representing whether a solder joint to be detected with a solder joint defect exists in the PCB image to be detected and solder joint position data of the solder joint to be detected with the solder joint defect when the solder joint to be detected with the solder joint defect exists.
In some embodiments, the solder joint type and the solder joint position data of the solder joint to be detected and the PCB image to be detected can be input into a defect detection layer in the trained detection model to detect the solder joint defect, so as to obtain a solder joint defect detection result of the PCB image to be detected.
In some embodiments, the image of the PCB to be detected may be cut according to the solder joint position data of the solder joint to be detected, to obtain an image area corresponding to the solder joint to be detected, and the image area corresponding to the solder joint to be detected and the solder joint type of the solder joint to be detected are input to the defect detection layer in the trained detection model to perform the detection of the solder joint defect, to obtain a detection result of the solder joint to be detected, and to obtain a solder joint defect detection result of the image of the PCB to be detected based on the detection result of each solder joint to be detected.
According to the PCB welding spot defect detection method, the defect detection is carried out on the PCB image to be detected based on the trained detection model comprising the welding spot detection layer and the defect detection layer, so that the detection efficiency is improved; and determining the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected through the welding spot detection layer, carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out defect detection through the defect detection layer based on the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected, so that the accuracy of the follow-up defect detection result is improved through improving the accuracy of the welding spot type identification, and the misjudgment rate of welding spot defect detection is reduced.
In some embodiments, in order to improve accuracy of the PCB solder joint defect detection result, image preprocessing can be performed on the PCB image to be detected, so that image quality of the PCB image to be detected is improved, reliability of subsequent solder joint types and solder joint position data is further ensured, and accuracy of the final PCB solder joint defect detection result is improved. Wherein, the image preprocessing includes, but is not limited to, one or more of color conversion, clipping, image resizing, and image enhancement. Image enhancement includes, but is not limited to, contrast enhancement, sharpness enhancement, gray scale equalization, and the like.
Specifically, as shown in fig. 8, fig. 8 is a flow chart of a method for identifying a welding spot based on image preprocessing according to an embodiment of the present application, where the method for identifying a welding spot based on image preprocessing includes steps 221 to 222:
step 221, performing image preprocessing on the PCB image to be detected to obtain a preprocessed image.
Step 222, inputting the preprocessed image into a welding spot detection layer in the trained detection model to perform welding spot recognition, and determining the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected.
In some embodiments, the preprocessed image may be input to the solder joint detection layer shown in fig. 5, and solder joint identification may be performed on the preprocessed image based on the defect detection layer provided in fig. 5, to determine the solder joint type and solder joint position data of the solder joint to be detected of the PCB image to be detected.
In some embodiments, in order to improve the detection efficiency and improve the reliability of the subsequent detection result of the welding spot defect of the PCB, when the welding spot is identified on the preprocessed image, the target detection may be performed on the preprocessed image first, whether there is a welding spot to be detected in the preprocessed image is determined, when there is a welding spot to be detected in the preprocessed image, the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected are determined, and when there is no welding spot to be detected in the preprocessed image, the welding spot defect detection operation is not performed according to the welding spot type and the welding spot position data to be detected and the image area corresponding to the welding spot to be detected in the PCB image to be detected, and the previous steps 221 to 222 are performed on the next PCB image to be detected, and the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected are determined. Specifically, as shown in fig. 9, fig. 9 is a flow chart of a method for identifying a welding spot according to an embodiment of the present application, where the method for identifying a welding spot includes steps 2221 to 2223:
step 2221, inputting the preprocessed image into the solder joint detection layer in the trained detection model to perform target detection.
In some embodiments, the preprocessed image may be compared to the reference PCB image, and target detection may be performed based on the comparison. For example, if the preprocessed image is matched with the reference PCB image, it is determined that the preprocessed image has a solder joint to be detected, and if the preprocessed image is not matched with the reference PCB image, it is determined that the preprocessed image does not have a solder joint to be detected. The reference PCB image may be a reference PCB image having the same type as the PCB corresponding to the PCB image to be detected.
In some embodiments, a target detection unit may be disposed in the solder joint detection layer, and target detection is performed on the input preprocessed image by the target detection unit, and when it is determined that the preprocessed image has a solder joint to be detected, the solder joint type and solder joint position data of the solder joint to be detected of the PCB image to be detected are determined. The target detection unit may be a detector based on a convolutional neural network or a detector based on machine learning.
When the welding spots are identified, the target detection unit carries out target detection on the input preprocessed image, determines whether the preprocessed image has welding spots to be detected, obtains welding spot position data of the welding spots to be detected based on position information of the welding spots to be detected in the preprocessed image when the preprocessed image is determined to have the welding spots to be detected, and inputs the preprocessed image to the feature extraction unit, the feature fusion unit and the identification unit to obtain welding spot types of the welding spots to be detected.
Step 2222, when it is determined that the preprocessed image has a solder joint to be detected, obtains solder joint position data of the solder joint to be detected based on position information of the solder joint to be detected in the preprocessed image.
The position information may be a coordinate position of a center point of the detection frame of the welding spot to be detected in the preprocessed image, and may be a coordinate position of a vertex of the detection frame of the welding spot to be detected in the preprocessed image.
In some embodiments, the location information of the weld spot to be detected in the preprocessed image may be determined as weld spot location data of the weld spot to be detected.
Step 2223, performing feature extraction and feature recognition on the image area where the welding spot to be detected is located in the preprocessed image, and determining the welding spot type of the welding spot to be detected.
In some embodiments, the preprocessed image may be cut based on the solder joint position data of the solder joint to be detected, to obtain an image area where the solder joint to be detected is located in the preprocessed image, and the image area where the solder joint to be detected is located in the preprocessed image is input to the feature extraction unit, the feature fusion unit and the recognition unit of the solder joint detection layer to perform feature extraction and feature recognition, so as to determine the solder joint type of the solder joint to be detected.
In some embodiments, after determining the solder joint type and the solder joint position data of the solder joint to be detected, the solder joint defect detection may be performed according to the solder joint type and the solder joint position data of the solder joint to be detected and the operation of the PCB image to be detected, so as to obtain a solder joint defect detection result of the PCB image to be detected.
In some embodiments, in order to realize defect detection of the to-be-detected welding spots of different welding spot types, a trained detection model is provided with defect detection layers corresponding to different welding spot types, when the welding spot detection layers determine the welding spot types of the to-be-detected welding spots of the to-be-detected PCB image, a target defect detection layer corresponding to the to-be-detected welding spots is determined based on the welding spot types of the to-be-detected welding spots of the to-be-detected PCB image, welding spot position data of the to-be-detected welding spots and an image area corresponding to the to-be-detected welding spots in the to-be-detected PCB image are input to the corresponding target defect detection layer in the trained detection model to detect welding spot defects, so as to obtain detection results of the to-be-detected welding spots, and welding spot defect detection results of the to-be-detected PCB image are obtained based on the detection results of the to-be-detected welding spots.
Illustratively, the solder joint type including a circular solder joint and an elongated solder joint is taken as an example, and as shown in fig. 10, fig. 10 is a schematic structural diagram of another trained test model provided in an embodiment of the present application, where the trained test model includes an input layer, a solder joint test layer, a first defect test layer, a second defect test layer, and an output layer. The first defect detection layer and the second defect detection layer are connected in parallel and then connected in series with the output layer, and the welding spot detection layer is respectively connected in series with the first defect detection layer and the second defect detection layer. The first defect detection layer may be used for detecting defects of a welding spot to be detected, wherein the welding spot type of the first defect detection layer is a round welding spot, and the second defect detection layer may be used for detecting defects of a welding spot to be detected, wherein the welding spot type of the second defect detection layer is an elongated welding spot.
When the welding spot detection layer determines the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected during the defect detection of the welding spot of the PCB, the image area corresponding to the welding spot to be detected is input into the first defect detection layer and/or the second defect detection layer for defect detection based on the welding spot type of the welding spot to be detected, the detection result of the welding spot to be detected is obtained, and then the welding spot defect detection result of the PCB image to be detected is obtained. It can be understood that at least one solder joint to be detected exists in the PCB image to be detected, and the solder joint types corresponding to the solder joints to be detected can be the same or different.
It should be noted that the trained detection model shown in fig. 10 is merely an exemplary illustration, and in a practical application scenario, a corresponding number of defect detection layers may be set according to the number of solder joint types, which is not specifically limited in this embodiment of the present application.
Based on the trained detection model shown in fig. 10, the embodiment of the present application provides a method for detecting a welding spot defect based on a welding spot type, as shown in fig. 11, fig. 11 is a schematic flow diagram of the method for detecting a welding spot defect based on a welding spot type, where the method for detecting a welding spot defect based on a welding spot type includes steps 231 to 233:
And step 231, performing image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected, and obtaining an image area corresponding to the welding spots to be detected in the PCB image to be detected.
And 232, determining a target defect detection layer matched with the welding spot type in the trained detection model according to the welding spot type of the welding spot to be detected.
In some embodiments, a target defect detection layer matching the solder joint type in the trained detection model may be determined based on a mapping relationship of the solder joint type and the defect detection layer in the trained detection model; the trained detection model comprises at least one defect detection layer, and each defect detection layer corresponds to one welding spot type.
And 233, performing welding spot defect detection on the image area corresponding to the welding spot to be detected based on the target defect detection layer to obtain a welding spot defect detection result of the PCB image to be detected.
In some embodiments, an image area corresponding to a solder joint to be detected may be input to a corresponding target defect detection layer, and solder joint defect detection is performed based on the target defect detection layer, so as to obtain a solder joint defect detection result of the PCB image to be detected. Specifically, based on the schematic structural diagram of the defect detection layer provided in fig. 5, step 233 includes steps a1 to a3:
And a step a1, carrying out feature extraction on an image area corresponding to the welding spot to be detected based on a feature extractor to obtain the defect image feature.
And a2, carrying out feature calculation on the defect image features based on a probability predictor to obtain the prediction probability of the defects of the welding spots to be detected.
And a step a3, obtaining a welding spot defect detection result of the PCB image to be detected based on the classifier and the prediction probability.
In some embodiments, the classifier compares the predicted probability of the defect of the welding spot to be detected with a preset probability threshold, if the predicted probability of the defect of the welding spot to be detected is greater than or equal to the preset probability threshold, the detection result of the welding spot to be detected is determined to be a defective welding spot, and if the predicted probability of the defect of the welding spot to be detected is less than the preset probability threshold, the detection result of the welding spot to be detected is determined to be a normal welding spot; and counting the detection results of all the welding spots to be detected to obtain welding spot defect detection results of the PCB image to be detected.
If the detection result of each welding spot to be detected is a normal welding spot, determining that the welding spot defect detection result of the PCB image to be detected is that a defect welding spot does not exist; if the detection result in each welding spot to be detected is the welding spot to be detected of the defect welding spot, determining that the welding spot defect detection result of the PCB image to be detected is the defect welding spot.
In order to ensure the detection accuracy of the trained detection model and thus the accuracy of the detection result of the PCB welding spot defect, the initial detection model needs to be trained based on the sample image to obtain the trained detection model. Wherein the sample image may be a sample image under different weld spot types acquired in advance. Wherein the model structure of the initial detection model is similar to the model structure of the trained detection model.
Considering that a welding spot detection layer and a defect detection layer exist in the initial detection model, in order to accelerate the convergence rate of the model, the welding spot detection layer and the defect detection layer can be alternately trained based on a sample image to obtain a trained detection model. Specifically, as shown in fig. 12, fig. 12 is a flow chart of a training method of a detection model according to an embodiment of the present application, where the training method of the detection model includes steps 1210 to 1260:
in step 1210, the first sample image of each solder joint type is input to an initial solder joint detection layer in the initial detection model, and training is performed on the initial solder joint detection layer to obtain an intermediate solder joint detection layer.
The intermediate welding spot detection layer refers to an intermediate product obtained by training in an initial detection model, the initial detection model comprises an initial welding spot detection layer and at least one initial defect detection layer, each initial defect detection layer corresponds to one welding spot type, and the initial welding spot detection layer is used for determining the welding spot type and the welding spot position data.
In some embodiments, an initial defect detection layer in an initial detection model may be fixed, a first sample image of each welding spot type is input to the initial welding spot detection layer in the initial detection model, a predicted welding spot type corresponding to the first sample image is obtained, a type training loss is determined according to a real welding spot type corresponding to the first sample image and a preset welding spot type, and parameters of the initial welding spot detection layer in the initial detection model are adjusted according to the type training loss, so as to obtain an intermediate welding spot detection layer. Parameters of the initial pad detection layer include, but are not limited to, network weights.
In some embodiments, a first sample image of each welding spot type may be input to an initial welding spot detection layer in an initial detection model to obtain a predicted welding spot type corresponding to the first sample image, and each first sample may be input to an initial defect detection layer to obtain a test structure corresponding to the first sample image by the initial defect detection layer; and adjusting parameters of an initial welding spot detection layer in the initial detection model according to errors between the type of the test welding spot in the test structure and the type of the real welding spot in the real recognition structure of the first sample image to obtain an intermediate welding spot detection layer. The test result of the initial defect detection layer on the first sample image comprises a test welding spot defect identification result and a test welding spot type, wherein the test welding spot defect identification result represents whether a defect welding spot exists in the first sample image, and the test welding spot type represents the welding spot type corresponding to the welding spot to be detected in the first sample image.
In some embodiments, the type of welding spots, the position data of the welding spots and whether the welding spots are defect welding spots of the historical PCB image can be marked by collecting the historical PCB image in the past period of time, a sample data set is constructed based on the marked historical PCB image, and the sample data set is divided to obtain a first sample image and a second sample image. Wherein the past period of time includes, but is not limited to, the past week, the past month, the past march, the past six months.
Specifically, the sample image construction method includes:
(1) Acquiring an initial sample dataset; the initial sample dataset includes initial sample images for each solder joint type.
The initial sample image may be a historical PCB board image over a period of time. The method comprises historical PCB images with welding spot defects under different welding spot types and historical PCB images without welding spot defects.
(2) Labeling the welding spots to be detected in each initial sample image, and determining the type of the real welding spots and the real recognition result of the welding spots to be detected in each initial sample image.
(3) And determining the marked initial sample image as a sample image to obtain a sample data set.
(4) And dividing the sample images in the sample data set according to the number of the images in the sample data set to obtain a first sample image and a second sample image.
Step 1220, inputting the second sample image of each solder joint type to the middle solder joint detection layer, to obtain the test solder joint type and the test solder joint position data of the solder joint to be detected in each second sample image.
In some embodiments, the type of the test welding spot and the position data of the test welding spot of the welding spot to be detected in each second sample image may be obtained according to the above step 220, which is not described herein.
In some embodiments, the type of the test welding spot and the position data of the test welding spot of the welding spot to be detected in each second sample image may be obtained by referring to the welding spot identification method based on image preprocessing provided in fig. 7, which is not described herein in detail.
Step 1230, inputting the type of the test welding spot and the position data of the test welding spot of the to-be-detected welding spot in each second sample image and each second sample image to different initial defect detection layers in the initial detection model respectively, so as to obtain the test result of each initial defect detection layer on the to-be-detected welding spot of each test welding spot type.
In some embodiments, the test results of each initial defect detection layer on the to-be-detected solder joints of each test solder joint type may be obtained by referring to the above step 220, and the embodiments of the present application will not be described herein.
In some embodiments, the test result of each initial defect detection layer on the to-be-detected solder joint of each test solder joint type may be obtained by referring to the solder joint defect detection method based on solder joint type provided in fig. 11, and the embodiments of the present application will not be described herein.
Step 1240, obtaining the test solder joint type associated with each initial defect detection layer according to the test result of each initial defect detection layer on the solder joint to be detected of each test solder joint type, and obtaining the mapping relation between the solder joint type and the defect detection layer.
In some embodiments, obtaining the test solder joint type associated with each initial defect detection layer refers to finding the initial defect detection layer recommended by the solder joint type according to the input second sample image of each solder joint type, and establishing a mapping relation between the solder joint type and the defect detection layer according to each solder joint type and the initial defect detection layer recommended by each solder joint type. The initial defect detection layer recommended by the welding spot type refers to an initial defect detection layer with the minimum recognition error obtained by performing defect detection on a second sample image of the welding spot type.
In some embodiments, the method for establishing the mapping relationship between the solder joint type and the defect detection layer includes steps b1 to b4:
and b1, obtaining a real recognition result corresponding to the welding spots to be detected of each test welding spot type.
In some embodiments, according to the second sample image of each welding spot type and the sample label corresponding to the second sample image, a real recognition result corresponding to the welding spot to be detected of the test welding spot type can be obtained from the sample label of the second sample image. The real recognition result comprises whether the welding spot to be detected of the type of the test welding spot is a defective welding spot or not.
And b2, determining and obtaining the identification error of each initial defect detection layer for each test welding spot type according to the test result of each initial defect detection layer for each test welding spot type to-be-detected welding spot and the real identification result corresponding to each test welding spot type to-be-detected welding spot.
And b3, aiming at each initial defect detection layer, determining the test welding spot type corresponding to the minimum identification error from the identification errors of each test welding spot type of the initial defect detection layer, and setting the test welding spot type corresponding to the minimum identification error as the test welding spot type corresponding to the initial defect detection layer.
In some embodiments, an identification error between a test result of each initial defect detection layer on a to-be-detected welding spot of each test welding spot type and a real identification result corresponding to the to-be-detected welding spot of each test welding spot type can be calculated through a preset loss function, an identification error of each initial defect detection layer on the test welding spot type is determined for each test welding spot type, an initial defect detection layer where the minimum error in the identification error of each initial defect detection layer on the test welding spot type is located is determined to be an initial defect detection layer associated with the test welding spot type, and the test welding spot type corresponding to each initial defect detection layer is obtained according to each test welding spot type and the initial defect detection layer associated with the test welding spot type.
Wherein the initial defect detection layer associated with the test pad type refers to a defect detection layer that matches each pad type. The initial defect detection layer where the minimum error is located refers to an initial defect detection layer which obtains the minimum value in the identification error of the type of the test welding spot. The preset loss function may be a cross entropy based loss function.
And b4, associating the test welding spot type corresponding to each initial defect detection layer with each initial defect detection layer to obtain the test welding spot type associated with each initial defect detection layer.
Step 1250, according to the test result of the to-be-detected welding spots of the test welding spot type associated with each initial defect detection layer, the initial model parameters of each initial defect detection layer and the parameters of the intermediate welding spot detection layer are adjusted to obtain the welding spot detection layer and each defect detection layer.
In some embodiments, after the intermediate welding spot detection layer is obtained, the second sample image is input to the intermediate welding spot detection layer to obtain the test welding spot type and the test welding spot position data of the welding spot to be detected in each sample image, the test welding spot type and the test welding spot position data of the welding spot to be detected in each second sample image and each second sample image are respectively input to different initial defect detection layers in the initial detection model to obtain the identification training loss of each initial defect detection layer, and the initial parameters of the initial defect detection layer are adjusted based on the identification training loss of each initial defect detection layer to obtain the welding spot detection layer and each defect detection layer.
Specifically, the training method of the initial defect detection layer includes steps c1 to c3:
step c1, obtaining a real recognition result corresponding to the welding spots to be detected of each test welding spot type.
And c2, obtaining the recognition training loss of each initial defect detection layer according to the test result of the to-be-detected welding spots of the test welding spot type associated with each initial defect detection layer and the real recognition result corresponding to the to-be-detected welding spots of the associated test welding spot type.
In some embodiments, similar to the training method of the initial solder joint detection layer, the intermediate solder joint detection layer is fixed, and for the second sample image of each test solder joint type, the test result of each initial defect detection layer for the second sample image of each test solder joint type is obtained, and the recognition training loss between the test result of each initial defect detection layer for the second sample image of each test solder joint type and the real recognition result corresponding to the second sample image of each test solder joint type is calculated through a preset loss function.
And c3, adjusting initial model parameters of each initial defect detection layer according to the identification training loss of each initial defect detection layer to obtain each defect detection layer.
In some embodiments, according to the identification training loss of each initial defect detection layer, initial model parameters of each initial defect detection layer are adjusted to obtain an intermediate defect detection layer, wherein the intermediate defect detection layer is an intermediate product obtained by training the initial defect detection layer associated with each welding spot type in the initial detection model according to the second sample image, and the accuracy of the intermediate defect detection layer needs to be further verified.
In some embodiments, when verifying the intermediate defect detection layer and the intermediate welding spot detection layer, a plurality of verification images of welding spot types in a preset test set can be input into the intermediate welding spot detection layer to obtain welding spot types and welding spot position data of welding spots to be detected in each verification image, and the welding spot types and the welding spot position data of the welding spots to be detected in each verification image and each verification image are input into the intermediate defect detection layer associated with the welding spot types of the welding spots to be detected in each verification image to obtain verification recognition results of each verification image; if the verification recognition result of each verification image is consistent with the real recognition result of each verification image, determining the middle welding spot detection layer as a welding spot detection layer, and determining the middle defect detection layer as a defect detection layer; if the verification image with the verification recognition result inconsistent with the real recognition result of the verification image exists, determining the recognition verification loss between the verification recognition result of the verification image and the real recognition result of the verification image according to a preset loss function, and adjusting the middle model parameters of the middle defect detection layer and the parameters of the middle welding spot detection layer according to the recognition verification loss to obtain the welding spot detection layer and the defect detection layer.
Optionally, if there is a verification image in which the verification recognition result of the verification image is inconsistent with the real recognition result of the verification image, determining a recognition type verification loss between the verification welding spot type of each verification image and the real welding spot type of the verification image according to a preset type loss function, adjusting parameters of the middle welding spot detection layer based on the recognition type verification loss to obtain the welding spot detection layer, determining a recognition verification loss between the verification recognition result of the verification image and the real recognition result of the verification image according to the preset loss function, and adjusting middle model parameters of the middle defect detection layer according to the recognition verification loss to obtain the defect detection layer. The predetermined type of loss function may be a loss function based on a mean square error.
Step 1260, obtaining a trained detection model according to the welding spot detection layers, the defect detection layers and the mapping relation between the welding spot types and the defect detection layers.
In some embodiments, the mapping relation between the welding spot type and the defect detection layer is written into the welding spot detection layer, and the initial detection model is updated according to the parameters of the welding spot detection layer and the model parameters of each defect detection layer, which are written into the mapping relation between the welding spot type and the defect detection layer, so as to obtain the trained detection model.
In some embodiments, after training of the detection model is completed, a trained detection model is obtained, the trained detection model is deployed in a target site, a PCB image to be detected in the target site is obtained, welding spot identification is carried out on the PCB image to be detected based on a welding spot detection layer in the trained detection model, welding spot type and welding spot position data of welding spots to be detected of the PCB image to be detected are determined, welding spot defect detection is carried out on the PCB image to be detected through a defect detection layer in the trained detection model according to the welding spot type and welding spot position data of the welding spots to be detected, and a welding spot defect detection result of the PCB image to be detected is obtained.
Considering that the welding spots of the PCB are different in different electronic products, after the trained detection model is deployed on the target site, the type of the PCB is changed, the type of the electronic product is changed and the like by the hardware change of the target site, the type of the welding spots to be detected of the PCB image to be detected acquired in the target site is changed, and the welding spot types and defect welding spots which cannot be identified by the trained detection model possibly appear, and the trained detection model is required to be trained again at the moment so as to improve the detection accuracy of the trained detection model and further reduce the misjudgment of the defect detection result of the PCB welding spot.
In some embodiments, the deployed trained detection model may be retrained based on a training method of transfer learning; in some embodiments, the deployed trained detection model can also be retrained by a meta-learning based training method.
According to the PCB welding spot defect detection method, defect detection is carried out on the PCB image to be detected based on the trained detection model comprising the welding spot detection layer and the defect detection layer, two different detection models are not required to be additionally deployed, and detection efficiency is improved; and the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected are determined through the welding spot detection layer, and then the defect detection is carried out through the defect detection layer based on the welding spot type and the welding spot position data of the welding spots to be detected and the PCB image to be detected, so that the accuracy of the follow-up defect detection result is improved through adapting the welding spot defect detection of different welding spot types, and the misjudgment rate of the welding spot defect detection is reduced.
In order to better implement the method for detecting a defect of a PCB solder joint according to the embodiment of the present application, based on the embodiment of the method for detecting a defect of a PCB solder joint, the embodiment of the present application provides a device for detecting a defect of a PCB solder joint, specifically, as shown in fig. 13, fig. 13 is a schematic structural diagram of the device for detecting a defect of a PCB solder joint according to the embodiment of the present application, where the device for detecting a defect of a PCB solder joint includes:
The acquisition module is used for acquiring the PCB image to be detected;
the detection module is used for carrying out welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected; the solder joint types include circular solder joints and elongated solder joints.
In some embodiments, the detection module is configured to:
determining a target defect detection layer matched with the welding spot type in the trained detection model according to the welding spot type of the welding spot to be detected;
And carrying out welding spot defect detection on an image area corresponding to the welding spot to be detected based on the target defect detection layer to obtain a welding spot defect detection result of the PCB image to be detected.
In some embodiments, the detection module is configured to:
determining a target defect detection layer matched with the welding spot type in the trained detection model based on the mapping relation between the welding spot type and the defect detection layer in the trained detection model; the trained detection model comprises at least one defect detection layer, and each defect detection layer corresponds to one welding spot type.
In some implementations, the target defect detection layer includes a feature extractor, a probability predictor, and a classifier, the detection module to:
performing feature extraction on an image area corresponding to the welding spot to be detected based on a feature extractor to obtain defect image features;
performing feature calculation on the defect image features based on the probability predictor to obtain the prediction probability of defects of the welding spots to be detected;
and obtaining a welding spot defect detection result of the PCB image to be detected based on the classifier and the prediction probability.
In some embodiments, the detection module is configured to:
performing image preprocessing on the PCB image to be detected to obtain a preprocessed image;
And inputting the preprocessed image into a welding spot detection layer in the trained detection model to perform welding spot recognition, and determining the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected.
In some embodiments, the detection module is configured to:
inputting the preprocessed image into a welding spot detection layer in a trained detection model to perform target detection;
when the fact that the preprocessed image has the welding spots to be detected is determined, based on the position information of the welding spots to be detected in the preprocessed image, welding spot position data of the welding spots to be detected are obtained;
and carrying out feature extraction and feature recognition on an image area where the welding spot to be detected is located in the preprocessed image, and determining the welding spot type of the welding spot to be detected.
In some embodiments, the PCB solder joint defect detection apparatus further comprises:
training module for:
inputting a first sample image of each welding spot type to an initial welding spot detection layer in an initial detection model, and training the initial welding spot detection layer to obtain an intermediate welding spot detection layer;
inputting second sample images of each welding spot type into the middle welding spot detection layer to obtain the test welding spot type and the test welding spot position data of the welding spots to be detected in each second sample image;
Respectively inputting the type of the test welding spot and the position data of the test welding spot of the welding spot to be detected in each second sample image and each second sample image into different initial defect detection layers in the initial detection model to obtain the test result of each initial defect detection layer on the welding spot to be detected of each test welding spot type;
obtaining test welding spot types associated with each initial defect detection layer according to the test result of each initial defect detection layer on the welding spots to be detected of each test welding spot type, and obtaining the mapping relation between the welding spot types and the defect detection layers;
according to the test result of the to-be-detected welding spots of the test welding spot type associated with each initial defect detection layer, adjusting initial model parameters of each initial defect detection layer and parameters of the middle welding spot detection layer to obtain a welding spot detection layer and each defect detection layer;
and obtaining a trained detection model according to the welding spot detection layers, the defect detection layers and the mapping relation between the welding spot types and the defect detection layers.
In some embodiments, the training module is to:
obtaining a real identification result corresponding to a to-be-detected welding spot of each test welding spot type;
obtaining the recognition training loss of each initial defect detection layer according to the test result of the to-be-detected welding spots of the test welding spot type associated with each initial defect detection layer and the real recognition result corresponding to the to-be-detected welding spots of the associated test welding spot type;
And adjusting initial model parameters of each initial defect detection layer and parameters of the middle welding spot detection layer according to the identification training loss of each initial defect detection layer to obtain the welding spot detection layer and each defect detection layer.
In some embodiments, the training module is to:
obtaining a real identification result corresponding to a to-be-detected welding spot of each test welding spot type;
determining and obtaining the identification error of each initial defect detection layer for each test welding spot type according to the test result of each initial defect detection layer for each test welding spot type to-be-detected welding spot and the real identification result corresponding to each test welding spot type to-be-detected welding spot;
for each initial defect detection layer, determining a test welding spot type corresponding to the minimum identification error from the identification errors of each test welding spot type by the initial defect detection layer, and setting the test welding spot type corresponding to the minimum identification error as the test welding spot type corresponding to the initial defect detection layer;
and associating the test welding spot type corresponding to each initial defect detection layer with each initial defect detection layer to obtain the test welding spot type associated with each initial defect detection layer.
In some embodiments, the training module is to:
Acquiring a real welding spot type corresponding to a first sample image of each welding spot type;
inputting a first sample image of each welding spot type to an initial welding spot detection layer in an initial detection model to obtain a predicted welding spot type corresponding to each first sample image;
determining type training loss according to the predicted welding spot type and the real welding spot type corresponding to each first sample image;
and adjusting parameters of an initial welding spot detection layer in the initial detection model according to the type training loss to obtain an intermediate welding spot detection layer.
In some embodiments, the training module is to:
acquiring an initial sample dataset; the initial sample data set comprises initial sample images of each welding spot type;
labeling the welding spots to be detected in each initial sample image, and determining the type of the real welding spots and the real recognition result of the welding spots to be detected in each initial sample image;
determining the marked initial sample image as a sample image to obtain a sample data set;
and dividing the sample images in the sample data set according to the number of the images in the sample data set to obtain a first sample image and a second sample image.
According to the PCB welding spot defect detection device, defect detection is carried out on the PCB image to be detected based on the trained detection model comprising the welding spot detection layer and the defect detection layer, two different detection models are not required to be additionally deployed, and the detection efficiency is improved; and the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected are determined through the welding spot detection layer, and then the defect detection is carried out through the defect detection layer based on the welding spot type and the welding spot position data of the welding spots to be detected and the PCB image to be detected, so that the accuracy of the follow-up defect detection result is improved through adapting the welding spot defect detection of different welding spot types, and the misjudgment rate of the welding spot defect detection is reduced.
The embodiment of the invention also provides an electronic device, as shown in fig. 14, which shows a schematic structural diagram of the electronic device according to the embodiment of the invention, specifically:
the electronic device may include one or more processing cores 'processors 1401, one or more computer-readable storage media's memory 1402, a power supply 1403, and an input unit 1404, among other components. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 14 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:
the processor 1401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 1402, and calling data stored in the memory 1402, thereby performing overall monitoring of the electronic device. Optionally, processor 1401 may include one or more processing cores; preferably, the processor 1401 may integrate an application processor and a modem processor, wherein the application processor primarily handles operating systems, user interfaces, applications, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 1401.
The memory 1402 may be used to store software programs and modules, and the processor 1401 performs various functional applications and data processing by executing the software programs and modules stored in the memory 1402. The memory 1402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the electronic device, etc. Further, memory 1402 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 1402 may also include a memory controller to provide processor 1401 access to memory 1402.
The electronic device further comprises a power supply 1403 for powering the various components, preferably the power supply 1403 may be logically connected to the processor 1401 by a power management system, such that functions of managing charging, discharging, and power consumption are performed by the power management system. Power supply 1403 may also include one or more of any components, such as a dc or ac power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The electronic device may also include an input unit 1404, which input unit 1404 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 1401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 1402 according to the following instructions, and the processor 1401 executes the application programs stored in the memory 1402, so as to implement various functions, as follows:
acquiring a PCB image to be detected;
performing welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected; the solder joint types include circular solder joints and elongated solder joints.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present invention provides a storage medium having stored therein a plurality of instructions that can be loaded by a processor to perform the steps of any of the PCB solder joint defect detection methods provided by the embodiments of the present invention. For example, the instructions may perform the steps of:
acquiring a PCB image to be detected;
performing welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, and carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected; the solder joint types include circular solder joints and elongated solder joints.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The instructions stored in the storage medium can execute the steps in any one of the methods for detecting the defects of the solder joints of the PCB provided by the embodiments of the present invention, so that the beneficial effects that any one of the methods for detecting the defects of the solder joints of the PCB provided by the embodiments of the present invention can be achieved can be realized, which are detailed in the previous embodiments and are not described herein.
The foregoing describes in detail a method, an apparatus, an electronic device, and a storage medium for detecting defects of a PCB solder joint according to embodiments of the present invention, and specific examples are applied to illustrate principles and implementations of the present invention, where the foregoing description of the embodiments is only for helping to understand the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.

Claims (13)

1. A method for detecting defects of solder joints of a PCB, the method comprising:
acquiring a PCB image to be detected;
performing welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for performing image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, performing welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected, and comprises the following steps: determining a target defect detection layer matched with the welding spot type in the trained detection model according to the welding spot type of the welding spot to be detected; performing welding spot defect detection on the image area corresponding to the welding spot to be detected based on the target defect detection layer to obtain a welding spot defect detection result of the PCB image to be detected; the welding spot types include circular welding spots and long welding spots.
2. The method for detecting a solder joint defect of a PCB according to claim 1, wherein the determining a target defect detection layer matching the solder joint type in the trained detection model according to the solder joint type of the solder joint to be detected comprises:
determining a target defect detection layer matched with the welding spot type in the trained detection model based on the mapping relation between the welding spot type and the defect detection layer in the trained detection model; the trained detection model comprises at least one defect detection layer, and each defect detection layer corresponds to one welding spot type.
3. The PCB solder joint defect detection method of claim 1, wherein the target defect detection layer comprises a feature extractor, a probability predictor, and a classifier;
and performing solder joint defect detection on the image area corresponding to the solder joint to be detected based on the target defect detection layer to obtain a solder joint defect detection result of the PCB image to be detected, wherein the method comprises the following steps:
performing feature extraction on the image area corresponding to the welding spot to be detected based on the feature extractor to obtain a defect image feature;
performing feature calculation on the defect image features based on the probability predictor to obtain the prediction probability of defects of the welding spots to be detected;
And obtaining a welding spot defect detection result of the PCB image to be detected based on the classifier and the prediction probability.
4. The method for detecting a solder joint defect of a PCB according to claim 1, wherein the step of performing solder joint recognition on the PCB image to be detected based on the solder joint detection layer in the trained detection model, and determining solder joint type and solder joint position data of the solder joint to be detected of the PCB image to be detected, comprises:
performing image preprocessing on the PCB image to be detected to obtain a preprocessed image;
and inputting the preprocessed image into a welding spot detection layer in the trained detection model to perform welding spot recognition, and determining the welding spot type and the welding spot position data of the welding spots to be detected of the PCB image to be detected.
5. The method for inspecting solder joint defects of a PCB according to claim 4, wherein the inputting the preprocessed image to the solder joint inspection layer in the trained inspection model for solder joint recognition, determining the solder joint type and solder joint position data of the solder joint to be inspected of the PCB image to be inspected, comprises:
inputting the preprocessed image into a welding spot detection layer in a trained detection model to perform target detection;
When the fact that the welding spots to be detected exist in the preprocessed image is determined, based on the position information of the welding spots to be detected in the preprocessed image, welding spot position data of the welding spots to be detected are obtained;
and carrying out feature extraction and feature recognition on an image area where the welding spot to be detected is located in the preprocessed image, and determining the welding spot type of the welding spot to be detected.
6. The method for detecting a solder joint defect of a PCB according to any one of claims 1 to 5, wherein before performing solder joint recognition on the PCB image to be detected based on a solder joint detection layer in a trained detection model and determining a solder joint type and solder joint position data of a solder joint to be detected of the PCB image to be detected, the method comprises:
inputting a first sample image of each welding spot type to an initial welding spot detection layer in an initial detection model, and training the initial welding spot detection layer to obtain an intermediate welding spot detection layer;
inputting a second sample image of each welding spot type to the middle welding spot detection layer to obtain a test welding spot type and test welding spot position data of a welding spot to be detected in each second sample image;
respectively inputting the type of the test welding spot and the position data of the test welding spot of the welding spot to be detected in each second sample image and each second sample image into different initial defect detection layers in the initial detection model to obtain a test result of each initial defect detection layer on the welding spot to be detected of each test welding spot type;
Obtaining test welding spot types associated with the initial defect detection layers according to test results of the initial defect detection layers on welding spots to be detected of each test welding spot type, and obtaining a mapping relation between the welding spot types and the defect detection layers;
according to the test result of the to-be-detected welding spots of the test welding spot type associated with each initial defect detection layer, initial model parameters of each initial defect detection layer and parameters of the middle welding spot detection layer are adjusted to obtain a welding spot detection layer and each defect detection layer;
and obtaining a trained detection model according to the welding spot detection layer, each defect detection layer and the mapping relation between the welding spot type and the defect detection layer.
7. The method for inspecting defects of a PCB solder joint according to claim 6, wherein the adjusting initial model parameters of each initial defect inspection layer and parameters of the intermediate solder joint inspection layer according to the test result of the solder joint to be inspected of the type of test solder joint associated with each initial defect inspection layer to obtain a solder joint inspection layer and each defect inspection layer comprises:
obtaining a real identification result corresponding to the welding spots to be detected of each test welding spot type;
Obtaining the recognition training loss of each initial defect detection layer according to the test result of the to-be-detected welding spot of the test welding spot type associated with each initial defect detection layer and the real recognition result corresponding to the to-be-detected welding spot of the associated test welding spot type;
and adjusting initial model parameters of each initial defect detection layer and parameters of the middle welding spot detection layer according to the identification training loss of each initial defect detection layer to obtain the welding spot detection layer and each defect detection layer.
8. The method of claim 6, wherein the obtaining the test pad type associated with each initial defect detection layer according to the test result of each initial defect detection layer on the to-be-detected pad of each test pad type comprises:
obtaining a real identification result corresponding to the welding spots to be detected of each test welding spot type;
determining and obtaining an identification error of each initial defect detection layer for each test welding spot type according to a test result of each initial defect detection layer for each welding spot to be detected of the test welding spot type and a real identification result corresponding to each welding spot to be detected of the test welding spot type;
For each initial defect detection layer, determining a test welding spot type corresponding to the minimum identification error from the identification errors of the initial defect detection layer to each test welding spot type, and setting the test welding spot type corresponding to the minimum identification error as the test welding spot type corresponding to the initial defect detection layer;
and associating the test welding spot type corresponding to each initial defect detection layer with each initial defect detection layer to obtain the test welding spot type associated with each initial defect detection layer.
9. The method for inspecting defects of solder joints of a PCB according to claim 6, wherein inputting the first sample image of each solder joint type to an initial solder joint inspection layer in an initial inspection model, training the initial solder joint inspection layer to obtain an intermediate solder joint inspection layer, comprises:
acquiring a real welding spot type corresponding to a first sample image of each welding spot type;
inputting a first sample image of each welding spot type to an initial welding spot detection layer in an initial detection model to obtain a predicted welding spot type corresponding to each first sample image;
determining type training loss according to the predicted welding spot type and the real welding spot type corresponding to each first sample image;
And adjusting parameters of an initial welding spot detection layer in the initial detection model according to the type training loss to obtain an intermediate welding spot detection layer.
10. The method for inspecting solder joint defects of a PCB according to claim 6, wherein the inputting the first sample image of each solder joint type to an initial solder joint inspection layer in an initial inspection model, training the initial solder joint inspection layer, and before obtaining an intermediate solder joint inspection layer, comprises:
acquiring an initial sample dataset; the initial sample data set comprises initial sample images of all welding spot types;
labeling the welding spots to be detected in each initial sample image, and determining the type of the real welding spots and the real recognition result of the welding spots to be detected in each initial sample image;
determining the marked initial sample image as a sample image to obtain a sample data set;
dividing the sample images in the sample data set according to the number of the images in the sample data set to obtain a first sample image and a second sample image.
11. A PCB solder joint defect detection apparatus, the apparatus comprising:
the acquisition module is used for acquiring the PCB image to be detected;
The detection module is used for carrying out welding spot defect detection on the PCB image to be detected based on the trained detection model to obtain a welding spot defect detection result of the PCB image to be detected;
the trained detection model comprises a welding spot detection layer and a defect detection layer, and is obtained by training an initial welding spot detection layer and an initial defect detection layer in an initial detection model simultaneously through a sample image; the welding spot detection layer is used for carrying out welding spot identification on the PCB image to be detected and determining the welding spot type and the welding spot position data of the welding spot to be detected of the PCB image to be detected; the defect detection layer is used for carrying out image processing on the PCB image to be detected according to the welding spot position data of the welding spots to be detected to obtain an image area corresponding to the welding spots to be detected in the PCB image to be detected, carrying out welding spot defect detection according to the welding spot type of the welding spots to be detected and the image area corresponding to the welding spots to be detected in the PCB image to be detected to obtain a welding spot defect detection result of the PCB image to be detected, and the detection module is further used for: determining a target defect detection layer matched with the welding spot type in the trained detection model according to the welding spot type of the welding spot to be detected; performing welding spot defect detection on the image area corresponding to the welding spot to be detected based on the target defect detection layer to obtain a welding spot defect detection result of the PCB image to be detected; the welding spot types include circular welding spots and long welding spots.
12. An electronic device comprising a memory and a processor; the memory stores an application program, and the processor is configured to run the application program in the memory to perform the operations in the PCB pad defect detection method of any one of claims 1 to 10.
13. A storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the PCB pad defect detection method of any one of claims 1 to 10.
CN202311027218.1A 2023-08-14 2023-08-14 PCB welding spot defect detection method and device, electronic equipment and storage medium Active CN117078620B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311027218.1A CN117078620B (en) 2023-08-14 2023-08-14 PCB welding spot defect detection method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311027218.1A CN117078620B (en) 2023-08-14 2023-08-14 PCB welding spot defect detection method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN117078620A CN117078620A (en) 2023-11-17
CN117078620B true CN117078620B (en) 2024-02-23

Family

ID=88705481

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311027218.1A Active CN117078620B (en) 2023-08-14 2023-08-14 PCB welding spot defect detection method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117078620B (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101210890A (en) * 2006-12-28 2008-07-02 夏普株式会社 Defect detecting device and method, image sensor device and module
CN105956613A (en) * 2016-04-25 2016-09-21 西安石油大学 Industrial x ray weld image circular, linear defect classification method
CN106651857A (en) * 2017-01-04 2017-05-10 中国农业大学 Printed circuit board patch defect detection method
CN109242853A (en) * 2018-11-05 2019-01-18 南京信息工程大学 A kind of PCB defect intelligent detecting method based on image procossing
CN111696108A (en) * 2020-08-17 2020-09-22 广东利元亨智能装备股份有限公司 Model training method, welding spot defect detection method and device and electronic equipment
US11216932B1 (en) * 2021-03-26 2022-01-04 Minds AI Technologies Ltd Electronic substrate defect detection
CN114596243A (en) * 2020-12-03 2022-06-07 中国联合网络通信集团有限公司 Defect detection method, device, equipment and computer readable storage medium
CN114723706A (en) * 2022-04-02 2022-07-08 杭州电子科技大学 Welding spot detection and positioning method based on machine vision
CN115937170A (en) * 2022-12-23 2023-04-07 正泰集团研发中心(上海)有限公司 Circuit board detection method and device, computer equipment and storage medium
CN116012285A (en) * 2022-10-24 2023-04-25 天津津航计算技术研究所 BGA welding spot defect detection system and detection method based on laser ultrasound
CN116342486A (en) * 2022-11-30 2023-06-27 浙江新涛电子科技有限公司 Solder joint defect detection method, computer device, readable storage medium and program product

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109934814B (en) * 2019-03-15 2022-02-01 英业达科技有限公司 Surface defect detection system and method thereof
US11475556B2 (en) * 2019-05-30 2022-10-18 Bruker Nano, Inc. Method and apparatus for rapidly classifying defects in subcomponents of manufactured component

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101210890A (en) * 2006-12-28 2008-07-02 夏普株式会社 Defect detecting device and method, image sensor device and module
CN105956613A (en) * 2016-04-25 2016-09-21 西安石油大学 Industrial x ray weld image circular, linear defect classification method
CN106651857A (en) * 2017-01-04 2017-05-10 中国农业大学 Printed circuit board patch defect detection method
CN109242853A (en) * 2018-11-05 2019-01-18 南京信息工程大学 A kind of PCB defect intelligent detecting method based on image procossing
CN111696108A (en) * 2020-08-17 2020-09-22 广东利元亨智能装备股份有限公司 Model training method, welding spot defect detection method and device and electronic equipment
CN114596243A (en) * 2020-12-03 2022-06-07 中国联合网络通信集团有限公司 Defect detection method, device, equipment and computer readable storage medium
US11216932B1 (en) * 2021-03-26 2022-01-04 Minds AI Technologies Ltd Electronic substrate defect detection
CN114723706A (en) * 2022-04-02 2022-07-08 杭州电子科技大学 Welding spot detection and positioning method based on machine vision
CN116012285A (en) * 2022-10-24 2023-04-25 天津津航计算技术研究所 BGA welding spot defect detection system and detection method based on laser ultrasound
CN116342486A (en) * 2022-11-30 2023-06-27 浙江新涛电子科技有限公司 Solder joint defect detection method, computer device, readable storage medium and program product
CN115937170A (en) * 2022-12-23 2023-04-07 正泰集团研发中心(上海)有限公司 Circuit board detection method and device, computer equipment and storage medium

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
LFF-YOLO: A YOLO Algorithm With Lightweight Feature Fusion Network for Multi-Scale Defect Detection;Xiaohong Qian等;《 IEEE Access 》;20221206;第10卷;130339-130349 *
Printed Circuit Board Defect Detection Methods Based on Image Processing, Machine Learning and Deep Learning: A Survey;Qin Ling等;《IEEE Access 》;20230210;第11卷;15921-15944 *
Surface Defect Detection Algorithm Based on Feature-Enhanced YOLO;Yongfang Xie等;《Cognitive Computation 》;20221017;第15卷;565–579 *
基于图像技术的PCB焊点缺陷特征提取与准确识别研究;孙家富;《中国优秀硕士学位论文全文数据库 (信息科技辑)》;20200615(第6期);I135-216 *
基于多特征的SVM多分类PCB焊点缺陷检测方法;陈寿宏等;《激光杂志》;20190625;第40卷(第06期);21-26 *
基于深度学习的PCB缺陷检测方法;廖鑫婷等;《电子产品可靠性与环境试验》;20220820;第40卷(第04期);30-34 *

Also Published As

Publication number Publication date
CN117078620A (en) 2023-11-17

Similar Documents

Publication Publication Date Title
CN110659660B (en) Automatic optical detection classification equipment using deep learning system and training equipment thereof
CN110060237B (en) Fault detection method, device, equipment and system
EP1808018B1 (en) Image sensor annotation method and apparatus
CN109727229B (en) Method and device for detecting false solder
CN110441316B (en) Battery surface defect detection method and detection system
CN110599484B (en) Defect detection method, device and storage medium
CN109060817B (en) Artificial intelligence reinspection system and method thereof
KR20200004822A (en) Display screen peripheral circuit detection method, device, electronic device and storage medium
WO2019155593A1 (en) System for creating learned model for component image recognition, and method for creating learned model for component image recognition
CN111814850A (en) Defect detection model training method, defect detection method and related device
CN111401418A (en) Employee dressing specification detection method based on improved Faster r-cnn
CN113744244B (en) Online visual detection system for measuring distance from edge of lithium battery pole piece to edge of pole lug
KR20200004823A (en) Display screen peripheral circuit detection method, device, electronic device and storage medium
JP2020060879A (en) Learning device, image generator, method for learning, and learning program
CN111310826A (en) Method and device for detecting labeling abnormity of sample set and electronic equipment
US20200090319A1 (en) Machine learning method implemented in aoi device
JP2021174456A (en) Abnormality determination method and abnormality determination device
CN113111903A (en) Intelligent production line monitoring system and monitoring method
KR102174424B1 (en) Method for Inspecting Component basesd Server and system and apparatus therefor
CN109657682B (en) Electric energy representation number identification method based on deep neural network and multi-threshold soft segmentation
US20220284699A1 (en) System and method of object detection using ai deep learning models
CN117078620B (en) PCB welding spot defect detection method and device, electronic equipment and storage medium
CN113077430A (en) Laser chip defect detection and classification method and system based on SSD algorithm
CN116703914A (en) Welding defect detection method, equipment and medium based on generation type artificial intelligence
CN113469991B (en) Visual online detection method for laser welding spot of lithium battery tab

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant