CN116563291B - SMT intelligent error-proofing feeding detector - Google Patents

SMT intelligent error-proofing feeding detector Download PDF

Info

Publication number
CN116563291B
CN116563291B CN202310841051.6A CN202310841051A CN116563291B CN 116563291 B CN116563291 B CN 116563291B CN 202310841051 A CN202310841051 A CN 202310841051A CN 116563291 B CN116563291 B CN 116563291B
Authority
CN
China
Prior art keywords
image
feature vector
semantic feature
detection
detected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310841051.6A
Other languages
Chinese (zh)
Other versions
CN116563291A (en
Inventor
潘磊
杨立志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Bluiris Technology Co ltd
Original Assignee
Shenzhen Bluiris Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Bluiris Technology Co ltd filed Critical Shenzhen Bluiris Technology Co ltd
Priority to CN202310841051.6A priority Critical patent/CN116563291B/en
Publication of CN116563291A publication Critical patent/CN116563291A/en
Application granted granted Critical
Publication of CN116563291B publication Critical patent/CN116563291B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30141—Printed circuit board [PCB]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

Discloses an SMT intelligent error-proofing feeding detector. Firstly, preprocessing a detection image of a detected PCB to obtain a preprocessed detection image, then, respectively carrying out image blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks, then, respectively passing the sequence of detection image blocks and the sequence of reference image blocks through a double detection model to obtain a detection semantic feature vector and a reference semantic feature vector, then, calculating a differential feature vector between the detection semantic feature vector and the reference semantic feature vector, and finally, passing the differential feature vector through a classifier to obtain a classification result for indicating whether components on the detected PCB are correctly placed. Thus, the efficiency and quality of the production line can be improved, and the risk of human errors can be reduced.

Description

SMT intelligent error-proofing feeding detector
Technical Field
The application relates to the field of intelligent detection, in particular to an SMT intelligent error-proof feeding detector.
Background
In the production process of SMT circuit boards, the correct placement and quality of components is critical to ensuring product quality and stability. If the components are placed incorrectly or have defects, the circuit board may not work properly or even fail, thereby affecting the quality and performance of the product and bringing significant economic loss to the enterprise. In addition, since the SMT production speed is high, efficient and accurate inspection and verification of each component is required in order to ensure the quality of the product. However, the conventional manual inspection method faces problems of low efficiency, error-prone, etc., and may be difficult to find and correct in time when an error occurs.
Therefore, an optimized SMT intelligent error-proofing feeding detector is desired that can automatically detect and correct component placement errors and defects, improve production line efficiency and quality, and reduce the risk of human error.
Disclosure of Invention
The present application has been made to solve the above-mentioned technical problems. The embodiment of the application provides an SMT intelligent error-proofing feeding detector. Firstly, preprocessing a detection image of a detected PCB to obtain a preprocessed detection image, then, respectively carrying out image blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks, then, respectively passing the sequence of detection image blocks and the sequence of reference image blocks through a double detection model to obtain a detection semantic feature vector and a reference semantic feature vector, then, calculating a differential feature vector between the detection semantic feature vector and the reference semantic feature vector, and finally, passing the differential feature vector through a classifier to obtain a classification result for indicating whether components on the detected PCB are correctly placed. Thus, the efficiency and quality of the production line can be improved, and the risk of human errors can be reduced.
According to one aspect of the application, there is provided an SMT intelligent error-proofing feeding detector, comprising:
the detection image acquisition module is used for acquiring a detection image of the PCB to be detected;
the reference image acquisition module is used for acquiring a reference image, wherein the reference image is an image of a PCB with standard component placement;
the image preprocessing module is used for preprocessing the detection image of the detected PCB to obtain a preprocessed detection image, wherein the preprocessing comprises image denoising, contrast adjustment and brightness adjustment;
the image blocking processing module is used for respectively carrying out image blocking processing on the preprocessed detection image and the reference image so as to obtain a sequence of detection image blocks and a sequence of reference image blocks;
the image semantic feature extraction module is used for respectively passing the sequence of the detection image blocks and the sequence of the reference image blocks through a double detection model comprising a first image encoder and a second image encoder to obtain detection semantic feature vectors and reference semantic feature vectors;
the feature difference module is used for calculating a difference feature vector between the detected semantic feature vector and the reference semantic feature vector; and the placement detection module is used for enabling the differential feature vector to pass through the classifier to obtain a classification result, wherein the classification result is used for indicating whether components on the PCB to be detected are placed correctly or not.
In the above-mentioned SMT intelligence mistake proofing material loading detector, the image segmentation processing module is used for:
and respectively carrying out image uniform blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks.
In the above SMT intelligent error-proofing feeding detector, the first image encoder and the second image encoder have the same network structure.
In the above-mentioned SMT intelligence mistake proofing material loading detector, the image semantic feature draws the module, includes:
a first image encoding unit configured to: performing two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transfer of layers by using each layer of the first image encoder to output the detection semantic feature vector by the last layer of the first image encoder, wherein the input of the first layer of the first image encoder is a sequence of the detection image blocks; and
a second image encoding unit configured to: and respectively carrying out two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transmission of layers by using each layer of the second image encoder to output the reference semantic feature vector by the last layer of the second image encoder, wherein the input of the first layer of the second image encoder is a sequence of the reference image blocks.
In the above-mentioned SMT intelligence mistake proofing material loading detector, the characteristic difference module includes:
the optimization factor calculation unit is used for calculating the Helmholtz class free energy factors of the detection semantic feature vector and the reference semantic feature vector to obtain a first Helmholtz class free energy factor and a second Helmholtz class free energy factor;
the weighted optimization unit is used for weighting the detected semantic feature vector and the reference semantic feature vector by taking the first Helmholtz class free energy factor and the second Helmholtz class free energy factor as weighted weights so as to obtain an optimized detected semantic feature vector and an optimized reference semantic feature vector; and
and the difference feature optimization unit is used for calculating the difference feature vector between the optimized detection semantic feature vector and the optimized reference semantic feature vector.
In the above SMT intelligent error-proofing feeding detector, the optimization factor calculating unit is configured to:
calculating the Helmholtz type free energy factors of the detected semantic feature vector and the reference semantic feature vector according to the following optimization formula to obtain the first Helmholtz type free energy factor and the second Helmholtz type free energy factor;
Wherein, the optimization formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,characteristic values representing respective positions in said detected semantic characteristic vector,/or->Representing the reference semantic feature vectorCharacteristic values of the respective positions in>Classification probability values representing said detected semantic feature vectors, ->Classification probability values representing the reference semantic feature vectors, and +.>Is the length of the feature vector, +.>Represents a logarithmic function with base 2, +.>Representing an exponential operation, ++>And->Representing the first helmholtz-like free energy factor and the second helmholtz-like free energy factor, respectively.
In the above-mentioned SMT intelligence mistake proofing material loading detector, the difference characteristic optimizing unit is used for:
calculating the differential feature vector between the optimized detection semantic feature vector and the optimized reference semantic feature vector according to the following differential calculation formula;
wherein, the difference calculation formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the optimized post-detection semantic feature vector, < >>Representing the optimized reference semantic feature vector，/>Representing the differential eigenvector,>representing the difference by location.
In the above-mentioned SMT intelligence mistake proofing material loading detector, place detection module, include:
The full-connection coding unit is used for carrying out full-connection coding on the differential feature vectors by using a full-connection layer of the classifier so as to obtain coded classification feature vectors; and the classification unit is used for inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
Compared with the prior art, the SMT intelligent error-proofing feeding detector provided by the application is characterized in that firstly, a detected image of a detected PCB is preprocessed to obtain a preprocessed detected image, then, the preprocessed detected image and the reference image are subjected to image blocking processing to obtain a sequence of detected image blocks and a sequence of reference image blocks, then, the sequence of detected image blocks and the sequence of reference image blocks are respectively subjected to a double detection model to obtain a detected semantic feature vector and a reference semantic feature vector, then, a differential feature vector between the detected semantic feature vector and the reference semantic feature vector is calculated, and finally, the differential feature vector is subjected to a classifier to obtain a classification result for indicating whether components on the detected PCB are correctly placed. Thus, the efficiency and quality of the production line can be improved, and the risk of human errors can be reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art. The following drawings are not intended to be drawn to scale, emphasis instead being placed upon illustrating the principles of the application.
Fig. 1 is an application scenario diagram of an SMT intelligent error-proofing feeding detector according to an embodiment of the application.
Fig. 2 is a schematic block diagram of an SMT intelligent error-proofing feeding detector according to an embodiment of the application.
Fig. 3 is a schematic block diagram of the image semantic feature extraction module in the SMT intelligent error-proofing feeding detector according to an embodiment of the present application.
Fig. 4 is a schematic block diagram of the characteristic difference module in the SMT intelligent error-proofing feeding detector according to an embodiment of the application.
Fig. 5 is a schematic block diagram of the placement detection module in the SMT intelligent error-proofing feeding detector according to an embodiment of the present application.
Fig. 6 is a flowchart of an SMT intelligent error-proofing feeding detection method according to an embodiment of the application.
Fig. 7 is a schematic diagram of a system architecture of an SMT intelligent error-proofing feeding detection method according to an embodiment of the present application.
Description of the embodiments
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are also within the scope of the application.
As used in the specification and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Although the present application makes various references to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative, and different aspects of the systems and methods may use different modules.
A flowchart is used in the present application to describe the operations performed by a system according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
As described above, in the SMT circuit board production process, if the component is placed incorrectly or defective, the circuit board may not work properly or even fail, thereby affecting the quality and performance of the product and bringing significant economic loss to the enterprise. In addition, since the SMT production speed is high, efficient and accurate inspection and verification of each component is required in order to ensure the quality of the product. However, the conventional manual inspection method faces problems of low efficiency, error-prone, etc., and may be difficult to find and correct in time when an error occurs. Therefore, an optimized SMT intelligent error-proofing feeding detector is desired that can automatically detect and correct component placement errors and defects, improve production line efficiency and quality, and reduce the risk of human error.
Specifically, in the technical scheme of the application, the visual recognition technology is expected to be used for detecting the PCB board of the SMT, so that whether the components on the circuit board are correctly placed or not can be checked, and the information including the position, the direction, the polarity and the like of the components is included. Specifically, a high-resolution detection image is shot through a vision system, and an algorithm is used for automatically identifying elements in the detection image so as to quickly and accurately determine the accurate positions and directions of the elements, thereby avoiding the problem that a circuit board cannot work normally due to misplacement or tilting of the elements. However, since the amount of information existing in the detected image is large, the feature information such as the position, direction, polarity, etc. of the components is small-scale fine feature information in the image, and it is difficult to capture and extract the feature information. In addition, in the process of image acquisition, a large amount of noise in the image can be caused, and the feature extraction effect of the image is affected. Based on the above, in the technical scheme of the application, feature comparison is expected to be carried out in a high-dimensional feature space by utilizing the detection image of the PCB and the reference image which is normally placed by the standard components, so that the classification detection on whether the components of the PCB are correctly placed is realized. In the process, the difficulty is how to mine the difference characteristic information about the component hidden characteristic distribution of the PCB in the detection image and the reference image so as to detect whether the component on the PCB is correctly placed or not, correct component placement errors and defects, improve the production line efficiency and quality, and reduce the risk of human errors.
In recent years, deep learning and neural networks have been widely used in the fields of computer vision, natural language processing, text signal processing, and the like. The development of deep learning and neural networks provides a new solution idea and scheme for mining the differential characteristic information about the component hidden characteristic distribution of the PCB in the detection image and the reference image.
Specifically, in the technical scheme of the application, firstly, a detection image of a detected PCB and a reference image are obtained, wherein the reference image is an image of the PCB with normal component placement. In particular, in one specific example of the present application, the inspection image is a live image of the inspected PCB board acquired by a camera, and the reference image is a design image provided based on computer-aided design software.
Then, considering that the acquired detection image may have a certain noise and illumination when the component on the PCB board is placed and detected, the image needs to be subjected to denoising, contrast adjustment, brightness adjustment and other treatments before the feature extraction is performed, so that the subsequent image blocking and feature extraction can be performed more accurately, and the abnormal component placement detection accuracy on the PCB board is improved. Specifically, in the technical scheme of the application, the detected image of the detected PCB is preprocessed to obtain the preprocessed detected image, wherein the preprocessing comprises image denoising, contrast adjustment and brightness adjustment. It should be understood that denoising can reduce interference information in an image, and adjusting contrast and brightness can enable details of components in the image to be clearer and more definite, so that accurate identification and classification of a subsequent model are facilitated.
Then, it is considered that since there may be a plurality of components in one large PCB image, and different components may have differences in different positions, directions, sizes, etc., processing the entire image may result in the occurrence of inaccurate detection or missed detection. Therefore, when the components are placed, the detected PCB image needs to be divided into a plurality of small blocks for processing, that is, the preprocessed detection image and the reference image are respectively subjected to image blocking processing, so that the image information of each small block in the image can be obtained, and therefore, each component can be detected and compared more accurately later, and the sequence of the detection image blocks and the sequence of the reference image blocks can be obtained.
Further, feature mining of the sequence of detected image blocks and the sequence of reference image blocks is performed using a convolutional neural network model having excellent performance in implicit feature extraction of images. In particular, considering that when detecting whether the components on the detected PCB board are correctly placed, in order to further improve the expression accuracy of the implicit feature placed on the PCB board in each image block of the detected image and each image block of the reference image, in the technical solution of the present application, the sequence of the detected image blocks and the sequence of the reference image blocks are respectively passed through a dual detection model including a first image encoder and a second image encoder to obtain a detected semantic feature vector and a reference semantic feature vector.
In particular, in the technical solution of the present application, the first image encoder and the second image encoder have the same network structure, and the first image encoder and the second image encoder use a first convolutional neural network and a second convolutional neural network to perform feature extraction of the sequence of the detection image blocks and the sequence of the reference image blocks, respectively, so as to extract implicit feature distribution information about component placement of a PCB board in each image block of the detection image and the reference image, respectively. It should be understood that the feature extraction of the sequence of the detected image blocks and the sequence of the reference image blocks by using the dual detection model including the image encoders with the same network structure can extract feature information of the image blocks with insignificant difference at the image source domain end, so as to improve the accuracy of detection on whether the components on the detected PCB board are correctly placed. In addition, the traditional method can only extract some shallow image features, and cannot accurately express complex information such as the shape, the size, the position, the direction and the like of the components. Through advanced technologies such as deep learning, images can be abstracted and captured on multiple levels, and richer and more accurate semantic features are extracted.
Then, considering that certain differences may exist in the shape, the size, the position, the direction and the like of different components, the differences can be better described by calculating the differential feature vector, and therefore the placement situation of the components can be more accurately identified and classified. Therefore, in the technical scheme of the application, a method for calculating the differential feature vector is further adopted, and the difference operation is carried out on the detected semantic feature vector and the reference semantic feature vector, so that the differential feature vector representing the difference between the detected semantic feature vector and the reference semantic feature vector is obtained. Therefore, the feature difference information about the component placement features on the PCB in the detection image and the reference image can be captured, and the component placement condition can be judged.
And then, taking the differential feature vector as a classification feature vector to carry out classification processing in a classifier so as to obtain a classification result for indicating whether the components on the PCB to be detected are correctly placed. That is, in the technical solution of the present application, the label of the classifier includes that the component on the detected PCB is correctly placed (first label) and that the component on the detected PCB is not correctly placed (second label), wherein the classifier determines to which classification label the classification feature vector belongs through a soft maximum function. It should be noted that the first tag p1 and the second tag p2 do not include a concept set by human, and in fact, during the training process, the computer model does not have a concept of "whether the component on the PCB to be tested is correctly placed" which is simply a probability that there are two kinds of classification tags and the output feature is under the two kinds of classification tags, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the components on the detected PCB board are correctly placed is actually the classification probability distribution converted from the classification label into the classification according to the natural law, and the physical meaning of the natural probability distribution of the label is essentially used instead of the language text meaning of whether the components on the detected PCB board are correctly placed. It should be understood that, in the technical solution of the present application, the classification label of the classifier is a detection judgment label for determining whether the components on the PCB board to be detected are correctly placed, so after the classification result is obtained, it is possible to detect whether the components on the PCB board are correctly placed based on the classification result, and correct component placement errors and defects, so as to improve the efficiency and quality of the production line and reduce the risk of human errors.
Correspondingly, in the technical scheme of the application, the detection image is a field image of the detected PCB acquired by a camera, and the reference image is a design image provided based on computer aided design software. Although preprocessing the detection image of the detected PCB significantly improves the image quality of the detection image, the preprocessed detection image and the reference image still have pixel-level differences at the image source domain side. Further, after the sequence of the detected image blocks and the sequence of the reference image blocks are twinned and encoded by the dual detection model including the first image encoder and the second image encoder, the difference between the preprocessed detected image and the reference image at the image source domain end is amplified, so that the integral feature distribution of the differential feature vector between the detected semantic feature vector and the reference semantic feature vector has a classification weak correlation distribution example judged relative to the classification of the classifier. That is, the overall distribution of the differential feature vector has low compatibility under the classification judgment of the classifier, and influences the accuracy of the classification result obtained by the differential feature vector through the classifier.
Based on the above, in the technical solution of the present application, helmholtz free energy factors of the detected semantic feature vector and the reference semantic feature vector are calculated respectively, and specifically expressed as:
and->Respectively represent the detected semantic feature vectors +.>And the reference semantic feature vector +.>Is a classification probability value of>Is the length of the feature vector.
Here, the detected semantic feature vector may be based on the helmholtz free energy formulaAnd the reference semantic feature vector +.>The respective feature value sets describe the energy value of the predetermined class label as the class free energy of the feature vector as a whole by using it to detect the semantic feature vector +.>And the reference semantic feature vector +.>Weighting is performed to detect the semantic feature vector +.>And the reference semantic feature vector +.>Focusing on the class-related prototype instance (prototype instance) distribution of features overlapping with the truth instance (groundtruth instance) distribution in the class object domain so as to detect semantic feature vectors +.>And the reference semantic feature vector +.>Under the condition that a class weak correlation distribution example exists in the integral feature distribution, incremental learning is realized by carrying out fuzzy labeling on the class weak correlation distribution example, so that the compatibility of the integral feature distribution under a class label is improved, the accuracy of differential extraction between the detection semantic feature vector and the reference semantic feature vector is improved, and the accuracy of a classification result obtained by the differential feature vector through a classifier is improved. Therefore, whether the components on the PCB are correctly placed or not can be accurately detected, and component placement errors and defects are corrected, so that the efficiency and quality of a production line are improved, and the risk of human errors is reduced.
Fig. 1 is an application scenario diagram of an SMT intelligent error-proofing feeding detector according to an embodiment of the application. As shown in fig. 1, in this application scenario, first, a detection image (e.g., D1 illustrated in fig. 1) of a detected PCB (e.g., N illustrated in fig. 1) and a reference image (e.g., D2 illustrated in fig. 1) are acquired, wherein the detection image may be acquired by a camera (e.g., C illustrated in fig. 1), the reference image is an image of a PCB in which components are placed in a specification, and then the detection image of the detected PCB and the reference image are input to a server (e.g., S illustrated in fig. 1) in which an SMT smart error-proofing feeding detection algorithm is deployed, wherein the server is capable of processing the detection image of the detected PCB and the reference image using the SMT smart error-proofing feeding detection algorithm to obtain a classification result for indicating whether components on the detected PCB are placed correctly.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.
Fig. 2 is a schematic block diagram of an SMT intelligent error-proofing feeding detector according to an embodiment of the application. As shown in fig. 2, an SMT intelligent error-proofing feeding detector 100 according to an embodiment of the application includes: the detection image acquisition module 110 is used for acquiring a detection image of the PCB to be detected; the reference image acquisition module 120 is configured to acquire a reference image, where the reference image is an image of a PCB board with a standard component placement; the image preprocessing module 130 is configured to preprocess the detected image of the detected PCB to obtain a preprocessed detected image, where the preprocessing includes image denoising, and adjusting contrast and brightness; the image blocking processing module 140 is configured to perform image blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks; an image semantic feature extraction module 150, configured to pass the sequence of detected image blocks and the sequence of reference image blocks through a dual detection model including a first image encoder and a second image encoder, respectively, to obtain a detected semantic feature vector and a reference semantic feature vector; a feature difference module 160 for calculating a differential feature vector between the detected semantic feature vector and the reference semantic feature vector; and a placement detection module 170, configured to pass the differential feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the components on the detected PCB board are placed correctly.
More specifically, in the embodiment of the present application, the detection image acquisition module 110 is configured to acquire a detection image of the PCB board to be detected. Because the information quantity in the detection image is large, and the characteristic information such as the position, the direction, the polarity and the like of the components is small-scale fine characteristic information in the image, the capturing and the extracting are difficult. In addition, in the process of image acquisition, a large amount of noise in the image can be caused, and the feature extraction effect of the image is affected. Therefore, feature comparison can be performed in a high-dimensional feature space by utilizing the detection image of the PCB and the reference image which is normally placed by the standard components, so that classification detection on whether the components of the PCB are correctly placed is realized. According to the technical scheme, the detection image and the reference image are mined to obtain the difference characteristic information about the component hidden characteristic distribution of the PCB, so that whether the component on the PCB is correctly placed or not is detected, component placement errors and defects are corrected, production line efficiency and quality are improved, and the risk of human errors is reduced.
More specifically, in the embodiment of the present application, the reference image acquisition module 120 is configured to acquire a reference image, where the reference image is an image of a PCB board with a standard placement of components. The reference image is a design image provided based on computer aided design software.
More specifically, in the embodiment of the present application, the image preprocessing module 130 is configured to preprocess the detected image of the detected PCB board to obtain a preprocessed detected image, where the preprocessing includes image denoising, and adjusting contrast and brightness. In consideration of the fact that when components on the PCB are placed and detected, the acquired detection images may have certain noise, illumination and other problems, therefore, the images need to be subjected to denoising, contrast adjustment, brightness adjustment and other treatments before feature extraction is carried out, subsequent image blocking and feature extraction can be carried out more accurately, and further abnormal component placement detection accuracy on the PCB is improved. It should be understood that denoising can reduce interference information in an image, and adjusting contrast and brightness can enable details of components in the image to be clearer and more definite, so that accurate identification and classification of a subsequent model are facilitated.
More specifically, in the embodiment of the present application, the image blocking processing module 140 is configured to perform image blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks, respectively. Because there may be multiple components in a large PCB image, and different components may differ in terms of location, direction, size, etc., processing the entire image may result in inaccurate or missed detection. Therefore, when the components are placed, the detected PCB image needs to be divided into a plurality of small blocks for processing, that is, the preprocessed detection image and the reference image are respectively subjected to image blocking processing, so that the image information of each small block in the image can be obtained, and therefore, each component can be detected and compared more accurately later, and the sequence of the detection image blocks and the sequence of the reference image blocks can be obtained.
Accordingly, in one specific example, the image blocking processing module 140 is configured to: and respectively carrying out image uniform blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks.
More specifically, in the embodiment of the present application, the image semantic feature extraction module 150 is configured to pass the sequence of detected image blocks and the sequence of reference image blocks through a dual detection model including a first image encoder and a second image encoder to obtain a detected semantic feature vector and a reference semantic feature vector, respectively. Feature mining of the sequence of detected image blocks and the sequence of reference image blocks is performed using a convolutional neural network model having excellent performance in implicit feature extraction of images.
It should be appreciated that convolutional neural network (Convolutional Neural Network, CNN) is an artificial neural network and has wide application in the fields of image recognition and the like. The convolutional neural network may include an input layer, a hidden layer, and an output layer, where the hidden layer may include a convolutional layer, a pooling layer, an activation layer, a full connection layer, etc., where the previous layer performs a corresponding operation according to input data, outputs an operation result to the next layer, and obtains a final result after the input initial data is subjected to a multi-layer operation.
Accordingly, in one specific example, the first image encoder and the second image encoder have the same network structure. It should be understood that the feature extraction of the sequence of the detected image blocks and the sequence of the reference image blocks by using the dual detection model including the image encoders with the same network structure can extract feature information of the image blocks with insignificant difference at the image source domain end, so as to improve the accuracy of detection on whether the components on the detected PCB board are correctly placed. In addition, the traditional method can only extract some shallow image features, and cannot accurately express complex information such as the shape, the size, the position, the direction and the like of the components. Through advanced technologies such as deep learning, images can be abstracted and captured on multiple levels, and richer and more accurate semantic features are extracted.
Accordingly, in one specific example, as shown in fig. 3, the image semantic feature extraction module 150 includes: a first image encoding unit 151 for: performing two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transfer of layers by using each layer of the first image encoder to output the detection semantic feature vector by the last layer of the first image encoder, wherein the input of the first layer of the first image encoder is a sequence of the detection image blocks; and a second image encoding unit 152 for: and respectively carrying out two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transmission of layers by using each layer of the second image encoder to output the reference semantic feature vector by the last layer of the second image encoder, wherein the input of the first layer of the second image encoder is a sequence of the reference image blocks.
More specifically, in an embodiment of the present application, the feature difference module 160 is configured to calculate a differential feature vector between the detected semantic feature vector and the reference semantic feature vector. Considering that certain differences may exist in the shape, size, position, direction and the like of different components, the differences can be better described by calculating the differential feature vector, so that the placement situation of the components can be more accurately identified and classified. Therefore, in the technical scheme of the application, a method for calculating the differential feature vector is further adopted, and the difference operation is carried out on the detected semantic feature vector and the reference semantic feature vector, so that the differential feature vector representing the difference between the detected semantic feature vector and the reference semantic feature vector is obtained. Therefore, the feature difference information about the component placement features on the PCB in the detection image and the reference image can be captured, and the component placement condition can be judged.
Accordingly, in one specific example, as shown in fig. 4, the feature difference module 160 includes: an optimization factor calculation unit 161, configured to calculate a helmholtz type free energy factor of the detected semantic feature vector and the reference semantic feature vector to obtain a first helmholtz type free energy factor and a second helmholtz type free energy factor; a weighted optimization unit 162, configured to weight the detected semantic feature vector and the reference semantic feature vector with the first helmholtz class free energy factor and the second helmholtz class free energy factor as weighted weights to obtain an optimized detected semantic feature vector and an optimized reference semantic feature vector; and a difference feature optimization unit 163 for calculating the difference feature vector between the post-optimization detected semantic feature vector and the post-optimization reference semantic feature vector.
Correspondingly, in the technical scheme of the application, the detection image is a field image of the detected PCB acquired by a camera, and the reference image is a design image provided based on computer aided design software. Although preprocessing the detection image of the detected PCB significantly improves the image quality of the detection image, the preprocessed detection image and the reference image still have pixel-level differences at the image source domain side. Further, after the sequence of the detected image blocks and the sequence of the reference image blocks are twinned and encoded by the dual detection model including the first image encoder and the second image encoder, the difference between the preprocessed detected image and the reference image at the image source domain end is amplified, so that the integral feature distribution of the differential feature vector between the detected semantic feature vector and the reference semantic feature vector has a classification weak correlation distribution example judged relative to the classification of the classifier. That is, the overall distribution of the differential feature vector has low compatibility under the classification judgment of the classifier, and influences the accuracy of the classification result obtained by the differential feature vector through the classifier. Based on the above, in the technical scheme of the application, helmholtz class free energy factors of the detected semantic feature vector and the reference semantic feature vector are calculated respectively.
Accordingly, in a specific example, the optimization factor calculating unit 161 is configured to: calculating the Helmholtz type free energy factors of the detected semantic feature vector and the reference semantic feature vector according to the following optimization formula to obtain the first Helmholtz type free energy factor and the second Helmholtz type free energy factor; wherein, the optimization formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,characteristic values representing respective positions in said detected semantic characteristic vector,/or->Characteristic values representing respective positions in said reference semantic feature vector, ->Classification probability values representing said detected semantic feature vectors, ->Classification probability values representing the reference semantic feature vectors, and +.>Is the length of the feature vector, +.>Represents a logarithmic function with base 2, +.>Representing an exponential operation, ++>And->Representing the first helmholtz-like free energy factor and the second helmholtz-like free energy factor, respectively.
Here, based on the helmholtz free energy formula, the energy value of each feature value set of the detected semantic feature vector and the reference semantic feature vector for a predetermined class label can be described by the class free energy of the feature vector, and by weighting the detected semantic feature vector and the reference semantic feature vector by the feature value set, the class correlation prototype instance distribution of the features, which have overlapping property with the true instance distribution, of the detected semantic feature vector and the reference semantic feature vector in the class target domain can be focused, so that incremental learning can be realized by carrying out fuzzy labeling on the feature value set of each feature value set of the detected semantic feature vector and the reference semantic feature vector under the condition that the class weak correlation instance exists in the overall feature distribution of the detected semantic feature vector and the reference semantic feature vector, thereby improving the compatibility of the overall feature distribution under the class label, and improving the accuracy of differential extraction between the detected semantic feature vector and the reference semantic feature vector, thereby improving the accuracy of the classification result obtained by the classifier. Therefore, whether the components on the PCB are correctly placed or not can be accurately detected, and component placement errors and defects are corrected, so that the efficiency and quality of a production line are improved, and the risk of human errors is reduced.
Accordingly, in one specific example, the difference feature optimizing unit 163 is configured to: calculating the differential feature vector between the optimized detection semantic feature vector and the optimized reference semantic feature vector according to the following differential calculation formula; wherein, the difference calculation formula is:wherein (1)>Representing the optimized post-detection semantic feature vector, < >>Representing the optimized reference semantic feature vector, < >>Representing the differential eigenvector,>representing the difference by location.
More specifically, in the embodiment of the present application, the placement detection module 170 is configured to pass the differential feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the component on the PCB board to be detected is placed correctly. After the classification result is obtained, whether the components on the PCB are correctly placed or not can be detected based on the classification result, and component placement errors and defects are corrected, so that the production line efficiency and quality are improved, and the risk of human errors is reduced.
It should be appreciated that the role of the classifier is to learn the classification rules and classifier using a given class, known training data, and then classify (or predict) the unknown data. Logistic regression (logistics), SVM, etc. are commonly used to solve the classification problem, and for multi-classification problems (multi-class classification), logistic regression or SVM can be used as well, but multiple bi-classifications are required to compose multiple classifications, but this is error-prone and inefficient, and the commonly used multi-classification method is the Softmax classification function.
Accordingly, in one specific example, as shown in fig. 5, the placement detection module 170 includes: a full-connection encoding unit 171, configured to perform full-connection encoding on the differential feature vector by using a full-connection layer of the classifier to obtain an encoded classification feature vector; and a classification unit 172, configured to input the encoded classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
In summary, the SMT intelligent error-proofing feeding detector 100 according to the embodiment of the present application is illustrated, firstly, a detected image of a detected PCB board is preprocessed to obtain a preprocessed detected image, then, image blocking processing is performed on the preprocessed detected image and the reference image to obtain a sequence of detected image blocks and a sequence of reference image blocks, then, the sequence of detected image blocks and the sequence of reference image blocks are respectively passed through a dual detection model to obtain a detected semantic feature vector and a reference semantic feature vector, then, a differential feature vector between the detected semantic feature vector and the reference semantic feature vector is calculated, and finally, the differential feature vector is passed through a classifier to obtain a classification result for indicating whether components on the detected PCB board are correctly placed. Thus, the efficiency and quality of the production line can be improved, and the risk of human errors can be reduced.
As described above, the SMT intelligent error-proofing feeding detector 100 according to the embodiment of the present application may be implemented in various terminal devices, for example, a server having the SMT intelligent error-proofing feeding detection algorithm according to the embodiment of the present application, and the like. In one example, the SMT intelligent error-proofing feeding detector 100 according to an embodiment of the present application may be integrated into the terminal device as a software module and/or a hardware module. For example, the SMT intelligent error-proofing feeding detector 100 according to the embodiment of the present application may be a software module in the operating system of the terminal device, or may be an application program developed for the terminal device; of course, the SMT intelligent error-proofing feeding detector 100 according to the embodiment of the present application may also be one of a plurality of hardware modules of the terminal device.
Alternatively, in another example, the SMT intelligent error-proofing feeding detector 100 according to an embodiment of the present application and the terminal device may be separate devices, and the SMT intelligent error-proofing feeding detector 100 may be connected to the terminal device through a wired and/or wireless network, and transmit the interaction information according to a agreed data format.
Fig. 6 is a flowchart of an SMT intelligent error-proofing feeding detection method according to an embodiment of the application. As shown in fig. 6, an SMT intelligent error-proofing feeding detection method according to an embodiment of the application includes: s110, obtaining a detection image of the PCB to be detected; s120, acquiring a reference image, wherein the reference image is an image of a PCB with standard component placement; s130, preprocessing the detection image of the detected PCB to obtain a preprocessed detection image, wherein the preprocessing comprises image denoising, contrast adjustment and brightness adjustment; s140, performing image blocking processing on the preprocessed detection image and the reference image respectively to obtain a sequence of detection image blocks and a sequence of reference image blocks; s150, respectively passing the sequence of the detection image blocks and the sequence of the reference image blocks through a double detection model comprising a first image encoder and a second image encoder to obtain a detection semantic feature vector and a reference semantic feature vector; s160, calculating a differential feature vector between the detected semantic feature vector and the reference semantic feature vector; and S170, enabling the differential feature vector to pass through a classifier to obtain a classification result, wherein the classification result is used for indicating whether components on the PCB to be detected are correctly placed.
Fig. 7 is a schematic diagram of a system architecture of an SMT intelligent error-proofing feeding detection method according to an embodiment of the present application. As shown in fig. 7, in the system architecture of the SMT intelligent error-proofing feeding detection method, first, a detection image of a PCB to be detected is obtained; then, a reference image is acquired, wherein the reference image is an image of a PCB with standard component placement; then, preprocessing the detection image of the detected PCB to obtain a preprocessed detection image, wherein the preprocessing comprises image denoising, contrast adjustment and brightness adjustment; then, respectively carrying out image blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks; then, the sequence of the detected image blocks and the sequence of the reference image blocks are respectively passed through a double detection model comprising a first image encoder and a second image encoder to obtain a detected semantic feature vector and a reference semantic feature vector; then, calculating a differential feature vector between the detected semantic feature vector and the reference semantic feature vector; and finally, the differential feature vector passes through a classifier to obtain a classification result, wherein the classification result is used for indicating whether components on the PCB to be detected are correctly placed.
In a specific example, in the above SMT intelligent error-proofing feeding detection method, performing image blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks, respectively, including: and respectively carrying out image uniform blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks.
In a specific example, in the above SMT smart error-proofing feeding detection method, the first image encoder and the second image encoder have the same network structure.
In a specific example, in the above SMT intelligent error-proofing feeding detection method, passing the sequence of the detected image block and the sequence of the reference image block through a dual detection model including a first image encoder and a second image encoder to obtain a detected semantic feature vector and a reference semantic feature vector, respectively, including: performing two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transfer of layers by using each layer of the first image encoder to output the detection semantic feature vector by the last layer of the first image encoder, wherein the input of the first layer of the first image encoder is a sequence of the detection image blocks; and performing two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transfer of layers by using each layer of the second image encoder to output the reference semantic feature vector by a last layer of the second image encoder, respectively, wherein an input of a first layer of the second image encoder is a sequence of the reference image blocks.
In a specific example, in the above SMT intelligent error-proofing feeding detection method, calculating a differential feature vector between the detected semantic feature vector and the reference semantic feature vector includes: calculating Helmholtz class free energy factors of the detection semantic feature vector and the reference semantic feature vector to obtain a first Helmholtz class free energy factor and a second Helmholtz class free energy factor; the first Helmholtz class free energy factor and the second Helmholtz class free energy factor are used as weighting weights to weight the detection semantic feature vector and the reference semantic feature vector so as to obtain an optimized detection semantic feature vector and an optimized reference semantic feature vector; and calculating the differential feature vector between the optimized detected semantic feature vector and the optimized reference semantic feature vector.
In a specific example, in the above SMT smart error-proofing feeding detection method, calculating a helmholtz class free energy factor of the detected semantic feature vector and the reference semantic feature vector to obtain a first helmholtz class free energy factor and a second helmholtz class free energy factor, includes: calculating the Helmholtz type free energy factors of the detected semantic feature vector and the reference semantic feature vector according to the following optimization formula to obtain the first Helmholtz type free energy factor and the second Helmholtz type free energy factor; wherein, the optimization formula is:
Wherein, the liquid crystal display device comprises a liquid crystal display device,characteristic values representing respective positions in said detected semantic characteristic vector,/or->Characteristic values representing respective positions in said reference semantic feature vector, ->Classification probability values representing said detected semantic feature vectors, ->Classification probability values representing the reference semantic feature vectors, and +.>Is the length of the feature vector, +.>Represents a logarithmic function with base 2, +.>Representing an exponential operation, ++>And->Representing the first helmholtz-like free energy factor and the second helmholtz-like free energy factor, respectively.
In a specific example, in the above SMT intelligent error-proofing feeding detection method, calculating the differential feature vector between the post-optimization detected semantic feature vector and the post-optimization reference semantic feature vector includes: calculating the difference between the post-optimization detected semantic feature vector and the post-optimization reference semantic feature vector in a difference calculation formula as followsA feature vector; wherein, the difference calculation formula is:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the optimized post-detection semantic feature vector, < >>Representing the optimized reference semantic feature vector, < >>Representing the differential eigenvector,>representing the difference by location.
In a specific example, in the above SMT intelligent error-proofing feeding detection method, the step of passing the differential feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether components on the PCB to be detected are correctly placed, includes: performing full-connection coding on the differential feature vector by using a full-connection layer of the classifier to obtain a coded classification feature vector; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
Here, it will be understood by those skilled in the art that the specific operations of the respective steps in the above-described SMT smart error-proofing feeding detection method have been described in detail in the above description of the SMT smart error-proofing feeding detector 100 with reference to fig. 1 to 5, and thus, repetitive descriptions thereof will be omitted.
According to another aspect of the present application there is also provided a non-volatile computer readable storage medium having stored thereon computer readable instructions which when executed by a computer can perform a method as described above.
Program portions of the technology may be considered to be "products" or "articles of manufacture" in the form of executable code and/or associated data, embodied or carried out by a computer readable medium. A tangible, persistent storage medium may include any memory or storage used by a computer, processor, or similar device or related module. Such as various semiconductor memories, tape drives, disk drives, or the like, capable of providing storage functionality for software.
All or a portion of the software may sometimes communicate over a network, such as the internet or other communication network. Such communication may load software from one computer device or processor to another. For example: a hardware platform loaded from a server or host computer of the video object detection device to a computer environment, or other computer environment implementing the system, or similar functioning system related to providing information needed for object detection. Thus, another medium capable of carrying software elements may also be used as a physical connection between local devices, such as optical, electrical, electromagnetic, etc., propagating through cable, optical cable, air, etc. Physical media used for carrier waves, such as electrical, wireless, or optical, may also be considered to be software-bearing media. Unless limited to a tangible "storage" medium, other terms used herein to refer to a computer or machine "readable medium" mean any medium that participates in the execution of any instructions by a processor.
The application uses specific words to describe embodiments of the application. Reference to "a first/second embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the application. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the application may be combined as suitable.
Furthermore, those skilled in the art will appreciate that the various aspects of the application are illustrated and described in the context of a number of patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the application may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.) or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the application may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the following claims. It is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims (6)

1. SMT intelligence mistake proofing material loading detector, its characterized in that includes:
the detection image acquisition module is used for acquiring a detection image of the PCB to be detected;
the reference image acquisition module is used for acquiring a reference image, wherein the reference image is an image of a PCB with standard component placement;
the image preprocessing module is used for preprocessing the detection image of the detected PCB to obtain a preprocessed detection image, wherein the preprocessing comprises image denoising, contrast adjustment and brightness adjustment;
The image blocking processing module is used for respectively carrying out image blocking processing on the preprocessed detection image and the reference image so as to obtain a sequence of detection image blocks and a sequence of reference image blocks;
the image semantic feature extraction module is used for respectively passing the sequence of the detection image blocks and the sequence of the reference image blocks through a double detection model comprising a first image encoder and a second image encoder to obtain detection semantic feature vectors and reference semantic feature vectors;
the feature difference module is used for calculating a difference feature vector between the detected semantic feature vector and the reference semantic feature vector; and
the placement detection module is used for enabling the differential feature vector to pass through a classifier to obtain a classification result, wherein the classification result is used for indicating whether components on the PCB to be detected are placed correctly or not;
the feature difference module comprises:
the optimization factor calculation unit is used for calculating the Helmholtz class free energy factors of the detection semantic feature vector and the reference semantic feature vector to obtain a first Helmholtz class free energy factor and a second Helmholtz class free energy factor;
the weighted optimization unit is used for weighting the detected semantic feature vector and the reference semantic feature vector by taking the first Helmholtz class free energy factor and the second Helmholtz class free energy factor as weighted weights so as to obtain an optimized detected semantic feature vector and an optimized reference semantic feature vector; and
The difference feature optimization unit is used for calculating the difference feature vector between the optimized detection semantic feature vector and the optimized reference semantic feature vector;
the optimization factor calculation unit is used for:
calculating the Helmholtz type free energy factors of the detected semantic feature vector and the reference semantic feature vector according to the following optimization formula to obtain the first Helmholtz type free energy factor and the second Helmholtz type free energy factor;
wherein, the optimization formula is:
wherein v is 1i Feature values, v, representing respective positions in the detected semantic feature vector 2i Feature values representing respective positions in the reference semantic feature vector, p 1 Classification probability value, p, representing the detected semantic feature vector 2 Represents the classification probability value of the reference semantic feature vector, L is the length of the feature vector, log represents a logarithmic function based on 2, exp (·) represents an exponential operation, w 1 And w 2 Representing the first helmholtz-like free energy factor and the second helmholtz-like free energy factor, respectively.
2. The SMT intelligent error-proofing feeding detector according to claim 1, wherein said image blocking processing module is configured to:
And respectively carrying out image uniform blocking processing on the preprocessed detection image and the reference image to obtain a sequence of detection image blocks and a sequence of reference image blocks.
3. The SMT intelligent error-proofing feeding detector according to claim 2, wherein said first image encoder and said second image encoder have the same network structure.
4. The SMT intelligent error-proofing feeding detector according to claim 3, wherein said image semantic feature extraction module comprises:
a first image encoding unit configured to: performing two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transfer of layers by using each layer of the first image encoder to output the detection semantic feature vector by the last layer of the first image encoder, wherein the input of the first layer of the first image encoder is a sequence of the detection image blocks; and
a second image encoding unit configured to: and respectively carrying out two-dimensional convolution processing, pooling processing along a channel dimension and nonlinear activation processing on input data in forward transmission of layers by using each layer of the second image encoder to output the reference semantic feature vector by the last layer of the second image encoder, wherein the input of the first layer of the second image encoder is a sequence of the reference image blocks.
5. The SMT intelligent error-proofing feeding detector according to claim 4, wherein said difference feature optimization unit is configured to:
calculating the differential feature vector between the optimized detection semantic feature vector and the optimized reference semantic feature vector according to the following differential calculation formula;
wherein, the difference calculation formula is:
wherein V is a Representing the post-optimization detected semantic feature vector, V b Representing the optimized reference semantic feature vector, V c The differential feature vector is represented as such,representing the difference by location.
6. The SMT intelligent error proofing feeding detector according to claim 5, wherein said placement detection module comprises:
the full-connection coding unit is used for carrying out full-connection coding on the differential feature vectors by using a full-connection layer of the classifier so as to obtain coded classification feature vectors; and
and the classification unit is used for inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
CN202310841051.6A 2023-07-11 2023-07-11 SMT intelligent error-proofing feeding detector Active CN116563291B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310841051.6A CN116563291B (en) 2023-07-11 2023-07-11 SMT intelligent error-proofing feeding detector

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310841051.6A CN116563291B (en) 2023-07-11 2023-07-11 SMT intelligent error-proofing feeding detector

Publications (2)

Publication Number Publication Date
CN116563291A CN116563291A (en) 2023-08-08
CN116563291B true CN116563291B (en) 2023-09-22

Family

ID=87503899

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310841051.6A Active CN116563291B (en) 2023-07-11 2023-07-11 SMT intelligent error-proofing feeding detector

Country Status (1)

Country Link
CN (1) CN116563291B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117726574B (en) * 2023-09-13 2024-04-26 东莞市言科新能源有限公司 Packaging system and method for producing polymer lithium ion battery
CN118072356A (en) * 2024-04-11 2024-05-24 克拉玛依市富城油气研究院有限公司 Well site remote monitoring system and method based on Internet of things technology

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112053318A (en) * 2020-07-20 2020-12-08 清华大学 Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning
CN116051500A (en) * 2023-01-09 2023-05-02 吉安市臻普达电子科技有限公司 Intelligent processing method and system for PCB
CN116152244A (en) * 2023-04-19 2023-05-23 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) SMT defect detection method and system
CN116295752A (en) * 2023-05-12 2023-06-23 深圳市蓝眼科技有限公司 Test strength control method and system for SMT (surface mounting technology) feeding equipment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11978197B2 (en) * 2021-10-28 2024-05-07 Hilti Aktiengesellschaft Inspection method for inspecting an object and machine vision system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112053318A (en) * 2020-07-20 2020-12-08 清华大学 Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning
CN116051500A (en) * 2023-01-09 2023-05-02 吉安市臻普达电子科技有限公司 Intelligent processing method and system for PCB
CN116152244A (en) * 2023-04-19 2023-05-23 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) SMT defect detection method and system
CN116295752A (en) * 2023-05-12 2023-06-23 深圳市蓝眼科技有限公司 Test strength control method and system for SMT (surface mounting technology) feeding equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
深度学习及其在图像物体分类与检测中的应用综述;刘栋;李素;曹志冬;;计算机科学(第12期);全文 *

Also Published As

Publication number Publication date
CN116563291A (en) 2023-08-08

Similar Documents

Publication Publication Date Title
CN116563291B (en) SMT intelligent error-proofing feeding detector
CN109615016B (en) Target detection method of convolutional neural network based on pyramid input gain
US11455528B2 (en) Automated optical inspection and classification apparatus based on a deep learning system and training apparatus thereof
CN111444921A (en) Scratch defect detection method and device, computing equipment and storage medium
JP6401648B2 (en) Defect classification apparatus and defect classification method
US20200134382A1 (en) Neural network training utilizing specialized loss functions
CN110648305A (en) Industrial image detection method, system and computer readable recording medium
CN116363738A (en) Face recognition method, system and storage medium based on multiple moving targets
CN116597377A (en) Intelligent monitoring management method and system for cattle breeding
CN116704431A (en) On-line monitoring system and method for water pollution
CN114494780A (en) Semi-supervised industrial defect detection method and system based on feature comparison
CN111758117A (en) Inspection system, recognition system, and learning data generation device
CN116482524A (en) Power transmission and distribution switch state detection method and system
CN116298880A (en) Micro-motor reliability comprehensive test system and method thereof
CN116624903A (en) Intelligent monitoring method and system for oil smoke pipeline
CN116295752A (en) Test strength control method and system for SMT (surface mounting technology) feeding equipment
CN114549414A (en) Abnormal change detection method and system for track data
CN112949785B (en) Object detection method, device, equipment and computer storage medium
CN116502899B (en) Risk rating model generation method, device and storage medium based on artificial intelligence
CN116052061B (en) Event monitoring method, event monitoring device, electronic equipment and storage medium
CN116402777A (en) Power equipment detection method and system based on machine vision
US11715288B2 (en) Optical character recognition using specialized confidence functions
CN114494152A (en) Unsupervised change detection method based on associated learning model
CN111160330B (en) Training method for improving image recognition accuracy with assistance of electronic tag recognition
CN114692887A (en) Semi-supervised learning system and semi-supervised learning method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant