CN116563831A - AR-based circuit board welding assembly quality analysis method and system - Google Patents
AR-based circuit board welding assembly quality analysis method and system Download PDFInfo
- Publication number
- CN116563831A CN116563831A CN202310171490.0A CN202310171490A CN116563831A CN 116563831 A CN116563831 A CN 116563831A CN 202310171490 A CN202310171490 A CN 202310171490A CN 116563831 A CN116563831 A CN 116563831A
- Authority
- CN
- China
- Prior art keywords
- circuit board
- quality analysis
- assembly
- assembly quality
- electronic component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 92
- 238000003466 welding Methods 0.000 title claims abstract description 67
- 238000001514 detection method Methods 0.000 claims abstract description 42
- 238000000034 method Methods 0.000 claims description 23
- 229910000679 solder Inorganic materials 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 9
- 238000013135 deep learning Methods 0.000 abstract description 7
- 230000005540 biological transmission Effects 0.000 abstract description 6
- 238000012545 processing Methods 0.000 abstract description 5
- 239000011521 glass Substances 0.000 description 17
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 5
- 238000005476 soldering Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 208000003464 asthenopia Diseases 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- 235000012766 Cannabis sativa ssp. sativa var. sativa Nutrition 0.000 description 1
- 235000012765 Cannabis sativa ssp. sativa var. spontanea Nutrition 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 235000009120 camo Nutrition 0.000 description 1
- 239000003990 capacitor Substances 0.000 description 1
- 235000005607 chanvre indien Nutrition 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000011487 hemp Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention provides an AR-based circuit board welding assembly quality analysis method, an AR-based circuit board welding assembly quality analysis system, a storage medium and electronic equipment, and relates to the technical field of circuit board welding assembly quality analysis. In the invention, a circuit board image is acquired based on AR equipment; identifying the position and category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment; according to the identification result of the electronic component, at least acquiring the assembly quality analysis result of the wrong welding and the missing welding of the circuit board based on a pre-constructed assembly template; and displaying the assembly quality analysis result in a mixed reality space based on the AR equipment. The problem of poor detection robustness of electronic components in traditional image processing is solved by adopting a target detection technology based on deep learning; and the lightweight electronic component detection model is directly deployed on the AR equipment, so that time delay and data transmission to the background are avoided.
Description
Technical Field
The invention relates to the technical field of circuit board welding assembly quality analysis, in particular to an AR-based circuit board welding assembly quality analysis method, an AR-based circuit board welding assembly quality analysis system, a storage medium and electronic equipment.
Background
The circuit board is an important basic component in the electromechanical system assembly, along with the rapid development of microelectronic technology and computer technology, electronic components become more miniaturized and diversified, and as many as hundreds of complicated circuit board welding points are easy to cause the conditions of miswelding and missing welding, so that accidents occur and economic losses are caused, and the welded circuit board needs to be subjected to assembly quality analysis so as to ensure the product performance.
Early-stage circuit board misplug and missing welding are mainly detected in a manual visual inspection mode, but the detection is greatly affected by the working experience of detection personnel and is easy to cause missing detection due to visual fatigue, so that the development of automatic misplug and missing welding based on computer vision has practical application requirements.
Electronic component detection algorithms are mainly divided into two main categories: one is a method based on conventional image classification, in which an image is input first, then features of the image are extracted through a complicated procedure, and classification is performed according to the extracted features. The other is a method based on deep learning, which comprises the steps of inputting an image, automatically extracting features through a convolutional neural network to generate a feature map, and positioning and classifying targets according to the feature map.
However, the detection mode based on the traditional image processing is poor in robustness and easy to misplug and miss, the method based on the deep learning is high in calculation force requirement and poor in instantaneity, and the circuit board is imaged based on the industrial camera at present, so that the assembly quality analysis result is not conveniently displayed.
Disclosure of Invention
(one) solving the technical problems
Aiming at the defects of the prior art, the invention provides an AR-based circuit board welding assembly quality analysis method, an AR-based circuit board welding assembly quality analysis system, a storage medium and electronic equipment, and solves the technical problem that robustness and instantaneity cannot be considered.
(II) technical scheme
In order to achieve the above purpose, the invention is realized by the following technical scheme:
a circuit board welding assembly quality analysis method based on AR includes:
acquiring a circuit board image based on the AR equipment;
identifying the position and category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment;
according to the identification result of the electronic component, at least acquiring the assembly quality analysis result of the wrong welding and the missing welding of the circuit board based on a pre-constructed assembly template;
and displaying the assembly quality analysis result in a mixed reality space based on the AR equipment.
Preferably, the method for analyzing the welding assembly quality of the circuit board based on AR further comprises the following steps: alarming the abnormal assembly quality analysis result; and recording at least the abnormal assembly quality analysis results in a device archive of the server.
Preferably, the lightweight electronic component detection model adopts an end-to-end YOLO V4 network model, and the feature extraction network of the YOLO V4 network model adopts a lightweight mobilet network.
Preferably, the process of obtaining the assembly quality analysis result of the circuit board error welding and the leak welding includes:
based on the principle of minimizing Euclidean distance, matching the identification result of the electronic component with the electronic component of the assembly template; if the detected label is inconsistent with the assembly template, judging that the label is wrongly welded; if the electronic components exist in the assembly template, but the corresponding labels are not detected, the assembly template is judged to be in a missing welding state.
Preferably, after correcting the recognition result of the electronic component, acquiring an assembly quality analysis result of the circuit board in the wrong welding and the missing welding based on the corrected recognition result.
Preferably, the correcting the identification result of the electronic component includes:
acquiring pose information of the circuit board image according to auxiliary positioning marks around the positioning frame; the positioning frame is a region for horizontally placing the circuit board;
according to the pose information, obtaining plane coordinates of each electronic component on the circuit board by affine transformation;
and unifying the shooting angles of the circuit board images according to the plane coordinates of the electronic components.
An AR-based circuit board solder assembly quality analysis system, comprising:
the acquisition module is used for acquiring the circuit board image based on the AR equipment;
the identification module is used for identifying the position and the category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment;
the analysis module is used for at least acquiring assembly quality analysis results of the circuit board in the wrong welding and the missing welding based on a pre-constructed assembly template according to the identification result of the electronic component;
and the display module is used for displaying the analysis assembly quality analysis result in the mixed reality space based on the AR equipment.
Preferably, the circuit board welding assembly quality analysis system further includes:
the alarm module is used for alarming abnormal assembly quality analysis results;
and the recording module is used for recording at least the abnormal assembly quality analysis result in a device archive of the server.
A storage medium storing a computer program for AR-based circuit board solder fitting quality analysis, wherein the computer program causes a computer to execute the circuit board solder fitting quality analysis method as described above.
An electronic device, comprising:
one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the circuit board solder assembly quality analysis method as described above.
(III) beneficial effects
The invention provides an AR-based circuit board welding assembly quality analysis method, an AR-based circuit board welding assembly quality analysis system, a storage medium and electronic equipment. Compared with the prior art, the method has the following beneficial effects:
in the invention, a circuit board image is acquired based on AR equipment; identifying the position and category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment; according to the identification result of the electronic component, at least acquiring the assembly quality analysis result of the wrong welding and the missing welding of the circuit board based on a pre-constructed assembly template; and displaying the assembly quality analysis result in a mixed reality space based on the AR equipment. The problem of poor detection robustness of electronic components in traditional image processing is solved by adopting a target detection technology based on deep learning; and the lightweight electronic component detection model is directly deployed on the AR equipment, so that time delay and data transmission to the background are avoided.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an AR-based circuit board welding assembly quality analysis method provided by an embodiment of the invention;
FIG. 2 is a block diagram of an AR-based circuit board welding assembly quality analysis method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a mobile-based electronic component detection model according to an embodiment of the present invention;
fig. 4 is a schematic view of circuit board shooting according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an inspection circuit board for wearing AR glasses according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
According to the AR-based circuit board welding assembly quality analysis method, system, storage medium and electronic equipment, the technical problem that robustness and instantaneity cannot be considered is solved.
The technical scheme in the embodiment of the application aims to solve the technical problems, and the overall thought is as follows:
the circuit board welding assembly quality analysis method provided by the embodiment of the invention is based on the front end (such as AR glasses) of the AR equipment and the background server. The light electronic component detection model is directly deployed to the AR glasses, so that the complexity of image data transmission to a background server is reduced, and the time consumption of welding assembly quality analysis is also reduced.
As shown in fig. 1, the technical scheme can be summarized as follows: collecting circuit board images based on AR glasses; extracting characteristics of categories and positions of electronic components on a circuit board by using a lightweight electronic component detection algorithm running on the AR glasses; analyzing the electronic component recognition result based on the circuit board assembly to find assembly problems such as misplacement welding, leakage welding and the like of the circuit board; the AR glasses can be used for displaying and alarming the miscelding and missing welding analysis results in the mixed reality space, and uploading the miscelding and missing welding analysis results to a server to record the miscelding and missing welding analysis results in a device archive of software.
The light end-to-end target detection algorithm is adopted for detection, so that the robustness of deep learning detection is realized, the computational power deployment requirement is reduced, and the detection speed is improved; in addition, based on the AR glasses, the assembly quality analysis results are displayed, and welding personnel are also conveniently guided to repair.
In order to better understand the above technical solutions, the following detailed description will refer to the accompanying drawings and specific embodiments.
Examples:
as shown in fig. 2, an embodiment of the present invention provides a circuit board welding assembly quality analysis method based on AR, including:
s1, acquiring a circuit board image based on AR equipment;
s2, identifying the position and the category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment;
s3, at least acquiring assembly quality analysis results of the circuit board in the wrong welding and the missing welding based on a pre-constructed assembly template according to the identification result of the electronic component;
s4, displaying the assembly quality analysis result in a mixed reality space based on the AR equipment;
s5, alarming abnormal assembly quality analysis results;
s6, at least recording the abnormal assembly quality analysis result in a device archive of the server.
The embodiment of the invention effectively solves the problem of visual fatigue in the current welding assembly quality analysis of the artificial circuit board; the problem of poor detection robustness of electronic components in traditional image processing is solved by adopting a target detection technology based on deep learning; the lightweight electronic component detection model is directly deployed on the AR equipment, so that time delay and data transmission to the background are avoided; interactive presentation and problem recording based on AR devices facilitates improved user experience.
The following will describe each step of the above technical solution in detail:
in step S1, a circuit board image is acquired based on the AR device.
In the step, a camera equipped with AR glasses adopts a high-definition video monitoring technology to realize high-definition acquisition, high-definition coding, high-definition transmission, high-definition storage and high-definition display of video image information.
In step S2, the position and type information of the electronic component on the circuit board image are identified according to the lightweight electronic component detection model deployed on the AR device.
It is easy to understand that the misloading and neglected loading of parts are common assembly problems in the electric assembly process, and in order to realize the automatic identification of abnormal assembly of the electric assembly process, firstly, the assembly characteristics of the electric assembly process need to be extracted, wherein the detection of electronic components such as resistors, capacitors, chips and the like is the key of the assembly characteristics, and the type and position information of the electronic components need to be acquired.
Because the electronic components on a typical circuit board are closely-packed hemp, the size difference is also large, the smaller electronic components bring larger detection difficulty, and meanwhile, real-time feedback is required to be achieved on the electronic components in order to ensure the use experience, so that the electronic components are detected by selecting an end-to-end YOLO V4 algorithm in the step, the universality of components with different scales is ensured by the hierarchical characteristics, the detection requirements are met by utilizing the end-to-end algorithm design, meanwhile, a lightweight Mobilene is adopted to replace a feature extraction network in YOLO V4, the detection speed is further improved, and the hardware requirement of a model is reduced, so that the detection model is conveniently deployed on front-end AR glasses.
The light-weight electronic component detection model structure is shown in fig. 3, and specifically comprises:
firstly, scaling the circuit board image acquired by the AR glasses in the step S1 into an input network with a specified size through the image; then, the Mobilene network is utilized to extract the hierarchical characteristics, and the Mobilene network uses depth separable convolution to replace common convolution, so that the calculation amount of the network is greatly reduced; and finally, up-sampling the upper layer features extracted by the Mobilene network by referring to the YOLO V4 algorithm, and fusing the upper layer features with the bottom layer features, and continuously using the fusion structure for three times to obtain corresponding three layers of features. And detecting all the three characteristics to obtain the position and category information of the electronic components.
In step S3, at least the assembly quality analysis results of the circuit board with the wrong soldering and the missing soldering are obtained based on the pre-constructed assembly template according to the identification result of the electronic component.
Particularly, before the assembly quality analysis in the step S3 is executed, the embodiment of the invention further includes correcting the identification result of the electronic component, and acquiring the assembly quality analysis result of the circuit board with wrong soldering and missing soldering based on the corrected identification result.
The electronic components on the circuit board are not arranged in any regularity, so that the accurate positions of the misplaced and the welding spots are determined by combining coordinate conversion operations between 3D and 2D projections by means of the space positioning capability of the AR glasses.
Specifically, the correcting the identification result of the electronic component includes:
s10, acquiring pose information of the circuit board image according to auxiliary positioning marks (marks with certain special shapes, such as triangular or circular non-reflective material stickers and the like shown in FIG. 4) around a positioning frame; the positioning frame is an area for horizontally placing the circuit board.
And S20, obtaining plane coordinates of each electronic component on the circuit board by affine transformation according to the pose information.
As shown in fig. 5, when different staff wear the AR to inspect the circuit board, an indefinite included angle and distance exist, and affine transformation can counteract image deformation and scaling caused by the affine transformation.
Exemplary, 3D to 2D projection coordinate transformation matrix operations are as follows
Denoted as Z 2D ＝E×Z′ 3D Since the thickness of the circuit board is almost negligible, z' 3 Can be 0, so E is reduced from a 2x3 matrix to a 2x1 matrix, denoted as Z 2D 。
And S30, unifying the shooting angles of the circuit board images according to the plane coordinates of the electronic components.
After the plane coordinates of the electronic components on the circuit board are obtained, the circuit board is corrected by using the more obvious electronic components (such as the electronic components with the largest size) on the circuit board, so that the circuit board on the acquired image and the circuit board of the template library are prevented from being upside down and left and right reversed.
The process of obtaining the assembly quality analysis result of the circuit board misplug and missing welding after the corrected identification result comprises the following steps:
based on the principle of minimizing Euclidean distance, matching the identification result of the electronic component with the electronic component of the assembly template; if the detected label is inconsistent with the assembly template, judging that the label is wrongly welded; if the electronic components exist in the assembly template, but the corresponding labels are not detected, the assembly template is judged to be in a missing welding state.
In step S4, the assembly quality analysis result is displayed in a mixed reality space based on the AR device.
In step S5, the abnormal assembly quality analysis result is alerted.
And the steps S4 to S5 can utilize the sensor technology and the mixed reality technology of AR glasses configuration to display and alarm the identified welding abnormality of the circuit board.
In step S6, at least the abnormal fitting quality analysis result is recorded in a device archive of the server.
The step provides informationized device management means, so that the information management problem of each device can be effectively solved, and the device (circuit board) assembly exception handling efficiency is accelerated.
For example, in a specific application, the embodiment of the invention can perform software development based on the AR device and the Unity3D development platform. The development of the front-end system of the AR glasses is completed based on the C# language, and finally the program is deployed into the AR glasses. Using the Unity3D plug-in MRTK (Mixed Reality Toolkit) provided by Microsoft corporation, it provides some components and functionality to speed up the development of applications, including providing input systems and building foundation for spatial interactions and UI interfaces, implementing simulation prototyping within an editor, etc. The client is mainly developed in Unity3D by using MRTK plug-in, after creating an application, the application is released into the UWP platform, and then the released application is deployed into AR glasses in Visual Studio 2019.
The embodiment of the invention provides an AR-based circuit board welding assembly quality analysis system, which comprises the following components:
the acquisition module is used for acquiring the circuit board image based on the AR equipment;
the identification module is used for identifying the position and the category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment;
the analysis module is used for at least acquiring assembly quality analysis results of the circuit board in the wrong welding and the missing welding based on a pre-constructed assembly template according to the identification result of the electronic component;
the display module is used for displaying the analysis assembly quality analysis result in a mixed reality space based on the AR equipment;
the alarm module is used for alarming abnormal assembly quality analysis results;
and the recording module is used for recording at least the abnormal assembly quality analysis result in a device archive of the server.
An embodiment of the present invention provides a storage medium storing a computer program for AR-based circuit board solder fitting quality analysis, wherein the computer program causes a computer to execute the circuit board solder fitting quality analysis method as described above.
The embodiment of the invention provides electronic equipment, which comprises:
one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the circuit board solder assembly quality analysis method as described above.
It may be understood that the AR-based circuit board welding assembly quality analysis system, the storage medium and the electronic device provided by the embodiments of the present invention correspond to the AR-based circuit board welding assembly quality analysis method provided by the embodiments of the present invention, and the explanation, the examples, the beneficial effects and other parts of the relevant content may refer to the corresponding parts in the circuit board welding assembly quality analysis method, which are not described herein again.
In summary, compared with the prior art, the method has the following beneficial effects:
1. the embodiment of the invention effectively solves the problem of visual fatigue in the current welding assembly quality analysis of the artificial circuit board.
2. The problem of poor detection robustness of electronic components in traditional image processing is solved by adopting a target detection technology based on deep learning.
3. The lightweight detection model deployment is realized by utilizing the Mobilene substrate, the detection model can be directly deployed on the AR glasses, and the time delay and data transmission transmitted to the background are avoided.
4. The interactive display and problem recording based on the AR equipment are beneficial to improving the use experience of users; for example, based on the AR glasses, the assembly quality analysis results are displayed, and the welding personnel can be guided to repair the welding personnel conveniently.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The AR-based circuit board welding assembly quality analysis method is characterized by comprising the following steps of:
acquiring a circuit board image based on the AR equipment;
identifying the position and category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment;
according to the identification result of the electronic component, at least acquiring the assembly quality analysis result of the wrong welding and the missing welding of the circuit board based on a pre-constructed assembly template;
and displaying the assembly quality analysis result in a mixed reality space based on the AR equipment.
2. The circuit board solder assembly quality analysis method of claim 1, further comprising: alarming the abnormal assembly quality analysis result; and recording at least the abnormal assembly quality analysis results in a device archive of the server.
3. The method for analyzing the quality of the welded assembly of the circuit board according to claim 1 or 2, wherein the lightweight electronic component detection model adopts an end-to-end YOLO V4 network model, and the feature extraction network of the YOLO V4 network model adopts a lightweight mobilent network.
4. The method for analyzing the quality of the welded assembly of the circuit board according to claim 1 or 2, wherein the process of obtaining the quality analysis result of the misplug and the miss-welding of the circuit board comprises the steps of:
based on the principle of minimizing Euclidean distance, matching the identification result of the electronic component with the electronic component of the assembly template; if the detected label is inconsistent with the assembly template, judging that the label is wrongly welded; if the electronic components exist in the assembly template, but the corresponding labels are not detected, the assembly template is judged to be in a missing welding state.
5. The method for analyzing the quality of the solder joint assembly of the circuit board according to claim 1 or 2, wherein the recognition result of the electronic component is corrected, and the assembly quality analysis result of the solder joint failure and the solder joint missing of the circuit board is obtained based on the corrected recognition result.
6. The method for analyzing the quality of solder assembly of a circuit board according to claim 5, wherein said correcting the recognition result of said electronic component comprises:
acquiring pose information of the circuit board image according to auxiliary positioning marks around the positioning frame; the positioning frame is a region for horizontally placing the circuit board;
according to the pose information, obtaining plane coordinates of each electronic component on the circuit board by affine transformation;
and unifying the shooting angles of the circuit board images according to the plane coordinates of the electronic components.
7. An AR-based circuit board solder assembly quality analysis system, comprising:
the acquisition module is used for acquiring the circuit board image based on the AR equipment;
the identification module is used for identifying the position and the category information of the electronic components on the circuit board image according to a lightweight electronic component detection model deployed on the AR equipment;
the analysis module is used for at least acquiring assembly quality analysis results of the circuit board in the wrong welding and the missing welding based on a pre-constructed assembly template according to the identification result of the electronic component;
and the display module is used for displaying the analysis assembly quality analysis result in the mixed reality space based on the AR equipment.
8. The circuit board solder assembly quality analysis system of claim 7, further comprising:
the alarm module is used for alarming abnormal assembly quality analysis results;
and the recording module is used for recording at least the abnormal assembly quality analysis result in a device archive of the server.
9. A storage medium, characterized in that it stores a computer program for AR-based circuit board solder fitting quality analysis, wherein the computer program causes a computer to execute the circuit board solder fitting quality analysis method according to any one of claims 1 to 6.
10. An electronic device, comprising:
one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the circuit board solder assembly quality analysis method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310171490.0A CN116563831A (en) | 2023-02-22 | 2023-02-22 | AR-based circuit board welding assembly quality analysis method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310171490.0A CN116563831A (en) | 2023-02-22 | 2023-02-22 | AR-based circuit board welding assembly quality analysis method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116563831A true CN116563831A (en) | 2023-08-08 |
Family
ID=87485056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310171490.0A Pending CN116563831A (en) | 2023-02-22 | 2023-02-22 | AR-based circuit board welding assembly quality analysis method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563831A (en) |
-
2023
- 2023-02-22 CN CN202310171490.0A patent/CN116563831A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111768386B (en) | Product defect detection method, device, electronic equipment and storage medium | |
| US10964025B2 (en) | Assembly monitoring method and device based on deep learning, and readable storage medium | |
| CN110378900B (en) | Method, device and system for detecting product defects | |
| CN111693534B (en) | Surface defect detection method, model training method, device, equipment and medium | |
| JP2008235504A (en) | Assembly inspection device | |
| CN101939765A (en) | A computer implemented method and system for remote inspection of an industrial process | |
| JP2023015989A (en) | Item identification and tracking system | |
| CN110049121B (en) | Data center inspection system based on augmented reality technology | |
| JP7311285B2 (en) | Automated coating quality inspection for aircraft | |
| EP3916355A1 (en) | Method and apparatus for indoor localization, electronic device, storage medium and computer program | |
| CN110555838A (en) | Image-based part fault detection method and device | |
| Ziaei et al. | Real-time markerless Augmented Reality for Remote Handling system in bad viewing conditions | |
| CN111770450B (en) | Workshop production monitoring server, mobile terminal and application | |
| CN110796738A (en) | Three-dimensional visualization method and device for tracking state of inspection equipment | |
| CN114782437B (en) | Computer mainboard quality detection method and system based on artificial intelligence | |
| JP2009289046A (en) | Operation support device and method using three-dimensional data | |
| CN113312951B (en) | Dynamic video target tracking system, related method, device and equipment | |
| Wang et al. | Construction Photo Localization in 3D Reality Models for Vision-Based Automated Daily Project Monitoring | |
| CN116563831A (en) | AR-based circuit board welding assembly quality analysis method and system | |
| US20230156161A1 (en) | Failure identification and handling method, and system | |
| CN117218633A (en) | Article detection method, device, equipment and storage medium | |
| CN115908977A (en) | Image data labeling method and device, electronic equipment and storage medium | |
| KR101845423B1 (en) | Apparatus and method for supporting inspection of structure | |
| KR102524151B1 (en) | Labeling learning method of artificial intelligence machine for smart products inspection | |
| CN112802112B (en) | Visual positioning method, device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |