CN113570550A - Gold wire bonding size detection method based on three-dimensional point cloud - Google Patents

Gold wire bonding size detection method based on three-dimensional point cloud Download PDF

Info

Publication number
CN113570550A
CN113570550A CN202110746521.1A CN202110746521A CN113570550A CN 113570550 A CN113570550 A CN 113570550A CN 202110746521 A CN202110746521 A CN 202110746521A CN 113570550 A CN113570550 A CN 113570550A
Authority
CN
China
Prior art keywords
gold wire
point
welding
point cloud
clouds
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110746521.1A
Other languages
Chinese (zh)
Other versions
CN113570550B (en
Inventor
汪俊
隆昆
谢乾
李大伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN202110746521.1A priority Critical patent/CN113570550B/en
Publication of CN113570550A publication Critical patent/CN113570550A/en
Application granted granted Critical
Publication of CN113570550B publication Critical patent/CN113570550B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer

Abstract

The invention discloses a gold wire bonding size detection method based on three-dimensional point cloud, which comprises the following steps: step 1, performing component detection on input integral point clouds based on a point cloud deep learning method, and extracting gold wire point clouds, welding spot point clouds, chip point clouds and bottom plate point clouds according to detection results; step 2, respectively dividing single gold wire point clouds and single welding point clouds from the extracted gold wire point clouds and welding point clouds, and then carrying out gold wire-welding point pairing according to the single gold wire point clouds and welding point clouds; and 3, constructing a gold wire bonding size calculation model, and calculating gold wire bonding size parameters according to the gold wire-welding point pairing combination and the gold wire bonding size calculation model. The gold wire bonding size detection method based on the three-dimensional point cloud aims to solve the problem of automatic detection of gold wire bonding size, is used for automatic detection of gold wire morphology characteristics, and is high in detection precision and simple and convenient to implement.

Description

Gold wire bonding size detection method based on three-dimensional point cloud
Technical Field
The invention belongs to the technical field of gold wire morphology feature detection, and particularly relates to a gold wire bonding size detection method based on three-dimensional point cloud.
Background
With the rapid development of microelectronic industry in China, significant progress is made in the research and integrated manufacturing of microelectronic chips in China. Microelectronic packaging is used as a key technology influencing the performance of microelectronic devices, and more than 90% of later packaging in the manufacturing process of semiconductor integrated circuits adopts a gold wire bonding technology, so the quality of gold wire bonding seriously influences the performance of the microelectronic devices. The integrated circuit is very small in size, so that the appearance and size of the gold wire bonding structure are difficult to detect, and at present, a high-precision camera is mostly adopted to collect pictures of the integrated circuit, and then welding points and the gold wires are positioned through an image processing algorithm for detection. However, the image as two-dimensional data cannot represent height information, and therefore, the gold wire morphology and the arc height cannot be measured.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a gold wire bonding dimension measurement and detection method based on three-dimensional point cloud, and aiming at the problem that gold wire bonding appearance dimension parameters are difficult to measure, the invention applies a three-dimensional laser scanning technology to gold wire bonding appearance measurement, and aims to realize automatic measurement of gold wire bonding dimensions, improve the gold wire bonding dimension measurement efficiency and simultaneously improve the accuracy and precision of gold wire bonding dimension measurement.
In order to achieve the purpose, the invention adopts the following technical scheme:
a gold wire bonding size detection method based on three-dimensional point cloud comprises the following steps:
step 1, performing component detection on input integral point clouds based on a point cloud deep learning method, and extracting gold wire point clouds, welding spot point clouds, chip point clouds and bottom plate point clouds according to detection results;
step 2, respectively dividing single gold wire point clouds and single welding point clouds from the extracted gold wire point clouds and welding point clouds, and then carrying out gold wire-welding point pairing according to the single gold wire point clouds and welding point clouds;
and 3, constructing a gold wire bonding size calculation model, and calculating gold wire bonding size parameters according to the gold wire-welding point pairing combination and the gold wire bonding size calculation model.
Further, step 1 specifically includes:
step 11, constructing a deep learning input data set, selecting at least 3 different pieces of point cloud data which respectively comprise gold wires, welding points, chips and a bottom plate as training data, verification data and test data, dividing the training data, the verification data and the test data into four parts of the gold wires, the welding points, the chips and the bottom plate, labeling corresponding data, and constructing a training set, a verification set and a test set based on the labeled data;
step 12, building a deep learning network framework PointNet + +, wherein the network comprises 3 SA layers, namely a Set Abstraction layer, used for feature extraction, and an FC layer, namely a full connected layer, used for feature classification;
step 13, inputting the training set data into the deep learning network for iterative training, inputting the verification data set into the PointNet + + network to verify the training model after finishing the training of the training set each time, and storing the training model with the best verification effect;
and step 14, inputting the test data into the training model with the best effect to predict the component category of the point cloud to be detected, and dividing the point cloud to be detected into gold wire point cloud, welding point cloud, chip point cloud and bottom plate point cloud according to the component detection classification result.
Further, the step 11 of marking data to construct a training set, a verification set and a test set specifically includes:
1) for the training set, respectively sampling 1024 points in each type of point set as points to be tested by adopting a random sampling method in four types of point sets of a gold wire, a welding point, a chip and a bottom plate; regarding the test set, taking each point in the test point cloud data as a point to be tested;
2) taking a point to be measured as a center and taking R as a radius structure in the original integral point cloudCreating a bounding sphere and obtaining a point cloud P in the bounding sphereRFor the point cloud P surrounding the interior of the ballR512 points are sampled by adopting a farthest point sampling method and are used as input data of the deep learning network PointNet + +.
Further, step 2 specifically includes:
step 21, fitting a plane gamma of the chip by using random sampling consistency, namely RANSAC (random sample consensus) plane fitting algorithm based on the point cloud of the chip detected in the step 1;
step 22, based on the gold wire point clouds and the welding spot point clouds detected in the step 1, adopting an European clustering algorithm to respectively cluster single gold wire point clouds and single welding spot point clouds to obtain a single gold wire point cloud set
Figure BDA0003143157010000021
And single solder joint point cloud
Figure BDA0003143157010000022
Wherein, WiThe point cloud is the ith gold wire point cloud, Sj is the jth welding point cloud, i is the serial number of the gold wire point cloud, and j is the serial number of the welding point cloud;
step 23, projecting the single gold wire point cloud in the single gold wire point cloud set and the single welding point cloud in the single welding point cloud set on the plane gamma to obtain a gold wire projection point cloud set
Figure BDA0003143157010000023
Cloud collection of projection points of welding spots
Figure BDA0003143157010000024
Figure BDA0003143157010000025
Wherein, Wi'is the ith gold wire projection point cloud, S'jJ is the j projection point cloud, k is the total number of gold wire point clouds, and k' is the total number of welding point clouds;
step 24, calculating the mass center s of the projection point cloud of each welding pointjAnd the mass center wi of each gold wire projection point cloud, and performing principal component analysis on each gold wire projection point cloud to calculate each gold wire projection point cloudMain direction of
Figure BDA0003143157010000026
Step 25, using the centroid w of the single gold wire projection pointiAnd a main direction
Figure BDA0003143157010000027
Line l where the projection of the ceramic gold wire is locatediBy calculating the centroid of the spot to the line liThe distance of the gold wire and the welding spot are matched.
Further, step 25 specifically includes:
251, projecting the centroid w with the gold wireiAs a starting point, the projected centroid s of any welding pointjConstructing a direction vector for the endpoint
Figure BDA0003143157010000031
According to the main direction of the wire
Figure BDA0003143157010000032
And the direction vector pointing to the welding point
Figure BDA0003143157010000033
The included angle θ divides all solder joints into two groups: the welding points with the included angle theta of 0 degrees and 90 degrees are divided into a group of welding points with the included angle theta of 90 degrees and 180 degrees]Dividing the welding spots in the range into another group;
step 252, for each group of welding points, calculating the projected centroid s of the single welding pointjLine l where projection to gold wire is locatediAnd d, taking the welding spot corresponding to the minimum distance as the welding spot distributed on one side of the gold wire, respectively finding the welding spot closest to the projection straight line where the gold wire is located in the two groups of welding spots as a matching welding spot of the gold wire to complete gold wire-welding spot matching, wherein one group of gold wire-welding spot matching comprises one gold wire, one welding spot connected with the gold wire on the bottom plate and one welding spot connected with the gold wire on the chip.
Further, step 3 specifically includes:
step 31, gold wire bonding size parameters are defined as gold wire arc height H, welding point height difference H and gold wire span L in a group of gold wire-welding point matching points;
step 32, for a set of gold wire-solder joint matching
Figure BDA0003143157010000034
Calculating point cloud W of gold wireiHighest point w oftpCalculating the point cloud of the welding spots on the chip
Figure BDA0003143157010000035
Lowest point of (2)
Figure BDA0003143157010000036
Calculating the point cloud of welding spots on the bottom plate
Figure BDA0003143157010000037
Lowest point of (2)
Figure BDA0003143157010000038
Step 33, based on the normal vector of the plane Γ where the chip is located
Figure BDA0003143157010000039
With the highest point w of the gold wiretpLowest point of chip pad
Figure BDA00031431570100000310
And the lowest point of the welding spot of the bottom plate
Figure BDA00031431570100000311
Defining the gold wire arc height H as the distance from the highest point of the gold wire to the plane of the chip, defining the height difference H of welding points as the vertical distance between two welding points, and defining the gold wire span L as the horizontal distance between the two welding points:
Figure BDA00031431570100000312
Figure BDA00031431570100000313
Figure BDA00031431570100000314
wherein the content of the first and second substances,
Figure BDA00031431570100000315
for the lowest point cloud of chip welding spots in a gold wire bonding structure
Figure BDA00031431570100000316
Pointing to the highest point w of the gold wiretpThe direction vector of (a) is,
Figure BDA00031431570100000317
is the lowest point of a base plate welding spot point cloud in a gold wire bonding structure
Figure BDA00031431570100000318
Point cloud lowest point of pointing chip
Figure BDA00031431570100000319
Direction vector of
The invention has the beneficial effects that: the method and the device aim at solving the problems that the gold wire bonding morphology is difficult to detect and accurate in determining the height, can realize automatic measurement of the gold wire bonding size, and are high in automation degree, high in measurement precision and simple and convenient to realize.
The method is used for establishing a mathematical model for the butt joint of the aircraft skin based on the three-dimensional point cloud data to calculate the butt joint step difference and the gap so as to judge the assembling quality of the aircraft skin.
The invention establishes a mathematical model for the gold wire bonding structure based on the three-dimensional point cloud data to calculate the arc height, the span and the height difference of the welding points of the gold wire so as to judge the connection quality of the gold wire bonding structure.
Drawings
FIG. 1 is a schematic representation of a gold wire bonding structure in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a circuit board according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a piece of point cloud data of a circuit board according to an embodiment of the invention;
FIG. 4 is a flowchart of a method for detecting bonding size of gold wire based on three-dimensional point cloud according to an embodiment of the present invention;
FIG. 5 is a diagram of a point cloud deep learning network framework in an embodiment of the invention;
fig. 6 is a schematic point cloud diagram of each component of a circuit board detected according to an embodiment of the present invention, in which fig. 6(a) is a schematic point cloud diagram of a chip, fig. 6(b) is a schematic point cloud diagram of a gold wire, fig. 6(c) is a schematic point cloud diagram of a bottom plate, and fig. 6(d) is a schematic point cloud diagram of a solder joint;
FIG. 7 shows the result of gold wire-matching performed according to an embodiment of the present invention;
fig. 8 is a model of gold wire bond size calculation according to an embodiment of the present invention.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
In order to solve the problem that the shape, the size and the parameters of a gold wire bonding structure in the prior art are difficult to measure, the invention provides a gold wire bonding size detection method based on three-dimensional point cloud, fig. 1 shows an optional gold wire bonding model structure, fig. 2 shows a schematic diagram of a local entity of a circuit board in the embodiment of the invention, and fig. 3 shows a schematic diagram of a three-dimensional point cloud structure of the circuit board.
An optional flowchart of the gold wire bonding size detection method based on three-dimensional point cloud is shown in fig. 4, and the method includes:
step 1, performing component detection on input integral point clouds based on a point cloud deep learning method, and extracting gold wire point clouds, welding spot point clouds, chip point clouds and bottom plate point clouds according to detection results;
step 2, respectively dividing single gold wire point clouds and single welding point clouds from the extracted gold wire point clouds and welding point clouds, and then performing gold wire-welding point pairing according to the single gold wire and welding point clouds;
and 3, constructing a gold wire bonding size calculation model, and calculating gold wire bonding size parameters according to the gold wire-welding point pairing combination and the gold wire bonding size calculation model.
By applying the technical scheme of the invention, the automatic measurement of the gold wire bonding structure can be realized. The invention establishes a mathematical model for the gold wire bonding structure to calculate the gold wire bonding appearance size based on the actually measured three-dimensional point cloud data of the circuit board so as to realize the appearance characteristic detection of the gold wire bonding structure and judge whether the gold wire bonding structure meets the design requirements. The detection method is high in automation degree and detection precision. FIG. 3 is a schematic diagram of a point cloud of a circuit board acquired using a three-dimensional scanner.
In a possible embodiment, the method for performing component detection on the whole input point cloud based on the point cloud deep learning method in step 1, and extracting point clouds of a gold wire, a welding point, a chip and a bottom plate according to detection results respectively, includes:
step 11, constructing a deep learning input data set, selecting at least 3 different pieces of point cloud data which respectively comprise four parts of a gold wire, a welding point, a chip and a bottom plate as training data, verification data and test data, dividing the four parts of the gold wire, the welding point, the chip and the bottom plate from the training data, the verification data and the test data, carrying out corresponding data labeling, and constructing the training set, the verification set and the test set based on the labeled data:
1) for the training set and the test set, respectively sampling 1024 points in each type of point set by adopting a random sampling method in the four types of point sets of a gold wire, a welding point, a chip and a bottom plate to be used as points to be tested, and for the test set, taking each point in test data as the points to be tested;
2) constructing a bounding sphere by taking the point to be measured as the center and taking R as the radius in the original point cloud, and acquiring a point cloud P in the bounding sphereRFor the point cloud P surrounding the interior of the ballR512 points are sampled by adopting a farthest point sampling method to serve as input data of the deep learning network, namely the dimension of the data input into the network is 512 x (3+3), wherein 512 represents the number of points, and (3+3) represents coordinate information and color information (xyz + rgb).
Step 12, building a deep learning network framework PointNet + +, wherein the network comprises 3 SA layers, namely a Set Abstraction layer, for feature extraction and one FC layer, namely a full connected layer, for feature classification, as shown in FIG. 5, the input feature dimension of SA1 is 512 x (3+3), the output feature dimension is 512 x (320+3), the output feature dimension of SA1 is 128 x (640+3) after SA2 downsampling and high-dimensional mapping, SA3 continues downsampling and high-dimensional mapping to obtain 1 x 1024-dimensional features, and continues to input into a full connection layer FC for regression to obtain the scores of input data belonging to each category;
step 13, inputting training set data into the deep learning network for iterative training, inputting a verification data set into the network to verify a training model after finishing training of the training set once, storing the training model with the best verification effect, and iterating the whole training process for 100 times and consuming 5.2 hours;
step 14, inputting the test data into the training model with the best verification effect to predict the component category of the point cloud to be tested, and dividing the point cloud to be tested into a gold wire point cloud, a welding point cloud, a chip point cloud and a bottom plate point cloud according to the component detection classification result, wherein the extraction result schematic diagram of each component of the test data is shown in fig. 6, wherein fig. 6(a) is a chip point cloud schematic diagram, fig. 6(b) is a gold wire point cloud schematic diagram, fig. 6(c) is a bottom plate point cloud schematic diagram, and fig. 6(d) is a welding point cloud schematic diagram.
In a possible embodiment, the step 2 of dividing a single gold wire point cloud and a single welding point cloud from the detected gold wire point cloud and welding point cloud respectively, and then performing gold wire-welding point pairing according to the single gold wire and welding point cloud includes:
step 21, fitting a plane gamma of the chip by using random sampling consistency, namely RANSAC (random sample consensus) plane fitting algorithm based on the point cloud of the chip detected in the step 1;
step 22, based on the gold wire point clouds and the welding spot point clouds detected in the step 1, adopting an European clustering algorithm to respectively cluster single gold wire point clouds WiAnd a single spot cloud SiObtaining a single gold wire point cloud set
Figure BDA0003143157010000061
And single solder joint point cloud
Figure BDA0003143157010000062
Step 23, projecting the single gold wire point cloud in the single gold wire point cloud set and the single welding point cloud in the single welding point cloud set on the plane gamma to obtain a gold wire projection point cloud set
Figure BDA0003143157010000063
Cloud collection of projection points of welding spots
Figure BDA0003143157010000064
Step 24, calculating the projection point cloud S of each welding pointj' center of mass sjAnd each gold wire projection point cloud Wi' center of mass wiAnalyzing the main components of each gold wire projection point cloud to calculate the main direction of each gold wire projection point cloud
Figure BDA0003143157010000065
Step 25, using the centroid w of the single gold wire projection pointiAnd a main direction
Figure BDA0003143157010000066
Constructing a line l where the projection of the gold wire is locatediBy calculating the centroid of the spot to the line liPerforming gold wire-solder joint matching:
1) using gold wire to project centroid wiAs a starting point, the projected centroid s of any welding pointjConstructing a direction vector for the endpoint
Figure BDA0003143157010000067
According to the main direction of the wire
Figure BDA0003143157010000068
And the direction vector pointing to the welding point
Figure BDA0003143157010000069
The included angle θ divides all solder joints into two groups: corresponding welding point with included angle theta in the range of 0 DEG and 90 DEGDivided into one group at 90 deg. and 180 deg. °]Dividing the corresponding welding spots in the range into another group;
2) for each set of welding points, calculating the projected centroid s of the welding pointsjLine l where projection to gold wire is locatediThe distance d is the welding point S corresponding to the minimum distanceiAs the welding spots distributed on one side of the gold wire, the welding spots closest to the straight line where the gold wire is located are respectively found in two groups of welding spots as the matching welding spots of the gold wire to complete gold wire-welding spot matching, wherein one group of gold wire-welding spot matching comprises one gold wire, one welding spot connected with the gold wire on the bottom plate and one welding spot connected with the gold wire on the chip, the gold wire-welding spot matching result is shown in figure 7, and a group of gold wires and welding spot point cloud in a dotted line frame represents a gold wire bonding structure.
In one possible embodiment, step 3 is to construct a gold wire bonding size calculation model, and to calculate gold wire bonding size parameters based on the gold wire-pad and gold wire bonding size calculation model, including:
step 31, gold wire bonding size parameters are defined as gold wire arc height H, welding point height difference H, gold wire span L and size parameters in a group of gold wire-welding point matching points as shown in figure 1;
step 32, matching a set of gold wire solder joints
Figure BDA00031431570100000610
Calculating point cloud W of gold wireiHighest point w oftpCalculating the point cloud of the welding spots on the chip
Figure BDA00031431570100000611
Lowest point of (2)
Figure BDA00031431570100000612
Calculating the point cloud of welding spots on the bottom plate
Figure BDA00031431570100000613
Lowest point of (2)
Figure BDA00031431570100000614
The model for calculating the gold wire bonding size is shown in FIG. 8;
step 33, based on the normal vector of the plane Γ where the chip is located
Figure BDA00031431570100000615
With the highest point w of the gold wiretpLowest point of chip pad
Figure BDA00031431570100000616
And the lowest point of the welding spot of the bottom plate
Figure BDA0003143157010000071
Defining the gold wire arc height H as the distance from the highest point of the gold wire to the plane of the chip, defining the height difference H of welding points as the vertical distance between two welding points, and defining the gold wire span L as the horizontal distance between the two welding points:
Figure BDA0003143157010000072
Figure BDA0003143157010000073
Figure BDA0003143157010000074
wherein the content of the first and second substances,
Figure BDA0003143157010000075
for the lowest point cloud of chip welding spots in a gold wire bonding structure
Figure BDA0003143157010000076
Pointing to the highest point w of the gold wiretpThe direction vector of (a) is,
Figure BDA0003143157010000077
is the lowest point of a base plate welding spot point cloud in a gold wire bonding structure
Figure BDA0003143157010000078
Point of directionChip point cloud lowest point
Figure BDA0003143157010000079
The direction vector of (2).
In the data used in this example, measurements of gold wire bond feature dimensions are shown in table 1, where the combined effect is shown in fig. 7:
TABLE 1 gold wire morphology dimension measurement results (unit: micrometer)
Combination of H h L
① 283.669 246.462 225.184
② 320.768 242.242 561.889
③ 333.655 196.641 1295.283
④ 315.946 225.872 1063.518
⑤ 397.504 235.456 1151.535
⑥ 273.577 183.345 515.740
⑦ 321.020 205.005 387.145
⑧ 265.040 207.416 169.946
⑨ 273.556 198.099 352.932
The method for detecting the bonding size of gold wire based on three-dimensional point cloud of the present invention is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof.
It should be noted that the terms "upper", "lower", "left", "right", "front", "back", etc. used in the present invention are for clarity of description only, and are not intended to limit the scope of the present invention, and the relative relationship between the terms and the terms is not limited by the technical contents of the essential changes.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.

Claims (6)

1. A gold wire bonding size detection method based on three-dimensional point cloud is characterized by comprising the following steps:
step 1, performing component detection on input integral point clouds based on a point cloud deep learning method, and extracting gold wire point clouds, welding spot point clouds, chip point clouds and bottom plate point clouds according to detection results;
step 2, respectively dividing single gold wire point clouds and single welding point clouds from the extracted gold wire point clouds and welding point clouds, and then carrying out gold wire-welding point pairing according to the single gold wire point clouds and welding point clouds;
and 3, constructing a gold wire bonding size calculation model, and calculating gold wire bonding size parameters according to the gold wire-welding point pairing combination and the gold wire bonding size calculation model.
2. The method for detecting the bonding size of the gold wire based on the three-dimensional point cloud as claimed in claim 1, wherein the step 1 is specifically as follows:
step 11, constructing a deep learning input data set, selecting at least 3 different pieces of point cloud data which respectively comprise gold wires, welding points, chips and a bottom plate as training data, verification data and test data, dividing the training data, the verification data and the test data into four parts of the gold wires, the welding points, the chips and the bottom plate, labeling corresponding data, and constructing a training set, a verification set and a test set based on the labeled data;
step 12, building a deep learning network framework PointNet + +, wherein the network comprises 3 Set Abstraction layers, namely a Set Abstraction layer, used for feature extraction, and a full connected layer, namely a full connection layer, used for feature classification;
step 13, inputting the training set data into the deep learning network for iterative training, inputting the verification data set into the PointNet + + network to verify the training model after finishing the training of the training set each time, and storing the training model with the best verification effect;
and step 14, inputting the test data into the training model with the best effect to predict the component category of the point cloud to be detected, and dividing the point cloud to be detected into gold wire point cloud, welding point cloud, chip point cloud and bottom plate point cloud according to the component detection classification result.
3. The method as claimed in claim 2, wherein the step 11 of constructing a training set, a verification set and a test set by labeling data specifically comprises:
1) for the training set, respectively sampling 1024 points in each type of point set as points to be tested by adopting a random sampling method in four types of point sets of a gold wire, a welding point, a chip and a bottom plate; regarding the test set, taking each point in the test point cloud data as a point to be tested;
2) constructing a bounding sphere by taking the point to be measured as the center and taking R as the radius in the original integral point cloud, and acquiring a point cloud P in the bounding sphereRFor the point cloud P surrounding the interior of the ballR512 points are sampled by adopting a farthest point sampling method and are used as input data of the deep learning network PointNet + +.
4. The method as claimed in claim 1, wherein the step 2 comprises:
step 21, fitting a plane gamma of the chip by using random sampling consistency, namely RANSAC (random sample consensus) plane fitting algorithm based on the point cloud of the chip detected in the step 1;
step 22, based on the gold wire point clouds and the welding spot point clouds detected in the step 1, adopting an European clustering algorithm to respectively cluster single gold wire point clouds and single welding spot point clouds to obtain a single gold wire point cloud set
Figure FDA0003143150000000021
And single solder joint point cloud
Figure FDA0003143150000000022
Wherein, WiAs the ith point cloud of gold wire, SjIs the jth welding point cloud, i is the serial number of the gold wire cloud, j is the serial number of the welding point cloud;
step 23, projecting the single gold wire point cloud in the single gold wire point cloud set and the single welding point cloud in the single welding point cloud set on the plane gamma to obtain a gold wire projection point cloud set
Figure FDA0003143150000000023
Cloud collection of projection points of welding spots
Figure FDA0003143150000000024
Figure FDA0003143150000000025
Wherein, W'iProjecting point cloud, S 'for the ith gold wire'jJ is the j projection point cloud, k is the total number of gold wire point clouds, and k' is the total number of welding point clouds;
step 24, calculating the mass center s of the projection point cloud of each welding pointjAnd the centroid w of each gold wire projection point cloudiAnalyzing the main components of each gold wire projection point cloud to calculate the main direction of each gold wire projection point cloud
Figure FDA0003143150000000026
Step 25, using the centroid w of the single gold wire projection pointiAnd a main direction
Figure FDA0003143150000000027
Constructing a line l where the projection of the gold wire is locatediBy calculating the centroid of the spot to the line liThe distance of the gold wire and the welding spot are matched.
5. The method as claimed in claim 4, wherein the step 25 specifically comprises:
251, projecting the centroid w with the gold wireiAs a starting point, the projected centroid s of any welding pointjConstructing a direction vector for the endpoint
Figure FDA0003143150000000028
According to the main direction of the wire
Figure FDA0003143150000000029
And the direction vector pointing to the welding point
Figure FDA00031431500000000210
The included angle θ divides all solder joints into two groups: the welding points with the included angle theta of 0 degrees and 90 degrees are divided into a group of welding points with the included angle theta of 90 degrees and 180 degrees]Dividing the welding spots in the range into another group;
step 252, for each group of welding points, calculating the projected centroid s of the single welding pointjLine l where projection to gold wire is locatediAnd d, taking the welding spot corresponding to the minimum distance as the welding spot distributed on one side of the gold wire, respectively finding the welding spot closest to the projection straight line where the gold wire is located in the two groups of welding spots as a matching welding spot of the gold wire to complete gold wire-welding spot matching, wherein one group of gold wire-welding spot matching comprises one gold wire, one welding spot connected with the gold wire on the bottom plate and one welding spot connected with the gold wire on the chip.
6. The method as claimed in claim 1, wherein the step 3 specifically comprises:
step 31, gold wire bonding size parameters are defined as gold wire arc height H, welding point height difference H and gold wire span L in a group of gold wire-welding point matching points;
step 32, for a set of gold wire-solder joint matching
Figure FDA00031431500000000211
Calculating point cloud W of gold wireiHighest point w oftpCalculating the point cloud of the welding spots on the chip
Figure FDA0003143150000000031
Lowest point of (2)
Figure FDA0003143150000000032
Calculating the point cloud of welding spots on the bottom plate
Figure FDA0003143150000000033
Lowest point of (2)
Figure FDA0003143150000000034
Step 33, based on the normal vector of the plane Γ where the chip is located
Figure FDA0003143150000000035
With the highest point w of the gold wiretpLowest point of chip pad
Figure FDA0003143150000000036
And the lowest point of the welding spot of the bottom plate
Figure FDA0003143150000000037
Defining the gold wire arc height H as the distance from the highest point of the gold wire to the plane of the chip, defining the height difference H of welding points as the vertical distance between two welding points, and defining the gold wire span L as the horizontal distance between the two welding points:
Figure FDA0003143150000000038
Figure FDA0003143150000000039
Figure FDA00031431500000000310
wherein the content of the first and second substances,
Figure FDA00031431500000000311
for the lowest point cloud of chip welding spots in a gold wire bonding structure
Figure FDA00031431500000000312
Pointing to the highest point w of the gold wiretpThe direction vector of (a) is,
Figure FDA00031431500000000313
is the lowest point of a base plate welding spot point cloud in a gold wire bonding structure
Figure FDA00031431500000000314
Point cloud lowest point of pointing chip
Figure FDA00031431500000000315
The direction vector of (2).
CN202110746521.1A 2021-07-01 2021-07-01 Gold wire bonding size detection method based on three-dimensional point cloud Active CN113570550B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110746521.1A CN113570550B (en) 2021-07-01 2021-07-01 Gold wire bonding size detection method based on three-dimensional point cloud

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110746521.1A CN113570550B (en) 2021-07-01 2021-07-01 Gold wire bonding size detection method based on three-dimensional point cloud

Publications (2)

Publication Number Publication Date
CN113570550A true CN113570550A (en) 2021-10-29
CN113570550B CN113570550B (en) 2022-04-01

Family

ID=78163364

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110746521.1A Active CN113570550B (en) 2021-07-01 2021-07-01 Gold wire bonding size detection method based on three-dimensional point cloud

Country Status (1)

Country Link
CN (1) CN113570550B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114170176A (en) * 2021-12-02 2022-03-11 南昌大学 Automatic detection method for steel grating welding seam based on point cloud
CN114627177A (en) * 2022-03-25 2022-06-14 南京航空航天大学 Aircraft skin gap and step difference measuring method based on image segmentation

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101339656A (en) * 2008-08-29 2009-01-07 中国电子科技集团公司第四十五研究所 Image processing process of full-automatic lead wire bonding machine image processing system
US20100094609A1 (en) * 2007-10-25 2010-04-15 Ki Jin Han Modeling electrical interconnections in three-dimensional structures
CN101813638A (en) * 2010-04-30 2010-08-25 华中科技大学 Method and device for detecting welding point defect of chip on line
US8674470B1 (en) * 2012-12-22 2014-03-18 Monolithic 3D Inc. Semiconductor device and structure
CN107590350A (en) * 2017-10-11 2018-01-16 北京航空航天大学 A kind of high density bonding wire impact touching methods of risk assessment based on the design of field mouth
CN108763703A (en) * 2018-05-18 2018-11-06 上海交通大学 A kind of bonding line impedance matching methods based on stepped impedance transmission line
US20190064364A1 (en) * 2016-01-29 2019-02-28 Motion Engine, Inc. METHODS AND SYSTEMS FOR MOTION DETERMINATION OF SENSOR ELEMENTS IN SENSOR SYSTEMS USING MEMS IMUs
CN110400285A (en) * 2019-05-30 2019-11-01 哈尔滨工业大学（深圳） Wire bonding pad defect location and classification method
CN111680542A (en) * 2020-04-17 2020-09-18 东南大学 Steel coil point cloud identification and classification method based on multi-scale feature extraction and Pointernet neural network
CN112233110A (en) * 2020-11-18 2021-01-15 苏州斯玛维科技有限公司 Method and device for cutting chip bonding wire
CN112816493A (en) * 2020-05-15 2021-05-18 奕目(上海)科技有限公司 Chip routing defect detection method and device

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100094609A1 (en) * 2007-10-25 2010-04-15 Ki Jin Han Modeling electrical interconnections in three-dimensional structures
CN101339656A (en) * 2008-08-29 2009-01-07 中国电子科技集团公司第四十五研究所 Image processing process of full-automatic lead wire bonding machine image processing system
CN101813638A (en) * 2010-04-30 2010-08-25 华中科技大学 Method and device for detecting welding point defect of chip on line
US8674470B1 (en) * 2012-12-22 2014-03-18 Monolithic 3D Inc. Semiconductor device and structure
US20190064364A1 (en) * 2016-01-29 2019-02-28 Motion Engine, Inc. METHODS AND SYSTEMS FOR MOTION DETERMINATION OF SENSOR ELEMENTS IN SENSOR SYSTEMS USING MEMS IMUs
CN107590350A (en) * 2017-10-11 2018-01-16 北京航空航天大学 A kind of high density bonding wire impact touching methods of risk assessment based on the design of field mouth
CN108763703A (en) * 2018-05-18 2018-11-06 上海交通大学 A kind of bonding line impedance matching methods based on stepped impedance transmission line
CN110400285A (en) * 2019-05-30 2019-11-01 哈尔滨工业大学（深圳） Wire bonding pad defect location and classification method
CN111680542A (en) * 2020-04-17 2020-09-18 东南大学 Steel coil point cloud identification and classification method based on multi-scale feature extraction and Pointernet neural network
CN112816493A (en) * 2020-05-15 2021-05-18 奕目(上海)科技有限公司 Chip routing defect detection method and device
CN112233110A (en) * 2020-11-18 2021-01-15 苏州斯玛维科技有限公司 Method and device for cutting chip bonding wire

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DEZHI SU 等: "Study of capillary tip states on the reliability of Al wire bonding in microelectronic package", 《2018 19TH INTERNATIONAL CONFERENCE ON ELECTRONIC PACKAGING TECHNOLOGY (ICEPT)》 *
郑佳美: "基于深度学习的焊缝缺陷检测方法研究", 《中国优秀硕士学位论文全文数据库 (工程科技Ⅰ辑)》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114170176A (en) * 2021-12-02 2022-03-11 南昌大学 Automatic detection method for steel grating welding seam based on point cloud
CN114170176B (en) * 2021-12-02 2024-04-02 南昌大学 Automatic detection method for welding seam of steel grating based on point cloud
CN114627177A (en) * 2022-03-25 2022-06-14 南京航空航天大学 Aircraft skin gap and step difference measuring method based on image segmentation

Also Published As

Publication number Publication date
CN113570550B (en) 2022-04-01

Similar Documents

Publication Publication Date Title
CN113570550B (en) Gold wire bonding size detection method based on three-dimensional point cloud
CN110017773B (en) Package volume measuring method based on machine vision
CN109506580B (en) Spot-facing quality determining method based on line laser 3-D scanning
CN104897062B (en) Visual measurement method and device for shape and position deviation of part non-coplanar parallel holes
CN101424520B (en) Method for detecting partial contour outline of object curved surface
CN100419378C (en) Apparatus and method for testing aspherical surface shape error of optical lens
CN104392950A (en) Positioning method of chip welding points
CN109238084B (en) Automatic guiding method for micro round hole measurement
CN109978938A (en) A kind of pillow spring detection method based on machine vision
CN107084671A (en) A kind of recessed bulb diameter measuring system and measuring method based on three wire configuration light
CN113052797B (en) BGA solder ball three-dimensional detection method based on depth image processing
JP2019197018A (en) Flatness detection method, flatness detection device and flatness detection program
CN113865570A (en) Steel structure circular stand column verticality measuring method
CN104406537B (en) The method of measurement part pit micrometer level depth
CN113781434A (en) Defect detection method and device, intelligent terminal and computer readable storage medium
CN116399241B (en) Patch type inductance geometric parameter measurement method and system
CN106323587B (en) The monocular video high-precision measuring method of wing model in wind tunnel flexible deformation
CN107063330A (en) Porous plate standard and joint error detection method for multi-sensor measurement system
CN109781004A (en) A kind of external screw thread multi-parameter assessment method based on machine vision and virtual instrument
CN115641326A (en) Sub-pixel size detection method and system for ceramic antenna PIN needle image
CN112434541A (en) Two-dimensional code arrangement method and device for measuring crack width
TWI735330B (en) Sphere height measurement system and method
CN114485468B (en) Multi-axis linkage composite measurement system and micro-part full-contour automatic measurement method
CN117308821B (en) Building decoration modeling precision inspection method and system based on scanner
CN117450883B (en) Quick measuring method for vertical probe

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant