CN117884786A - Solder ball laser welding defect detection method - Google Patents
Solder ball laser welding defect detection method Download PDFInfo
- Publication number
- CN117884786A CN117884786A CN202410294483.4A CN202410294483A CN117884786A CN 117884786 A CN117884786 A CN 117884786A CN 202410294483 A CN202410294483 A CN 202410294483A CN 117884786 A CN117884786 A CN 117884786A
- Authority
- CN
- China
- Prior art keywords
- solder ball
- air pressure
- image
- convolution
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 229910000679 solder Inorganic materials 0.000 title claims abstract description 77
- 238000003466 welding Methods 0.000 title claims abstract description 60
- 230000007547 defect Effects 0.000 title claims abstract description 51
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- ATJFFYVFTNAWJD-UHFFFAOYSA-N Tin Chemical compound [Sn] ATJFFYVFTNAWJD-UHFFFAOYSA-N 0.000 claims abstract description 41
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 39
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000011176 pooling Methods 0.000 claims description 33
- 238000005070 sampling Methods 0.000 claims description 23
- 238000012544 monitoring process Methods 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 11
- 239000002131 composite material Substances 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 10
- 238000002474 experimental method Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000010200 validation analysis Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 abstract description 8
- 238000013528 artificial neural network Methods 0.000 abstract description 3
- 230000004927 fusion Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 5
- 239000010931 gold Substances 0.000 description 5
- 229910052737 gold Inorganic materials 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 229910001128 Sn alloy Inorganic materials 0.000 description 3
- 239000007789 gas Substances 0.000 description 3
- 238000009776 industrial production Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000007796 conventional method Methods 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000005507 spraying Methods 0.000 description 2
- 241000489455 Sitta europaea Species 0.000 description 1
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000002844 melting Methods 0.000 description 1
- 230000008018 melting Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000001301 oxygen Substances 0.000 description 1
- 229910052760 oxygen Inorganic materials 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Landscapes
- Electric Connection Of Electric Components To Printed Circuits (AREA)
Abstract
The invention relates to the technical field of laser welding, in particular to a tin ball laser welding defect detection method. The detection method provided by the invention skillfully fuses the laser head air pressure, the solder ball temperature and solder ball appearance information in the welding process, especially considers the influence of the laser head air pressure and the solder ball temperature, considers the change of the three on the time scale, and can realize defect detection based on a simple convolutional neural network; the method has the advantages that the three information fusion judgment of laser head air pressure, solder ball temperature and solder ball appearance are utilized, the detection precision of solder ball welding defects is improved, and the problems that the internal defects of solder ball welding are difficult to detect, the time information cannot be considered by a simple neural network, and the detection precision is low are solved.
Description
Technical Field
The invention relates to the technical field of laser welding, in particular to a tin ball laser welding defect detection method.
Background
Modern large integrated circuits typically employ tin alloys as the primary material for solder joints and conductive nodes. After the tin alloy reaches above the melting point, the tin alloy is easy to react with oxygen and evaporate impurities in the laser ball-spraying welding process, so that the defects of air holes, warping of a gold plate and the like are generated in the tin ball. Since these defects are generated inside the solder balls, the detection difficulty is high, and a great deal of labor and cost are often required.
The traditional manual post-welding detection method has low efficiency, high cost and low precision, can not meet the actual industrial production requirement, and needs to be researched and separated from manual quick detection method.
In recent years, a learner proposes to collect the shape of a solder ball welded by laser in real time based on a visual sensor, and diagnose whether defects are generated in the solder ball through a convolutional neural network model according to characteristic changes of the shape of the solder ball. Although the method can get rid of manpower and realize rapid detection of welding defects, the accuracy is limited and further improvement is needed. In addition, defects such as air holes in the solder balls and warping of the gold plate are closely related to the air pressure of the laser head, and the existing defect detection method based on visual information ignores the air pressure information of the laser head in the welding process, so that higher detection precision cannot be realized. However, it is difficult for a general neural network to process air pressure and visual image information at the same time, and it is difficult to further improve the accuracy of detecting weld defects.
Disclosure of Invention
In order to overcome the problems in the prior art, the invention provides a tin ball laser welding defect detection method.
The technical scheme for solving the technical problems is as follows:
a tin ball laser welding defect detection method comprises the following steps:
step 100, acquiring monitoring signal data in the welding process, wherein the monitoring signal data comprise laser head air pressure data, solder ball temperature data and solder ball appearance images;
step 200, converting the acquired laser head air pressure data into a laser head air pressure two-dimensional graph and converting the tin ball temperature data into a tin ball temperature two-dimensional graph; arranging tin ball shape images according to a time sequence, sampling at fixed intervals to obtain 8 tin ball shape time sequence sampling images; splicing the laser head air pressure two-dimensional graph, the solder ball temperature two-dimensional graph and the first solder ball appearance time sequence sampling image in the channel direction to obtain a composite image;
step 300, presetting a convolutional neural network model, wherein the convolutional neural network model comprises an input layer, 8 convolutional units, a full-connection layer and an output layer which are sequentially connected, and the convolutional units comprise a convolutional layer and a pooling layer; inputting the composite image into a first convolution unit to obtain an image after the first convolution, splicing a second tin ball shape time sequence sampling image with the image after the first convolution after passing through a feature extraction module, inputting the image into the second convolution unit, splicing an 8 th tin ball shape time sequence sampling image with the image after the 7 convolutions after passing through the feature extraction module in sequence, inputting the image into an 8 th convolution unit, converting the data volume passing through the 8 th convolution unit into a one-dimensional vector, inputting the one-dimensional vector into a full connection layer, obtaining a vector after passing through the full connection layer, and outputting the vector through an output layer;
step 400, obtaining a plurality of groups of monitoring signal data and corresponding defect conditions thereof through a pre-experiment to form a database; training a preset convolutional neural network model by utilizing a database, learning a mapping relation between monitoring signal data and welding defects to obtain a trained convolutional neural network model, and identifying whether the welding defects are generated by solder ball laser welding by utilizing the trained convolutional neural network model.
Further, in step 200, the air pressure range on the curve of the air pressure data of the laser head over time is converted into the interval of [0, 255], and then the continuous air pressure values are converted into the two-dimensional image of air pressure of the laser head in a rectangular close-packed manner.
Further, in the step 200, the temperature range on the curve of the temperature data of the solder ball changing with time is converted into the interval of [0, 255], and then the continuous temperature values are converted into the two-dimensional image of the temperature of the solder ball according to the rectangular close-packed mode.
Further, in the step 300, the convolution kernel size of the convolution layer is 3×3, and the step size is 1×1.
Further, in the step 300, the pooling layer adopts a maximum pooling operation, and the convolution kernels of the first pooling layer and the second pooling layer are 2×2, and the step size is 1×1; the convolution kernels of the third to eighth pooling layers are 2×2, and the step size is 2×2.
Further, in the step 400, the database is used for the following steps: 2: the scale of 1 is divided into a training set, a validation set and a test set.
Further, in the step 400, a random gradient descent method is adopted to perform iterative training on a preset convolutional neural network model.
Further, in the step 400, the convolutional neural network model is trained, and the loss function adopts a cross entropy function.
Compared with the prior art, the invention has the following technical effects:
according to the invention, on one hand, the air pressure information and the temperature information are unfolded on a time scale and woven into a two-dimensional image, so that the convolutional neural network model has the capability of processing the air pressure information and the temperature information; on the other hand, the method and the device for the convolution of the solder ball images periodically sample solder ball images in a period of time and then superimpose the solder ball images on the channel dimension of the corresponding convolution image, so that time information is concentrated, and the conventional convolution neural network model can extract the time change characteristics of the solder ball. Compared with the conventional method, the method has the advantages that the detection precision of the welding defects is obviously improved, the difficulty of manual inspection after welding can be effectively reduced, the efficiency of industrial production is improved, and the reject ratio of products is reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a laser welding monitoring apparatus employed in the present invention;
FIG. 2 is a schematic diagram of a signal processing process employing the present invention;
FIG. 3 is a schematic diagram of a process for performing defect detection on a convolutional neural network model of the present invention;
FIG. 4 is a schematic diagram of defect detection using a convolutional neural network model conventional in the art;
FIG. 5 is a graph comparing the effect of defect detection with a conventional convolutional neural network model using the present invention.
In the drawings, the list of component names indicated by the respective reference numerals is as follows:
1. an electronic barometer; 2. an infrared thermometer; 3. an industrial camera; 4. solder balls; 5. and a laser focusing device.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention to achieve the preset purpose, the following detailed description is given below of the specific implementation, structure, features and effects of the technical solution according to the present invention with reference to the accompanying drawings and preferred embodiments. The particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The embodiment of the invention provides a tin ball laser welding defect detection method, which can realize high-precision identification of defects such as air holes in a tin ball, warping of a gold plate and the like without damage, and specifically comprises the following steps:
step 100, acquiring monitoring signal data in the welding process, wherein the monitoring signal data comprise laser head air pressure data, solder ball temperature data and solder ball appearance images;
step 200, converting the acquired laser head air pressure data into a laser head air pressure two-dimensional graph and converting the tin ball temperature data into a tin ball temperature two-dimensional graph; arranging tin ball shape images according to a time sequence, sampling at fixed intervals to obtain 8 tin ball shape time sequence sampling images; splicing the laser head air pressure two-dimensional graph, the solder ball temperature two-dimensional graph and the first solder ball appearance time sequence sampling image in the channel direction to obtain a composite image;
step 300, presetting a convolutional neural network model, wherein the convolutional neural network model comprises an input layer, 8 convolutional units, a full-connection layer and an output layer which are sequentially connected, and the convolutional units comprise a convolutional layer and a pooling layer; inputting the composite image into a first convolution unit to obtain an image after the first convolution, splicing a second tin ball shape time sequence sampling image with the image after the first convolution after passing through a feature extraction module, inputting the image into the second convolution unit, splicing an 8 th tin ball shape time sequence sampling image with the image after the 7 convolutions after passing through the feature extraction module in sequence, inputting the image into an 8 th convolution unit, converting the data volume passing through the 8 th convolution unit into a one-dimensional vector, inputting the one-dimensional vector into a full connection layer, obtaining a vector after passing through the full connection layer, and outputting the vector through an output layer;
step 400, obtaining a plurality of groups of monitoring signal data and corresponding defect conditions thereof through a pre-experiment to form a database; training a preset convolutional neural network model by utilizing a database, learning a mapping relation between monitoring signal data and welding defects to obtain a trained convolutional neural network model, and identifying whether the welding defects are generated by solder ball laser welding by utilizing the trained convolutional neural network model.
The following detailed development of each step is performed:
step 100, monitoring signal data in the welding process are obtained, wherein the monitoring signal data comprise laser head air pressure data, solder ball temperature data and solder ball appearance images.
Fig. 1 is a schematic diagram of a laser welding monitoring device adopted by the invention, wherein the laser welding monitoring device comprises an electronic barometer 1, an infrared thermometer 2 and an industrial camera 3, and the electronic barometer 1, the infrared thermometer 2 and the industrial camera 3 are utilized to acquire laser head air pressure, solder ball temperature and solder ball 4 morphology in a period of time in a welding process. The electronic barometer 1 and the infrared thermometer 2 are installed above the laser focusing device 5, and the industrial camera 3 is installed on the same plane as the welding platform, wherein the laser focusing device 5 is implemented by adopting the prior art, and details are omitted here. As shown in fig. 1, data of air pressure change of a laser head are collected into an electronic barometer 1, data of temperature change of a solder ball are collected into an infrared thermometer 2, and reflected light on the surface of the solder ball is sent into an industrial camera 3 horizontally installed with a welding platform.
The electronic barometer 1, the infrared thermometer 2 and the industrial camera 3 are connected to a computer through a TPC/IP protocol, and synchronous trigger signals are sent to the electronic barometer 1, the infrared thermometer 2 and the industrial camera 3 through the computer to start to collect air pressure, temperature and tin ball appearance information until welding is finished. And transmitting the acquired air pressure data, temperature data and solder ball image data to a computer through a data acquisition card, and waiting for subsequent processing.
Specifically, the measurement of the air pressure of the laser head is realized by adopting a MEOKON MD-S821 type electronic barometer 1, and the response speed of the sensor is set to be 0.2ms, so that the actual acquisition speed of air pressure data is 5000Hz; the measurement of the temperature of the solder ball is realized by adopting a Kleiber KMGA740 type high-speed infrared thermometer 2, and the response speed of the sensor is set to be 0.2 and ms, so that the actual acquisition speed of temperature data is 5000Hz; the shooting of the solder ball morphology is realized by an imaging source 23GM021 type industrial camera 3, the trigger time of the camera shutter is set to be 1ms, and the actual acquisition speed of the solder ball image is 1000 Hz.
Step 200, converting the acquired laser head air pressure data into a laser head air pressure two-dimensional graph and converting the tin ball temperature data into a tin ball temperature two-dimensional graph; arranging tin ball shape images according to a time sequence, sampling at fixed intervals to obtain 8 tin ball shape time sequence sampling images; and splicing the laser head air pressure two-dimensional graph, the solder ball temperature two-dimensional graph and the first solder ball appearance time sequence sampling image in the channel direction to obtain a composite image.
The step 200 specifically includes:
and 210, preprocessing the acquired laser head air pressure data to obtain a laser head air pressure two-dimensional map.
The air pressure range on the curve of the air pressure of the laser head along with the time change is converted into the interval of [0, 255], and then the continuous air pressure values are converted into the two-dimensional image of the air pressure of the laser head in a rectangular close-packed mode.
In a specific embodiment, step 210 includes the steps of:
step 2101, dividing the collected air pressure data in units of milliseconds, and collecting 1000 air pressure data in 200 millisecondsAccording to the acquisition time sequence, then generating 9 extra data points between two adjacent air pressure points by means of linear interpolation, and for the known two adjacent air pressure data points A #x 0 ，y 0 ) And B is%x 1 ，y 1 ) Interpolation air pressure data P #x，y) The method comprises the following steps:
；
in the above-mentioned method, the step of,xthe abscissa of the interpolated barometric pressure data is represented,yan ordinate representing the interpolated barometric pressure data;x0 represents the abscissa of the air pressure data point a,y0 represents the ordinate of the barometric pressure data point a;x1 represents the abscissa of the barometric pressure data point B,y1 represents the ordinate of the barometric pressure data point B.
In this example, the interpolated air pressure was 1.9Bar when the air pressure data measured in 0.2ms is 1.8Bar and 2Bar, respectively. The above interpolation process is repeated 9 times every 0.2ms, so that every 1ms the measured air pressure data points are expanded from 5 to 50, and then all air pressure data points are expanded from 1000 to 10000. In this example, the interpolation process described above was continued, interpolation was continued between 1.8Bar and 2.0Bar, and 2.0Bar and 2.5Bar, and the process was extrapolated until 50 barometric pressure data points were obtained every 1 ms.
Step 2102. Converting the 10000 barometric pressure data to [0, 255]]Within (2), if the maximum air pressure is P max And a minimum value of P min For any air pressure value P, the conversion method is as follows:
；
in the above formula, P represents any air pressure value, pmax represents an air pressure maximum value, and Pmin represents an air pressure minimum value.
According to this conversion, the lowest air pressure in the 10000 air pressure data becomes 0 and the highest air pressure becomes 255. The lowest gas pressure measured during the laser welding process in this example was 1.6 and the highest gas pressure was 3.0, and the data point at which the gas pressure was measured at a certain time at 2.5 ℃ was at a value of 164 after conversion.
And 220, preprocessing the obtained solder ball temperature data to obtain a solder ball temperature two-dimensional graph.
The temperature range on the curve of the temperature change of the solder ball along with time is converted into the interval of [0, 255], and then the continuous temperature values are converted into a two-dimensional image format according to a rectangular close-packed mode.
In a specific embodiment, step 220 includes the steps of:
step 2201, dividing the acquired solder ball temperature data by taking millisecond as a unit, arranging 1000 solder ball temperature data acquired within 200 milliseconds according to acquisition time, generating 9 extra data points between two adjacent temperature points in a linear interpolation mode, and obtaining a known adjacent two temperature data point Cx 2 ，y 2 ) And D is%x 3 ，y 3 ) Interpolation temperature data P #x t ，y t ) The method comprises the following steps:
；
in the above-mentioned method, the step of,xt represents the abscissa of the interpolated temperature data,yt represents the ordinate of the interpolated temperature data;x2 represents the abscissa of the temperature data point C,y2 represents the ordinate of the temperature data point C;x3 represents the abscissa of the temperature data point D,y3 represents the ordinate of the temperature data point D.
In this example, the interpolation temperature is 298K when the measured temperature data in the continuous 0.2ms are 296K (kelvin) and 300K, respectively. The above interpolation process is repeated 9 times every 0.2ms, so that every 1ms the measured temperature data points are extended from 5 to 50, and then all the temperature data points are extended from 1000 to 10000. In this example, the interpolation process described above is continued, interpolation is continued between 296K and 300K, 298K and 305K, and the process is extrapolated until 50 temperature data points are obtained every 1 ms.
Step 2202. Transform the 10000 temperature data to [0, 255]]Within (2), if the maximum and minimum values of the temperature are T respectively max And T min For any temperatureThe degree value T is converted by the following steps:
；
in the above formula, T represents any temperature value; tmax represents a temperature maximum value; tmin represents a temperature minimum.
According to this conversion mode, the lowest temperature of the 10000 temperature data becomes 0K, and the highest temperature becomes 255K. The lowest temperature measured during laser welding in this example is 296K and the highest temperature is 550K, then at some point the data point at which the temperature measured is 450K has a transformed value of 155K.
Step 2203, arranging 10000 transformed temperature data into a two-dimensional matrix according to a transverse-longitudinal mode, arranging 100 data in each row in the two-dimensional matrix to obtain a two-dimensional matrix for recording temperature information, wherein the size is 100100, stored in a JPG format in a computer.
Step 230, arranging the solder ball morphology images according to time sequence, sampling at fixed intervals to obtain 8 solder ball morphology time sequence sampling images.
For the 211 Zhang Xi spherical morphology images acquired in 210 ms, the 1 st slice was taken out and the rest was evenly divided into 7 aliquots of 30 images each. Taking out and storing 31, 61, 91 and … 211 images, and cutting and sampling the size of each image to 100100 pixels, resulting in 8 sheets of 1 +.>100/>100, a solder ball timing image.
Step 300, presetting a convolutional neural network model, wherein the convolutional neural network model comprises an input layer, 8 convolutional units, a full-connection layer and an output layer which are sequentially connected, and the convolutional units comprise a convolutional layer and a pooling layer; the method comprises the steps of inputting a composite image into a first convolution unit to obtain an image after first convolution, splicing a second tin ball shape time sequence sampling image with the image after first convolution after passing through a feature extraction module, inputting the image into the second convolution unit, splicing an 8 th tin ball shape time sequence sampling image with the image after 7 convolutions after passing through the feature extraction module, inputting the image into an 8 th convolution unit, converting the data volume passing through the 8 th convolution unit into a one-dimensional vector, inputting the one-dimensional vector into a full connection layer, obtaining a vector after passing through the full connection layer, and outputting the vector through a softmax function of an output layer.
The convolutional neural network model structure used in this example is shown in fig. 3, and the convolutional neural network model includes: the first convolution layer, the first maximum pooling layer, the second convolution layer, the second maximum pooling layer, the third convolution layer, the third maximum pooling layer, the fourth convolution layer, the fourth maximum pooling layer, the fifth convolution layer, the fifth maximum pooling layer, the sixth convolution layer, the sixth maximum pooling layer, the seventh convolution layer, the seventh maximum pooling layer, the eighth convolution layer, the eighth maximum pooling layer, the first fully connected layer, the second fully connected layer, the third fully connected layer and the output layer are sequentially connected; the data volume after the eighth maximum pooling layer is converted into a one-dimensional vector, the one-dimensional vector is input into the full-connection layer, and the vector obtained after the full-connection layer is output through the output layer softmax function.
The convolutional neural network model structure is shown in table 1 below. The convolution kernel size of the first convolution layer is 3×3, the step size is 1×1, the convolution kernel of the first maximum pooling layer is 2×2, and the step size is 1×1; the convolution kernel size of the second convolution layer is 3×3, the step size is 1×1, the convolution kernel of the second maximum pooling layer is 2×2, and the step size is 1×1; the convolution kernel size of the third convolution layer is 3×3, the step size is 1×1, the convolution kernel of the third maximum pooling layer is 2×2, and the step size is 2×2; the convolution kernel size of the fourth convolution layer is 3×3, the step length is 1×1, the convolution kernel of the fourth maximum pooling layer is 2×2, and the step length is 2×2; the convolution kernel size of the fifth convolution layer is 3×3, the step size is 1×1, the convolution kernel of the fifth maximum pooling layer is 2×2, and the step size is 2×2; the convolution kernel size of the sixth convolution layer is 3×3, the step size is 1×1, the convolution kernel of the sixth maximum pooling layer is 2×2, and the step size is 2×2; the convolution kernel size of the seventh convolution layer is 3×3, the step size is 1×1, the convolution kernel of the seventh maximum pooling layer is 2×2, and the step size is 2×2; the convolution kernel size of the eighth convolution layer is 3×3, the step size is 1×1, the convolution kernel of the eighth max-pooling layer is 2×2, and the step size is 2×2.
TABLE 1 convolutional neural network model structure
In a specific embodiment, referring to fig. 2, the foregoing two-dimensional map of air pressure of the laser head, two-dimensional map of solder ball temperature, and first time-series image of solder ball are spliced again in the channel dimension to obtain a piece of solder ball with a size of 3100/>100, inputting the composite image into a first convolution layer and a first maximum pooling layer, splicing the 31 st image with the image convolved by the first convolution layer through a feature extraction module, inputting the composite image into a second convolution layer and a second maximum pooling layer, splicing the image at intervals of 30ms with the corresponding convolved image through the feature extraction module, and finally completing the convolution step. The data quantity after the convolution step is converted into a one-dimensional vector, the one-dimensional vector is input into a full-connection layer, and the vector obtained after the full-connection layer is output through the softmax function of an output layer. The feature extraction module is specifically a resize function for extracting 1 +.>100/>100, adding a subsequent convolution step after the image size is adjusted; for example, the image output size after convolution of the first convolution layer is 3 + ->99/>99, image of 31 st sheet is adjusted to 1 +.>99/>99。
Step 400, obtaining a plurality of groups of monitoring signal data and corresponding defect conditions thereof through a pre-experiment to form a database; training a preset convolutional neural network model by utilizing a database, learning a mapping relation between monitoring signal data and welding defects to obtain a trained convolutional neural network model, and identifying whether the welding defects are generated by solder ball laser welding by utilizing the trained convolutional neural network model. The method specifically comprises the following steps:
the method comprises the steps of collecting air pressure and tin ball appearance data of random tin ball laser welding through a pre-experiment, changing parameters such as laser power, welding speed and the like in the pre-experiment, cutting the tin ball after welding, and observing whether the tin ball generates defects such as air holes and gold plate warpage or not in a manual detection mode to obtain welding spots with defects and defects in the inside.
Establishing a database for monitoring data and welding defects, ensuring that the total group number of data acquisition is more than 5000 groups, and simultaneously enabling the duration proportion of defective and non-defective welding spots to be close to 1:1. taking laser head air pressure, solder ball temperature and solder ball appearance image of each laser ball spraying welding as a group of data, and marking a defect label as 1 if a defect is generated in the solder ball during the period; if no defect is generated, the defect label is marked as 0. The air pressure, temperature and solder ball image data of each laser welding are taken as 1 group of data together, and more than 5000 groups of data are obtained together.
The 5000 sets of data were prepared according to 7:2: the scale of 1 is divided into a training set, a validation set and a test set, and comprises 3500 groups of data, 1000 groups of data and 500 groups of data respectively. The convolutional neural network model is trained on a training set, a gradient descent method is used in the training process, a loss function adopts a cross entropy function, training rounds are 500, learning rate is 0.02, learning rate is halved after every 50 rounds of training, training effect is verified on a verification set after every 5 rounds of training, and accuracy of the neural network model is tested on a test set after all training is completed.
Specifically, if the final output value of the convolutional neural network model is in the range of [0,0.5], the corresponding solder ball welding point in 1s is well formed and has no defect; if the final output value of the convolutional neural network model is in the range of (0.5, 1), the corresponding solder ball welding point in 1s is indicated to have air holes or a gold plate warping defect.
Compared with the traditional detection method, the tin ball laser welding defect detection method provided by the embodiment of the invention has obvious advantages, as shown in fig. 4, the traditional detection method adopts a conventional convolutional neural network model, only tin ball appearance image information can be processed, laser head air pressure information and tin ball temperature information cannot be considered, and meanwhile, the neural network can only input one image in each operation, and the time change characteristic of the tin ball cannot be considered. According to the invention, on one hand, the air pressure information and the temperature information are unfolded on a time scale and woven into a two-dimensional image, so that the convolutional neural network model has the capability of processing the air pressure information and the temperature information; on the other hand, the method and the device for the convolution of the solder ball images periodically sample solder ball images in a period of time and then superimpose the solder ball images on the channel dimension of the corresponding convolution image, so that time information is concentrated, and the conventional convolution neural network model can extract the time change characteristics of the solder ball. As shown in FIG. 5, the detection precision of the welding defect is obviously improved compared with that of the conventional method, the difficulty of manual inspection after welding can be effectively reduced, the efficiency of industrial production is improved, and the reject ratio of products is reduced.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (8)
1. The tin ball laser welding defect detection method is characterized by comprising the following steps of:
step 100, acquiring monitoring signal data in the welding process, wherein the monitoring signal data comprise laser head air pressure data, solder ball temperature data and solder ball appearance images;
step 200, converting the acquired laser head air pressure data into a laser head air pressure two-dimensional graph and converting the tin ball temperature data into a tin ball temperature two-dimensional graph; arranging tin ball shape images according to a time sequence, sampling at fixed intervals to obtain 8 tin ball shape time sequence sampling images; splicing the laser head air pressure two-dimensional graph, the solder ball temperature two-dimensional graph and the first solder ball appearance time sequence sampling image in the channel direction to obtain a composite image;
step 300, presetting a convolutional neural network model, wherein the convolutional neural network model comprises an input layer, 8 convolutional units, a full-connection layer and an output layer which are sequentially connected, and the convolutional units comprise a convolutional layer and a pooling layer; inputting the composite image into a first convolution unit to obtain an image after the first convolution, splicing a second tin ball shape time sequence sampling image with the image after the first convolution after passing through a feature extraction module, inputting the image into the second convolution unit, splicing an 8 th tin ball shape time sequence sampling image with the image after the 7 convolutions after passing through the feature extraction module in sequence, inputting the image into an 8 th convolution unit, converting the data volume passing through the 8 th convolution unit into a one-dimensional vector, inputting the one-dimensional vector into a full connection layer, obtaining a vector after passing through the full connection layer, and outputting the vector through an output layer;
step 400, obtaining a plurality of groups of monitoring signal data and corresponding defect conditions thereof through a pre-experiment to form a database; training a preset convolutional neural network model by utilizing a database, learning a mapping relation between monitoring signal data and welding defects to obtain a trained convolutional neural network model, and identifying whether the welding defects are generated by solder ball laser welding by utilizing the trained convolutional neural network model.
2. The method for detecting a solder ball laser welding defect according to claim 1, wherein,
in step 200, the air pressure range on the curve of the air pressure data of the laser head changing along with time is converted into the interval of [0, 255], and then the continuous air pressure values are converted into the two-dimensional image of the air pressure of the laser head according to the rectangular close-packed mode.
3. The method according to claim 1, wherein in the step 200, the temperature range on the curve of the temperature data of the solder ball with time is converted into a range of [0, 255], and then the continuous temperature values are converted into a two-dimensional image of the temperature of the solder ball in a rectangular close-packed manner.
4. The method according to claim 1, wherein in the step 300, the convolution kernel size of the convolution layer is 3×3, and the step size is 1×1.
5. The method according to claim 1, wherein in the step 300, the pooling layer adopts a maximum pooling operation, and the convolution kernels of the first pooling layer and the second pooling layer are 2×2, and the step length is 1×1; the convolution kernels of the third to eighth pooling layers are 2×2, and the step size is 2×2.
6. The method as claimed in claim 1, wherein in step 400, the database is used for determining the defect of the solder ball according to the following formula 7:2: the scale of 1 is divided into a training set, a validation set and a test set.
7. The method for detecting a solder ball laser welding defect according to claim 1, wherein the step 400 is characterized in that a random gradient descent method is adopted to perform iterative training on a preset convolutional neural network model.
8. The method for detecting solder ball laser welding defects according to claim 7, wherein the step 400 is performed by training a convolutional neural network model, and the loss function is a cross entropy function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410294483.4A CN117884786B (en) | 2024-03-15 | 2024-03-15 | Solder ball laser welding defect detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410294483.4A CN117884786B (en) | 2024-03-15 | 2024-03-15 | Solder ball laser welding defect detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117884786A true CN117884786A (en) | 2024-04-16 |
| CN117884786B CN117884786B (en) | 2024-05-28 |
Family
ID=90641390
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410294483.4A Active CN117884786B (en) | 2024-03-15 | 2024-03-15 | Solder ball laser welding defect detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117884786B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107402217A (en) * | 2017-07-27 | 2017-11-28 | 哈尔滨工业大学（威海） | The inline diagnosis method of the laser gain material manufacturing defect of view-based access control model sensing |
| WO2018072326A1 (en) * | 2016-10-21 | 2018-04-26 | 广州视源电子科技股份有限公司 | Image identification-based solder joint type detection and apparatus |
| CN113329836A (en) * | 2018-11-22 | 2021-08-31 | 普雷茨特两合公司 | Monitoring a laser machining process by means of a deep convolutional neural network |
| US20220040776A1 (en) * | 2018-12-21 | 2022-02-10 | Mycronic AB | Method for controlling an ejector, and related system |
| CN114309895A (en) * | 2021-12-17 | 2022-04-12 | 山东大学 | Deep learning method for predicting weld morphology by welding pool image |
| CN115187500A (en) * | 2021-04-01 | 2022-10-14 | 桂林电子科技大学 | Solder ball defect detection method based on deep learning and adaptive threshold segmentation |
| CN115397595A (en) * | 2020-04-20 | 2022-11-25 | 乔治洛德方法研究和开发液化空气有限公司 | Method, device and computer program for determining the performance of a welding method via digital processing of images of a welded workpiece |
| CN116419815A (en) * | 2020-09-09 | 2023-07-11 | 普雷茨特两合公司 | Method for monitoring the condition of a laser processing head and laser processing system for carrying out said method |
| CN116893177A (en) * | 2023-09-11 | 2023-10-17 | 四川大学 | Method for monitoring internal defects in laser additive manufacturing process based on CCD and pyrometer |
| CN117464182A (en) * | 2023-12-21 | 2024-01-30 | 哈尔滨工业大学（威海） | Laser welding state discrimination method based on convolutional neural network |
-
2024
- 2024-03-15 CN CN202410294483.4A patent/CN117884786B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018072326A1 (en) * | 2016-10-21 | 2018-04-26 | 广州视源电子科技股份有限公司 | Image identification-based solder joint type detection and apparatus |
| CN107402217A (en) * | 2017-07-27 | 2017-11-28 | 哈尔滨工业大学（威海） | The inline diagnosis method of the laser gain material manufacturing defect of view-based access control model sensing |
| CN113329836A (en) * | 2018-11-22 | 2021-08-31 | 普雷茨特两合公司 | Monitoring a laser machining process by means of a deep convolutional neural network |
| US20220040776A1 (en) * | 2018-12-21 | 2022-02-10 | Mycronic AB | Method for controlling an ejector, and related system |
| CN115397595A (en) * | 2020-04-20 | 2022-11-25 | 乔治洛德方法研究和开发液化空气有限公司 | Method, device and computer program for determining the performance of a welding method via digital processing of images of a welded workpiece |
| CN116419815A (en) * | 2020-09-09 | 2023-07-11 | 普雷茨特两合公司 | Method for monitoring the condition of a laser processing head and laser processing system for carrying out said method |
| CN115187500A (en) * | 2021-04-01 | 2022-10-14 | 桂林电子科技大学 | Solder ball defect detection method based on deep learning and adaptive threshold segmentation |
| CN114309895A (en) * | 2021-12-17 | 2022-04-12 | 山东大学 | Deep learning method for predicting weld morphology by welding pool image |
| CN116893177A (en) * | 2023-09-11 | 2023-10-17 | 四川大学 | Method for monitoring internal defects in laser additive manufacturing process based on CCD and pyrometer |
| CN117464182A (en) * | 2023-12-21 | 2024-01-30 | 哈尔滨工业大学（威海） | Laser welding state discrimination method based on convolutional neural network |
Non-Patent Citations (1)
| Title |
|---|
| 王建林;付雪松;黄展超;郭永奇;王汝童;赵利强;: "改进YOLOv2卷积神经网络的多类型合作目标检测", 光学精密工程, no. 01, 15 January 2020 (2020-01-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117884786B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109239102B (en) | CNN-based flexible circuit board appearance defect detection method | |
| CN105092607B (en) | Spherical optics element surface flaw evaluation method | |
| CN107705293A (en) | A kind of hardware dimension measurement method based on CCD area array cameras vision-based detections | |
| US11715190B2 (en) | Inspection system, image discrimination system, discrimination system, discriminator generation system, and learning data generation device | |
| CN112233067A (en) | Hot rolled steel coil end face quality detection method and system | |
| CN112540089A (en) | Application method of digital imaging system in concrete bridge crack detection and analysis | |
| CN114494185B (en) | Electrical equipment fault detection method based on RGB-T multi-scale feature fusion | |
| CN110503623A (en) | A method of Bird's Nest defect on the identification transmission line of electricity based on convolutional neural networks | |
| CN116797977A (en) | Method and device for identifying dynamic target of inspection robot and measuring temperature and storage medium | |
| CN117884786B (en) | Solder ball laser welding defect detection method | |
| CN115631411A (en) | Method for detecting damage of insulator in different environments based on STEN network | |
| CN117455917B (en) | Establishment of false alarm library of etched lead frame and false alarm on-line judging and screening method | |
| CN113642558A (en) | X-ray image identification method and device for strain clamp crimping defects | |
| CN116091506A (en) | Machine vision defect quality inspection method based on YOLOV5 | |
| CN110619365B (en) | Method for detecting falling water | |
| CN114119445A (en) | Pad voidage calculation method based on automatic X-ray imaging | |
| CN113034359A (en) | Intelligent diagnosis method and device for generator set stator and rotor defects, computer equipment and storage medium | |
| CN116721083A (en) | Back clamp for detecting flaws of power equipment and detection method thereof | |
| CN115661094A (en) | Industrial flaw detection method based on improved YOLOX model | |
| CN115147591A (en) | Transformer equipment infrared image voltage heating type defect diagnosis method and system | |
| CN112257514B (en) | Infrared vision intelligent detection shooting method for equipment fault inspection | |
| CN114266725A (en) | Equipment fault diagnosis system based on coaxial light path imaging | |
| CN114037914A (en) | Unmanned aerial vehicle inspection pipeline thermal insulation fault identification and positioning method | |
| CN114596296A (en) | High-sensitivity hot-rolled steel coil end surface defect identification system and method | |
| CN113034432A (en) | Product defect detection method, system, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |