CN117911416A - Welding quality online detection method, device, equipment and storage medium - Google Patents
Welding quality online detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117911416A CN117911416A CN202410316449.2A CN202410316449A CN117911416A CN 117911416 A CN117911416 A CN 117911416A CN 202410316449 A CN202410316449 A CN 202410316449A CN 117911416 A CN117911416 A CN 117911416A
- Authority
- CN
- China
- Prior art keywords
- welding
- image
- current
- frame
- welding image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003466 welding Methods 0.000 title claims abstract description 453
- 238000001514 detection method Methods 0.000 title claims abstract description 75
- 230000007547 defect Effects 0.000 claims abstract description 88
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims description 62
- 238000004590 computer program Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000001186 cumulative effect Effects 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 5
- 238000011897 real-time detection Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 229910000831 Steel Inorganic materials 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 239000010959 steel Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000010953 base metal Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 229910001338 liquidmetal Inorganic materials 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000725 suspension Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a welding quality online detection method, a device, equipment and a storage medium. The method comprises the steps of collecting a current frame welding image, welding current and wind speed in real time in a welding process, calculating the molten pool width from the welding image, calculating the molten pool width of the current frame welding image, the average molten pool width of a previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image, inputting the obtained state score of the current frame welding image into a pre-trained defect detection model for processing, updating the accumulated state score of the historical frame welding image, realizing real-time detection of the quality of a welding seam, intervening in time, avoiding the problem of time and labor consumption of overall reworking when the welding seam is completely finished, and improving the welding quality and efficiency.
Description
Technical Field
The present invention relates to welding technologies, and in particular, to a method, an apparatus, a device, and a storage medium for online detection of welding quality.
Background
The steel structure has the advantages of high strength and small dead weight in a large structure. The method is widely applied to occasions such as large suspension bridges, cable-stayed bridges, high-rise buildings, immersed tube tunnels and the like.
The connection quality requirement of the large-scale structure on the steel structure is higher, and the connection of the steel structure in the field is affected by a plurality of factors, so that the welding quality is not guaranteed.
Disclosure of Invention
The invention provides a welding quality online detection method, a welding quality online detection device, welding quality online detection equipment and a storage medium, so as to improve welding quality and welding efficiency.
In a first aspect, the present invention provides a welding quality online detection method, including:
Acquiring a welding image, welding current and wind speed of a current frame in real time in a welding process;
Calculating a weld pool width from the welding image;
Counting the average molten pool width, the historical average molten pool width, the average welding current and the historical average welding current of the previous K frame welding images;
The molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image are input into a pre-trained defect detection model to be processed, the state score of the current frame welding image is obtained, and the accumulated state score of the historical frame welding image is updated and used for representing the welding quality.
Optionally, calculating the weld puddle width from the welding image includes:
acquiring a gray value of each pixel point in a welding image of a current frame;
threshold segmentation is carried out on the welding image of the current frame based on the gray value, and a molten pool is identified from the welding image of the current frame;
and counting the pixel width of the molten pool as the molten pool width.
Optionally, threshold segmentation is performed on the welding image of the current frame based on the gray value, and a molten pool is identified from the welding image of the current frame, including:
Comparing the gray value of the pixel in the welding image of the current frame with a preset gray threshold;
and taking an area formed by pixels with gray values larger than the gray threshold value as a molten pool.
Optionally, the defect detection model includes an input layer, a hidden layer and an output layer, where the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the cumulative state score of the historical frame welding image are input into a pre-trained defect detection model to be processed, so as to obtain the state score of the current frame welding image, and the cumulative state score of the historical frame welding image is updated, where the state score is used for characterizing welding quality, and includes:
Normalizing the molten pool width of the current frame of welding image, the average molten pool width of the previous K frame of welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame of welding image, the average welding current of the previous K frame of welding image, the historical average welding current, the state score of the previous frame of welding image and the accumulated state score of the historical frame of welding image in the input layer;
convoluting and pooling the normalized data in the hidden layer;
Performing feature mapping on the convolved and pooled data in the output layer to obtain a probability value of welding defects of the welding image of the current frame;
determining a state score of the welding image of the current frame based on the probability value;
And adding the state score of the current frame welding image to the accumulated state score of the history frame welding image to update the accumulated state score of the history frame welding image.
Optionally, determining a status score of the welding image of the current frame based on the probability value includes:
comparing the probability value with a preset probability threshold value;
when the probability value is greater than or equal to the probability threshold value, recording the state score of the welding image of the current frame as a first score representing that a defect exists;
and when the probability value is smaller than the probability threshold value, recording the state score of the welding image of the current frame as a second score which indicates that no defect exists.
Optionally, before acquiring the welding image, the welding current and the wind speed of the current frame, the method further comprises:
Training the defect detection model by adopting a batch of training samples, and predicting the predicted probability value of each training sample for existence of welding defects, wherein the batch comprises a plurality of training samples;
Calculating the square of the difference between the actual probability value of the welding defect of the training sample and the predicted probability value to obtain a first numerical value, wherein the actual probability value is 1 when the welding defect exists in the training sample, and the actual probability value is 0 when the welding defect does not exist in the training sample;
calculating the sum of the first values corresponding to all the training samples in the batch to obtain the loss value of the training samples in the batch;
Comparing the loss value with a preset loss threshold value;
When the loss value is larger than the loss threshold value, acquiring a next batch of training samples, updating parameters of the defect detection model, and returning to execute the step of training the defect detection model by adopting one batch of training samples to predict the predicted probability value of each training sample that the welding defect exists until the loss value is smaller than or equal to the loss threshold value.
Optionally, after obtaining the status score of the welding image of the current frame, the method further includes:
and marking the current frame welding image when the state score of the current frame welding image is a first score indicating the existence of the defect, and sending a welding quality risk prompt to an operator.
In a second aspect, the present invention also provides a welding quality online detection device, including:
The data acquisition module is used for acquiring a welding image, welding current and wind speed of a current frame in real time in the welding process;
A molten pool width calculation module for calculating a molten pool width from the welding image;
The data statistics module is used for counting the average molten pool width, the historical average molten pool width, the average welding current and the historical average welding current of the previous K frame welding images;
The defect prediction module is used for inputting the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image into a pre-trained defect detection model for processing, obtaining the state score of the current frame welding image, and updating the accumulated state score of the historical frame welding image, wherein the state score is used for representing the welding quality.
In a third aspect, the present invention also provides an electronic device, including:
one or more processors;
A storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the weld quality online detection method as provided by the first aspect of the present invention.
In a fourth aspect, the present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the welding quality online detection method as provided in the first aspect of the present invention.
The welding quality online detection method provided by the invention comprises the following steps: the method comprises the steps of collecting a current frame welding image, welding current and wind speed in real time in a welding process, calculating a molten pool width from the welding image, counting the average molten pool width of a previous K frame welding image, the historical average molten pool width, the average welding current of a previous K frame welding image and the historical average welding current, inputting the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of a previous frame welding image and the accumulated state score of the historical frame welding image into a pre-trained defect detection model for processing, obtaining the state score of the current frame welding image, and updating the accumulated state score of the historical frame welding image, wherein the state score is used for representing welding quality. According to the invention, the relation between the welding quality and a plurality of parameters in field operation is excavated through the neural network model, the weld defect is predicted and identified, and the weld defect is intervened in time, so that the problem of time and labor consumption for overall reworking when the weld is completely finished is avoided, and the welding quality and efficiency are improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a welding quality online detection method provided by an embodiment of the invention;
fig. 2 is a schematic structural diagram of a welding auxiliary device according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a process of a defect detection model according to an embodiment of the present invention;
Fig. 4 is a schematic structural diagram of an online welding quality detection device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a flowchart of a welding quality online detection method provided by an embodiment of the present invention, where the embodiment is applicable to online detection of welding quality under outdoor operation, and the method may be performed by a welding quality online detection device provided by the embodiment of the present invention, where the device may be implemented by software and/or hardware, and is typically configured in an electronic device, as shown in fig. 1, and the welding quality online detection method may include the following steps:
s101, acquiring a welding image, welding current and wind speed of a current frame in real time in a welding process.
In the embodiment of the invention, the welding image of the current frame is acquired in real time in the field steel structure welding process, and the welding current and the wind speed corresponding to the welding image of the current frame are acquired.
Fig. 2 is a schematic structural diagram of a welding auxiliary device according to an embodiment of the present invention, and as shown in fig. 2, the device includes a welding gun 110, and a welding tip 111 is disposed at a front end of the welding gun 110. The welding gun 110 is provided with a camera 120, and the camera 120 is used for acquiring welding images in real time in the welding process. The welding gun 110 is also provided with a wind pressure sensor 130, and the wind pressure sensor 130 is used for collecting the wind speed parallel to the welding line direction in real time in the welding process. In addition, a display 140 may be disposed on the welding gun 110, where the display 140 is used to display the image acquired by the camera 120 in real time.
S102, calculating the width of a molten pool from a welding image.
In an embodiment of the present invention, the puddle width is calculated from the current frame welding image after the current frame welding image is acquired. The molten pool is a base metal portion melted into a pool shape by heat of a welding arc, and a liquid metal portion having a certain geometry formed on a weldment at the time of welding is called a molten pool. The weld pool width refers to the maximum width perpendicular to the weld direction.
Illustratively, in some embodiments of the present invention, the step S102 includes the following sub-steps:
s1011, acquiring the gray value of each pixel point in the welding image of the current frame.
And after the welding image of the current frame is acquired, acquiring the gray value of each pixel point in the welding image of the current frame.
And S1012, threshold segmentation is carried out on the welding image of the current frame based on the gray value, and a molten pool is identified from the welding image of the current frame.
Illustratively, in the welding image, the temperature of the area where the molten pool is located is obviously higher than that of other areas, and the area is in white heating, and the gray value of the area is obviously higher than that of the other areas. Thus, the current frame welding image may be thresholded based on the gray value, from which the puddle is identified. Illustratively, the gray value of the pixel in the welding image of the current frame is compared with a preset gray threshold value, and the area formed by the pixel with the gray value larger than the gray threshold value is taken as a molten pool.
S1013, counting the pixel width of the molten pool as the molten pool width.
In the embodiment of the invention, the maximum pixel width of the molten pool perpendicular to the welding line direction is counted as the molten pool width, and the pixel width is the number of pixels in the width direction.
S103, counting the average molten pool width, the historical average molten pool width, the average welding current and the historical average welding current of the previous K frame welding images.
In the embodiment of the invention, the average molten pool width of the previous K frame welding images before the current frame, the historical average molten pool width of all the historical welding images, the average welding current of the previous K frame welding images and the historical average welding current are counted.
S104, inputting the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image into a pre-trained defect detection model for processing to obtain the state score of the current frame welding image, and updating the accumulated state score of the historical frame welding image, wherein the state score is used for representing the welding quality.
Fig. 3 is a schematic diagram of a processing of a defect detection model provided in an embodiment of the present invention, as shown in fig. 3, in an embodiment of the present invention, a molten pool width of a current frame welding image, an average molten pool width of a previous K frame welding image, a historical average molten pool width, a wind speed, a welding current corresponding to the current frame welding image, an average welding current of the previous K frame welding image, a historical average welding current, a status score of a previous frame welding image, and an accumulated status score of a historical frame welding image are input into a pre-trained defect detection model to be processed, so as to obtain a status score of the current frame welding image, and an accumulated status score of the historical frame welding image is updated. The status score is used to characterize the quality of the weld, and illustratively, is 1 when a weld defect is detected and 0 when no weld defect is detected.
Illustratively, in some embodiments of the present invention, the defect detection model includes an input layer, a hidden layer, and an output layer, and the step S104 includes:
S1041, carrying out normalization processing on the molten pool width of the welding image of the current frame, the average molten pool width of the welding image of the previous K frame, the historical average molten pool width, the wind speed, the welding current corresponding to the welding image of the current frame, the average welding current of the welding image of the previous K frame, the historical average welding current, the state score of the welding image of the previous frame and the accumulated state score of the welding image of the historical frame in an input layer.
The input layer includes 9 neurons for receiving 9 parameters of input, and normalizes the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image in the input layer. As shown in fig. 3, the molten pool width of the welding image of the current frame, the average molten pool width of the welding image of the previous K frames, the historical average molten pool width, the wind speed, the welding current corresponding to the welding image of the current frame, the average welding current of the welding image of the previous K frames and the historical average welding current are taken as input parameters of the current frame. The normalization process is a basic work of data mining, different evaluation indexes often have different dimensions and dimension units, the situation can influence the result of data analysis, and in order to eliminate the dimension influence among indexes, the data normalization process is needed to solve the comparability among the data indexes. After the original data is subjected to data normalization processing, all indexes are in the same order of magnitude, and the method is suitable for comprehensive comparison and evaluation.
S1042, convoluting and pooling the normalized data in the hidden layer.
By way of example, the hidden layer may include multiple layers, e.g., multiple convolutional layers, multiple pooled layers, and embodiments of the invention are not limited in this regard. In the hidden layer, the normalized data is subjected to convolution and pooling processing, and the relation between a plurality of input parameters and the relation between the input parameters and the predicted result are mined.
S1043, performing feature mapping on the convolved and pooled data in the output layer to obtain a probability value of welding defects of the welding image of the current frame.
The output layer is typically a full connection layer, and the convolved and pooled data is subjected to feature mapping in the output layer to obtain a probability value that a welding defect exists in the welding image of the current frame.
S1044, determining a state score of the welding image of the current frame based on the probability value.
The calculated probability value is compared with a preset probability threshold, when the probability value is larger than or equal to the probability threshold, the defect exists in the welding image of the current frame, the state score of the welding image of the current frame is recorded as a first score 1 representing the defect, and when the probability value is smaller than the probability threshold, the state score of the welding image of the current frame is recorded as a second score 0 representing the defect does not exist.
S1045, adding the state score of the current frame welding image to the accumulated state score of the history frame welding image to update the accumulated state score of the history frame welding image.
After the state score of the current frame welding image is obtained, adding the state score of the current frame welding image to the accumulated state score of the history frame welding image to update the accumulated state score of the history frame welding image.
Illustratively, in some embodiments of the present invention, after obtaining the status score of the current frame welding image, the method further comprises:
When the state score of the welding image of the current frame is a first score indicating that a defect exists, the welding image of the current frame is marked, and a welding quality risk prompt is sent to an operator, so that the welder can adjust welding parameters in time for correction.
In the embodiment of the invention, before the defect detection model is applied, the defect detection model needs to be trained, and the specific training process is as follows:
1. And training the defect detection model by adopting a batch of training samples, and predicting the predicted probability value of each training sample for existence of welding defects, wherein one batch comprises a plurality of training samples.
The training samples of one batch include a molten pool width, a welding current and a wind speed corresponding to the samples of continuous multi-frame welding images, and for each frame, the training samples further include an average molten pool width, a historical average molten pool width, an average welding current, a historical average welding current, a state score and an accumulated state score of a previous frame welding image of a K frame welding image before the frame, the data are input into a defect detection model to be trained, the defect detection model predicts a predicted probability value that a defect exists in the frame welding image, the state score of the frame welding image is determined, and the accumulated state score of the historical frame welding image is updated.
2. And calculating the square of the difference between the actual probability value of the welding defect of the training sample and the predicted probability value to obtain a first numerical value, wherein the actual probability value is 1 when the welding defect exists in the training sample, and the actual probability value is 0 when the welding defect does not exist in the training sample.
After obtaining the predicted probability value of each training sample with the welding defect, calculating the square of the difference between the actual probability value of the training sample with the welding defect and the predicted probability value to obtain a first numerical value, wherein the actual probability value is 1 when the training sample has the welding defect, and the actual probability value is 0 when the training sample does not have the welding defect.
3. And calculating the sum of the first values corresponding to all the training samples in the batch to obtain the loss value of the training samples in the batch.
And calculating the sum of the first values corresponding to all the training samples in the batch to obtain the loss value of the training samples in the batch. Illustratively, the loss value calculation formula is as follows:
Lloss=Σ(Qi-qi)2
Wherein, the actual probability value of the i frame of Q i, and the model of the i frame of Q i output the predicted probability value.
4. And comparing the loss value with a preset loss threshold value.
5. When the loss value is larger than the loss threshold value, acquiring a next batch of training samples, updating parameters of the defect detection model, and returning to execute the step of training the defect detection model by adopting one batch of training samples to predict the predicted probability value of the welding defect of each training sample until the loss value is smaller than or equal to the loss threshold value, and fixing the model parameters.
The welding quality online detection method provided by the embodiment of the invention comprises the following steps: the method comprises the steps of collecting a current frame welding image, welding current and wind speed in real time in a welding process, calculating a molten pool width from the welding image, counting the average molten pool width of a previous K frame welding image, the historical average molten pool width, the average welding current of a previous K frame welding image and the historical average welding current, inputting the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of a previous frame welding image and the accumulated state score of the historical frame welding image into a pre-trained defect detection model for processing, obtaining the state score of the current frame welding image, and updating the accumulated state score of the historical frame welding image, wherein the state score is used for representing welding quality. According to the invention, the relation between the welding quality and a plurality of parameters in field operation is excavated through the neural network model, the weld defect is predicted and identified, and the weld defect is intervened in time, so that the problem of time and labor consumption for overall reworking when the weld is completely finished is avoided, and the welding quality and efficiency are improved.
Fig. 4 is a schematic structural diagram of an online welding quality detection device according to an embodiment of the present invention, where, as shown in fig. 4, the online welding quality detection device includes:
the data acquisition module 201 is used for acquiring a welding image, welding current and wind speed of a current frame in real time in a welding process;
A puddle width calculation module 202 for calculating a puddle width from the welding image;
The data statistics module 203 is configured to count an average molten pool width of the previous K frame welding images, a historical average molten pool width, an average welding current of the previous K frame welding images, and a historical average welding current;
The defect prediction module 204 is configured to input the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image, and the cumulative state score of the historical frame welding image into a pre-trained defect detection model for processing, obtain the state score of the current frame welding image, and update the cumulative state score of the historical frame welding image, where the state score is used to characterize welding quality.
In some embodiments of the invention, the bath width calculation module 202 includes:
The gray value acquisition sub-module is used for acquiring the gray value of each pixel point in the welding image of the current frame;
The molten pool identification sub-module is used for carrying out threshold segmentation on the welding image of the current frame based on the gray value, and identifying a molten pool from the welding image of the current frame;
And the molten pool width determining submodule is used for counting the pixel width of the molten pool as the molten pool width.
In some embodiments of the invention, the puddle identification submodule includes:
The gray value comparison unit is used for comparing the gray value of the pixel in the welding image of the current frame with a preset gray threshold value;
and the molten pool determining unit is used for taking a region formed by pixels with gray values larger than the gray threshold value as a molten pool.
In some embodiments of the present invention, the defect detection model includes an input layer, a hidden layer, and an output layer, and the defect prediction module 204 includes:
An input sub-module, configured to normalize, in the input layer, a molten pool width of a current frame welding image, an average molten pool width of a previous K frame welding image, a historical average molten pool width, a wind speed, a welding current corresponding to the current frame welding image, an average welding current of the previous K frame welding image, a historical average welding current, a state score of a previous frame welding image, and an accumulated state score of the historical frame welding image;
The middle sub-module is used for convoluting and pooling the normalized data in the hidden layer;
The output sub-module is used for carrying out feature mapping on the convolved and pooled data in the output layer to obtain a probability value of welding defects of the welding image of the current frame;
A score determining sub-module for determining a status score of the welding image of the current frame based on the probability value;
and the score updating sub-module is used for adding the state score of the current frame welding image to the accumulated state score of the history frame welding image to update the accumulated state score of the history frame welding image.
In some embodiments of the invention, the score determination submodule includes:
the probability value comparison unit is used for comparing the probability value with a preset probability threshold value;
a first score determining unit configured to score a state score of the welding image of the current frame as a first score indicating the presence of a defect when the probability value is greater than or equal to the probability threshold;
and the second score determining unit is used for recording the state score of the welding image of the current frame as a second score representing that no defect exists when the probability value is smaller than the probability threshold value.
In some embodiments of the present invention, the welding quality online detection apparatus further includes:
the model training module is used for training the defect detection model by adopting a batch of training samples before acquiring the welding image, the welding current and the wind speed of the current frame, and predicting the prediction probability value of each training sample for existence of the welding defect, wherein the batch comprises a plurality of training samples;
The prediction probability module is used for calculating the square of the difference between the actual probability value of the welding defect of the training sample and the prediction probability value to obtain a first numerical value, wherein the actual probability value is 1 when the welding defect exists in the training sample, and the actual probability value is 0 when the welding defect does not exist in the training sample;
The loss value calculation module is used for calculating the sum of the first values corresponding to all the training samples in the batch to obtain the loss value of the training samples in the batch;
The loss value comparison module is used for comparing the loss value with a preset loss threshold value;
And the parameter updating module is used for acquiring a next batch of training samples when the loss value is larger than the loss threshold value, updating parameters of the defect detection model, and returning to execute the step of training the defect detection model by adopting one batch of training samples to predict the predicted probability value of each training sample that the welding defect exists until the loss value is smaller than or equal to the loss threshold value.
In some embodiments of the present invention, the welding quality online detection apparatus further includes:
And the prompting module is used for marking the welding image of the current frame and sending a welding quality risk prompt to an operator when the state score of the welding image of the current frame is a first score indicating that the defect exists after the state score of the welding image of the current frame is obtained.
The welding quality online detection device can execute the welding quality online detection method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the welding quality online detection method.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 can perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device can also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the electronic device are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the weld quality online detection method.
In some embodiments, the weld quality online detection method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the welding quality online detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the weld quality online detection method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
Embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements a welding quality online detection method as provided by any of the embodiments of the present application.
Computer program product in the implementation, the computer program code for carrying out operations of the present invention may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. An on-line welding quality detection method is characterized by comprising the following steps:
Acquiring a welding image, welding current and wind speed of a current frame in real time in a welding process;
Calculating a weld pool width from the welding image;
Counting the average molten pool width, the historical average molten pool width, the average welding current and the historical average welding current of the previous K frame welding images;
The molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image are input into a pre-trained defect detection model to be processed, the state score of the current frame welding image is obtained, and the accumulated state score of the historical frame welding image is updated and used for representing the welding quality.
2. The welding quality online detection method according to claim 1, wherein calculating a puddle width from the welding image includes:
acquiring a gray value of each pixel point in a welding image of a current frame;
threshold segmentation is carried out on the welding image of the current frame based on the gray value, and a molten pool is identified from the welding image of the current frame;
and counting the pixel width of the molten pool as the molten pool width.
3. The welding quality online detection method according to claim 2, wherein threshold segmentation is performed on the current frame welding image based on the gray value, and the molten pool is identified from the current frame welding image, comprising:
Comparing the gray value of the pixel in the welding image of the current frame with a preset gray threshold;
and taking an area formed by pixels with gray values larger than the gray threshold value as a molten pool.
4. A method for online detection of welding quality according to any one of claims 1 to 3, wherein the defect detection model includes an input layer, a hidden layer and an output layer, the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image, and the cumulative state score of the historical frame welding image are input into the defect detection model trained in advance for processing, the state score of the current frame welding image is obtained, and the cumulative state score of the historical frame welding image is updated, and the state score is used for representing the welding quality and includes:
Normalizing the molten pool width of the current frame of welding image, the average molten pool width of the previous K frame of welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame of welding image, the average welding current of the previous K frame of welding image, the historical average welding current, the state score of the previous frame of welding image and the accumulated state score of the historical frame of welding image in the input layer;
convoluting and pooling the normalized data in the hidden layer;
Performing feature mapping on the convolved and pooled data in the output layer to obtain a probability value of welding defects of the welding image of the current frame;
determining a state score of the welding image of the current frame based on the probability value;
And adding the state score of the current frame welding image to the accumulated state score of the history frame welding image to update the accumulated state score of the history frame welding image.
5. The welding quality online detection method of claim 4, wherein determining a status score for a current frame welding image based on the probability value comprises:
comparing the probability value with a preset probability threshold value;
when the probability value is greater than or equal to the probability threshold value, recording the state score of the welding image of the current frame as a first score representing that a defect exists;
and when the probability value is smaller than the probability threshold value, recording the state score of the welding image of the current frame as a second score which indicates that no defect exists.
6. A welding quality online detection method according to any one of claims 1-3, further comprising, prior to acquiring the current frame welding image, welding current, and wind speed:
Training the defect detection model by adopting a batch of training samples, and predicting the predicted probability value of each training sample for existence of welding defects, wherein the batch comprises a plurality of training samples;
Calculating the square of the difference between the actual probability value of the welding defect of the training sample and the predicted probability value to obtain a first numerical value, wherein the actual probability value is 1 when the welding defect exists in the training sample, and the actual probability value is 0 when the welding defect does not exist in the training sample;
calculating the sum of the first values corresponding to all the training samples in the batch to obtain the loss value of the training samples in the batch;
Comparing the loss value with a preset loss threshold value;
When the loss value is larger than the loss threshold value, acquiring a next batch of training samples, updating parameters of the defect detection model, and returning to execute the step of training the defect detection model by adopting one batch of training samples to predict the predicted probability value of each training sample that the welding defect exists until the loss value is smaller than or equal to the loss threshold value.
7. A welding quality online detection method according to any one of claims 1 to 3, further comprising, after obtaining the status score of the current frame welding image:
and marking the current frame welding image when the state score of the current frame welding image is a first score indicating the existence of the defect, and sending a welding quality risk prompt to an operator.
8. An on-line welding quality detection device, comprising:
The data acquisition module is used for acquiring a welding image, welding current and wind speed of a current frame in real time in the welding process;
A molten pool width calculation module for calculating a molten pool width from the welding image;
The data statistics module is used for counting the average molten pool width, the historical average molten pool width, the average welding current and the historical average welding current of the previous K frame welding images;
The defect prediction module is used for inputting the molten pool width of the current frame welding image, the average molten pool width of the previous K frame welding image, the historical average molten pool width, the wind speed, the welding current corresponding to the current frame welding image, the average welding current of the previous K frame welding image, the historical average welding current, the state score of the previous frame welding image and the accumulated state score of the historical frame welding image into a pre-trained defect detection model for processing, obtaining the state score of the current frame welding image, and updating the accumulated state score of the historical frame welding image, wherein the state score is used for representing the welding quality.
9. An electronic device, comprising:
one or more processors;
A storage means for storing one or more programs;
When executed by the one or more processors, causes the one or more processors to implement the weld quality online detection method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the welding quality online detection method as claimed in any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410316449.2A CN117911416B (en) | 2024-03-20 | 2024-03-20 | Welding quality online detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410316449.2A CN117911416B (en) | 2024-03-20 | 2024-03-20 | Welding quality online detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117911416A true CN117911416A (en) | 2024-04-19 |
| CN117911416B CN117911416B (en) | 2024-05-31 |
Family
ID=90686232
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410316449.2A Active CN117911416B (en) | 2024-03-20 | 2024-03-20 | Welding quality online detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117911416B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477066A (en) * | 2009-01-09 | 2009-07-08 | 华南理工大学 | Circuit board element mounting/welding quality detection method and system based on super-resolution image reconstruction |
| CN111862067A (en) * | 2020-07-28 | 2020-10-30 | 中山佳维电子有限公司 | Welding defect detection method and device, electronic equipment and storage medium |
| CN114266286A (en) * | 2021-11-18 | 2022-04-01 | 江苏徐工工程机械研究院有限公司 | Online detection method and device for welding process information |
| CN115439429A (en) * | 2022-08-26 | 2022-12-06 | 武汉铁路职业技术学院 | Weld quality real-time online evaluation method and device, storage medium and terminal |
| CN116468705A (en) * | 2023-04-24 | 2023-07-21 | 上海市机械施工集团有限公司 | Welding spot image detection method and device, electronic equipment and storage medium |
| CN116618878A (en) * | 2023-05-30 | 2023-08-22 | 江苏徐工工程机械研究院有限公司 | Pre-welding process parameter determination method, welding quality online prediction method, device and storage medium |
| US20230264285A1 (en) * | 2020-10-16 | 2023-08-24 | Kabushiki Kaisha Kobe Seiko Sho (Kobe Steel, Ltd.) | Welding system, welding method, welding support device, program, learning device, and method of generating trained model |
| KR102572304B1 (en) * | 2023-03-09 | 2023-08-28 | 박종영 | Method and apparatus for providing real-time welding inspection and defect prevention service based on artificial intelligence algorithm |
-
2024
- 2024-03-20 CN CN202410316449.2A patent/CN117911416B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477066A (en) * | 2009-01-09 | 2009-07-08 | 华南理工大学 | Circuit board element mounting/welding quality detection method and system based on super-resolution image reconstruction |
| CN111862067A (en) * | 2020-07-28 | 2020-10-30 | 中山佳维电子有限公司 | Welding defect detection method and device, electronic equipment and storage medium |
| US20230264285A1 (en) * | 2020-10-16 | 2023-08-24 | Kabushiki Kaisha Kobe Seiko Sho (Kobe Steel, Ltd.) | Welding system, welding method, welding support device, program, learning device, and method of generating trained model |
| CN114266286A (en) * | 2021-11-18 | 2022-04-01 | 江苏徐工工程机械研究院有限公司 | Online detection method and device for welding process information |
| CN115439429A (en) * | 2022-08-26 | 2022-12-06 | 武汉铁路职业技术学院 | Weld quality real-time online evaluation method and device, storage medium and terminal |
| KR102572304B1 (en) * | 2023-03-09 | 2023-08-28 | 박종영 | Method and apparatus for providing real-time welding inspection and defect prevention service based on artificial intelligence algorithm |
| CN116468705A (en) * | 2023-04-24 | 2023-07-21 | 上海市机械施工集团有限公司 | Welding spot image detection method and device, electronic equipment and storage medium |
| CN116618878A (en) * | 2023-05-30 | 2023-08-22 | 江苏徐工工程机械研究院有限公司 | Pre-welding process parameter determination method, welding quality online prediction method, device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 赵为松;: "桥梁工程焊接质量管理的研究", 电焊机, no. 10, 20 October 2009 (2009-10-20) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117911416B (en) | 2024-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3937128A2 (en) | Image defect detection method and apparatus, electronic device, storage medium and product | |
| CN113436100B (en) | Method, apparatus, device, medium, and article for repairing video | |
| CN113947571A (en) | Training method of vehicle damage detection model and vehicle damage identification method | |
| CN115294332A (en) | Image processing method, device, equipment and storage medium | |
| CN115937101A (en) | Quality detection method, device, equipment and storage medium | |
| CN116245865A (en) | Image quality detection method and device, electronic equipment and storage medium | |
| CN117911416B (en) | Welding quality online detection method, device, equipment and storage medium | |
| CN116952958A (en) | Defect detection method, device, electronic equipment and storage medium | |
| CN116468705A (en) | Welding spot image detection method and device, electronic equipment and storage medium | |
| CN115761362B (en) | Construction stage intelligent recognition model, method and device based on feature fusion | |
| CN117333443A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN116309471A (en) | Quality defect early warning method and device, electronic equipment and medium | |
| CN113807209A (en) | Parking space detection method and device, electronic equipment and storage medium | |
| CN117764904A (en) | Cloud detection system and method for bridge inhaul cable detection robot | |
| CN117032262B (en) | Machine control method, device, electronic equipment and storage medium | |
| CN116778276A (en) | Safe production model training method, application method, device, equipment and medium | |
| CN117576077A (en) | Defect detection method, device, equipment and storage medium | |
| CN118096698A (en) | Visual detection method and device, electronic equipment and storage medium | |
| CN117829611A (en) | Subcontractor management risk assessment early warning method based on artificial intelligence | |
| CN116539989A (en) | Transformer maintenance method | |
| CN117764961A (en) | Method and device for processing disconnection scratch connection, electronic equipment and storage medium | |
| CN118134880A (en) | Foreign matter detection method, device, equipment and medium | |
| CN117670501A (en) | Service early warning method and device, electronic equipment and storage medium | |
| CN116260973A (en) | Time domain filtering method and device, electronic equipment and storage medium | |
| CN117745699A (en) | Defect detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |