CN114167830A - Welding process monitoring and quality diagnosis method based on machine vision and edge intelligence - Google Patents
Welding process monitoring and quality diagnosis method based on machine vision and edge intelligence Download PDFInfo
- Publication number
- CN114167830A CN114167830A CN202111529178.1A CN202111529178A CN114167830A CN 114167830 A CN114167830 A CN 114167830A CN 202111529178 A CN202111529178 A CN 202111529178A CN 114167830 A CN114167830 A CN 114167830A
- Authority
- CN
- China
- Prior art keywords
- welding
- image
- quality
- edge
- molten pool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000003466 welding Methods 0.000 title claims abstract description 147
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000012544 monitoring process Methods 0.000 title claims abstract description 54
- 230000008569 process Effects 0.000 title claims abstract description 46
- 238000003745 diagnosis Methods 0.000 title claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 20
- 238000005516 engineering process Methods 0.000 claims abstract description 15
- 238000003860 storage Methods 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 13
- 230000005540 biological transmission Effects 0.000 claims abstract description 8
- 238000004364 calculation method Methods 0.000 claims abstract description 7
- 238000010801 machine learning Methods 0.000 claims abstract description 7
- 238000013473 artificial intelligence Methods 0.000 claims description 38
- 230000007547 defect Effects 0.000 claims description 22
- 238000010295 mobile communication Methods 0.000 claims description 13
- 230000002159 abnormal effect Effects 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000002360 preparation method Methods 0.000 claims description 6
- 230000006399 behavior Effects 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- 230000001052 transient effect Effects 0.000 claims description 5
- 238000004140 cleaning Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000010191 image analysis Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000002405 diagnostic procedure Methods 0.000 claims 9
- 238000004519 manufacturing process Methods 0.000 abstract description 17
- 238000001514 detection method Methods 0.000 abstract description 6
- 230000007246 mechanism Effects 0.000 abstract description 2
- 238000003908 quality control method Methods 0.000 abstract description 2
- 238000004804 winding Methods 0.000 abstract description 2
- 238000004891 communication Methods 0.000 abstract 1
- 230000003993 interaction Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 229910052721 tungsten Inorganic materials 0.000 description 4
- 239000010937 tungsten Substances 0.000 description 4
- 229910052751 metal Inorganic materials 0.000 description 3
- 239000002184 metal Substances 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000000739 chaotic effect Effects 0.000 description 2
- 238000013079 data visualisation Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000010223 real-time analysis Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- WFKWXMTUELFFGS-UHFFFAOYSA-N tungsten Chemical compound [W] WFKWXMTUELFFGS-UHFFFAOYSA-N 0.000 description 2
- 238000003326 Quality management system Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 239000011324 bead Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001704 evaporation Methods 0.000 description 1
- 230000008020 evaporation Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000009659 non-destructive testing Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000000275 quality assurance Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/41875—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by quality surveillance of production
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/32—Operator till task planning
- G05B2219/32368—Quality control
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- General Factory Administration (AREA)
- Manipulator (AREA)
Abstract
The invention provides a welding process monitoring and quality diagnosis method based on machine vision and edge intelligence, and belongs to the technical field of industrial internet and intelligent welding manufacturing. Aiming at the problem that the existing robot and automatic welding equipment lack quality on-line detection and diagnosis means, an end-side deployment vision sensing system is utilized to realize continuous collection of welding process molten pool images, edge cloud image processing is realized by combining ultra-long-range and large-delay reliable transmission wireless network communication technologies such as 5G, quality on-line comprehensive judgment is realized based on a machine learning model, diagnosis results and data are uploaded to a cloud server to realize inference training, and quick iteration updating and data sharing multiplexing of a new model are realized. The invention can realize the online monitoring and the self-learning of the welding quality and the continuous shooting and storage of the welding process for a very long time, provides key technical support for the quality control and the tracing mechanism of the whole life cycle of sound products, greatly saves the calculation cost of the end side and reduces the winding loss of cables.
Description
Technical Field
The invention relates to the technical field of industrial internet and intelligent welding manufacturing. The robot welding method based on machine vision and edge intelligence is widely applied to robot automatic welding and manufacturing of large metal components such as box beams, boilers, pressure vessels, storage tanks, long-distance pipelines and the like in the fields of energy equipment, logistics transportation equipment, hoisting machinery, ships, ocean engineering equipment and the like.
Background
Welding has heretofore been the most widespread and important method of achieving permanent joining of materials in global engineering. Welding science and technology, important guarantee for major engineering and equipment manufacturing. With the large-scale development and the improvement of the reliability requirement of metal structural parts in the fields of energy equipment, logistics transportation equipment, hoisting machinery, ships, ocean engineering equipment and the like, higher requirements are provided for the automatic welding quality guarantee and the production efficiency of the robot. However, the quality assurance of equipment manufacturing, especially the detection and diagnosis of welding defects, still generally relies heavily on the off-line nondestructive detection after welding, and the low detection efficiency and high labor intensity cannot meet the practical requirements of batch production and the improvement of the intelligent level of the welding manufacturing process; on the other hand, the development and continuous perfection of the related technologies of artificial intelligence and intelligent manufacturing not only provides strong power for welding robot manufacturers to accelerate and improve the intelligent level of equipment, but also promotes the data value of the mining welding manufacturing process to become a most urgent subject for welding robot users.
The welding process monitoring is an important support for ensuring the welding quality and is also a key technology of an intelligent welding manufacturing system. The welding process monitoring based on machine vision can continuously acquire the transient process information of the welding pool area, not only can provide the most intuitive welding quality information feedback for the actual operation and training of welding operators, but also can provide a basis for guiding the optimization of the welding process. At present, the visual imaging monitoring of a molten pool is realized in some specific welding occasions, but the oriented object is mainly an operator who adopts naked eye observation, the method cannot be applied to automatic and intelligent manufacturing equipment such as robots, and the key problems which need to be solved urgently exist as follows:
firstly, due to the existence of complex physical and chemical phenomena in the molten pool, the heat transfer influence of convection, conduction, evaporation and radiation and the combined action of a plurality of interfacial forces cause the flow in the molten pool to be complex, and the establishment of quantitative or qualitative relation between the dynamic molten pool appearance and the welding quality is difficult.
And secondly, the welding quality is evaluated and traced on line by monitoring the dynamic behavior of the molten pool, the method is particularly significant in the aspects of perfecting a product full life cycle quality management system and the like, the welding defect is monitored in real time, and the method has important value in improving the welding production efficiency. However, the existing welding process monitoring technology and system have only basic molten pool observation capability or only basic image processing capability, do not have large-scale data real-time analysis processing and machine learning capability, and only collect and record welding process parameter data, and are rarely related to the diagnosis of welding quality and welding defects.
And thirdly, the existing molten pool monitoring system has the problem of data isolated island, data interconnection and intercommunication and dynamic closed loop in the welding process cannot be realized, and efficient and rapid iterative updating and data sharing multiplexing of a new model are not supported.
Through the search of the prior art documents and patents, the chinese invention patent with patent application number 20151026126.9, a method for designing a big data driven cloud robot, discloses a method for designing a big data driven cloud robot, and realizes the remote control of the robot according to the design idea of "thin terminal (robot terminal) + fat network (cloud platform)". The invention patent of China invention with the patent application number of 201610288460.8 discloses a welding molten pool dynamic process on-line monitoring system and method, which adopts the monitoring system to realize the on-line monitoring of welding defects and can quickly adjust the welding process and the motion parameters according to the defect information. Chinese patent application No. 201811032020.1, "cloud robot interaction method based on cloud platform, cloud robot and cloud platform", discloses a cloud robot interaction method based on cloud platform, cloud robot and cloud platform, which realizes information interaction with the cloud platform through a transmission control protocol/internet protocol. The invention discloses a multi-information acquisition monitoring system and a method for a robot welding process, which are invented in Chinese patent with the patent application number of 201910044303.6, and realizes real-time diagnosis and prediction of welding quality and wireless remote access of welding process information such as welding voltage, current and the like through a wireless local area network technology. However, the data processing, model training and other time-consuming and complex tasks in the system are still performed at the equipment end, the equipment hardware limits the real-time performance of welding quality diagnosis and the real-time analysis and processing capability of large-scale data, and the adopted wireless local area network technology has transmission distance limitation, so that the remote online adjustment of welding parameters and the remote deployment and updating of models cannot be met. The technical scheme does not relate to welding process monitoring and quality diagnosis technology based on machine vision and edge intelligence.
In summary, most of the existing welding process monitoring and quality diagnosis technologies at home and abroad only relate to welding quality monitoring based on an expert database, welding process information access based on short-distance wireless transmission, welding bead automatic tracking technology, analysis processing of welding data at an equipment end and welding process monitoring, and a welding process monitoring and quality diagnosis method based on machine vision and edge intelligence is not reported in public at present.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a welding process monitoring and quality diagnosis method based on machine vision and edge intelligence so as to realize the visual and intelligent monitoring of the welding manufacturing process of a large metal structure robot.
In order to achieve the purpose, the invention adopts the following technical scheme:
a welding process monitoring and quality diagnosis method based on machine vision and edge intelligence comprises the following steps:
1) the industrial edge cloud server downloads and deploys the welding quality artificial intelligence prediction model from the cloud artificial intelligence computing server; respectively adjusting the angles of a light source and a lens of the molten pool vision monitoring unit to enable the light source and the lens to be respectively aligned to the areas to be welded; setting the frame number of the continuous acquisition image sequence; the welding robot starts welding after receiving an arc starting command sent by the equipment controller;
2) the molten pool visual monitoring unit starts to acquire molten pool images after receiving an image acquisition instruction sent by the equipment controller; after transmission pretreatment of each frame of acquired molten pool image, uploading the image to an industrial edge cloud server through a broadband mobile communication network for image pretreatment, and then performing image edge storage and output;
3) after the continuous collection image sequence is collected, the industrial edge cloud server sequentially performs image processing and image analysis on the molten pool image sequence output in the step 2), and then extracts and outputs characteristic vectors describing the transient morphology of the molten pool and the dynamic behavior of the molten pool; classifying and labeling the weld pool image sequence by adopting the artificial intelligent welding quality prediction model in the step 1); uploading the molten pool image judged to be abnormal and the annotation information thereof to a cloud artificial intelligence calculation server;
4) the equipment controller judges whether the current welding operation task of the welding robot is finished or not by a program, if the current welding operation task of the welding robot is finished, an arc extinguishing instruction is sent to the welding robot, and the cloud artificial intelligence computing server sequentially performs data preparation, model training, model testing, iterative model updating and cloud deployment on data received in the current welding operation task; if not, returning to the step 2), and repeating the steps 2) to 4).
In the technical scheme, the welding quality artificial intelligence prediction model in the step 1) is trained by a machine learning algorithm or a light-weight deep learning algorithm based on feature engineering, is trained in advance by a cloud artificial intelligence computing server and is issued to an industrial edge cloud server.
In the above technical solution, the 5G/6G industrial module is integrated in the molten pool vision monitoring unit in the step 2), and the broadband mobile communication network in the step 3) is a 5G/6G network.
In the above technical solution, the image processing in step 2) includes image noise reduction and molten pool contour enhancement.
In the above technical solution, the image edge storage in step 2) adopts a time sequence database technology, so as to achieve storage space saving and high-speed read-write operation; the image edge storage adopts JPEG 75% quality compressed image, which accounts for 1-10% of the total image acquisition amount.
In the above technical solution, the feature vector in step 3) is composed of spatial domain feature values of each frame of molten pool image and temporal domain feature values of image sequence.
In the above technical solution, the labeling information in step 3) at least includes a defect type and a defect generation time.
In the above technical solution, the data preparation in step 4) includes data preprocessing, data labeling verification, and data enhancement.
In the technical scheme, the data preprocessing comprises the steps of cleaning data, screening out misjudged images and storing new defect images into a typical defect image database.
In the above technical solution, the data labeling adopts manual labeling or semi-automatic labeling.
The invention has the following advantages and prominent technical effects: according to the invention, a molten pool image end side sensing and broadband mobile communication network data transmission technology is adopted, edge cloud reasoning is realized based on a machine learning model, and online conjecture and intelligent quality diagnosis of the welding state are realized by combining a cloud service cooperation mode, so that on one hand, the position of a potential defect can be pointed out for post-welding nondestructive detection, and the detection efficiency is obviously improved; on the other hand, the problem that the traditional nondestructive testing technology cannot acquire, record and mine the state and quality information of the welding and manufacturing process of the product can be solved, the on-site multi-station welding process can be continuously shot and stored for a very long time, the calculation cost at the end side is greatly saved, traceable data are provided for on-line comprehensive judgment and abnormal reason analysis of welding quality, process optimization and personnel training, and the quality control and tracing mechanism of the whole life cycle of the product is perfected; the problem of data isolated island of each field-end molten pool monitoring system on a production line can be solved, data interconnection and intercommunication and dynamic closed loop in the welding process are realized, efficient and rapid iterative updating and data sharing multiplexing of a new model are supported, and the intelligent level of a welding manufacturing system is improved; the cloud can be supported to perform operation maintenance such as software updating, equipment debugging, function expansion and the like, and the system maintenance cost is greatly reduced; in addition, the method has the advantages of simplicity and easiness in deployment, can reduce the winding loss of the cable to the maximum extent, and provides an effective scheme for solving the problem of scale application of the online monitoring and diagnosis of the welding process and the quality. The invention can be applied to the robot or automatic welding production process, and is particularly suitable for application scenes with equipment mobility requirements such as industrial robot welding and mobile robot welding operation facing long welding seams of large-scale structural members.
Drawings
FIG. 1 is a block diagram of a process flow for welding process monitoring and quality diagnostics based on machine vision and edge intelligence.
Fig. 2 is a schematic structural diagram of a welding process monitoring and quality diagnosing system according to an embodiment of the present invention.
In the figure: 1-a device controller; 2-a welding robot; 3-a molten pool visual monitoring unit; 4-broadband mobile communication network; 5-an industrial edge cloud server; 6, a cloud artificial intelligence computing server; 7-welding quality artificial intelligence prediction model; 8-5G/6G industrial modules; 9-a human-computer interaction module; 10 — a light source 10; 11-a lens; 12-database.
FIG. 3 is an image of a weld puddle in accordance with an embodiment of the present invention.
In the figure: 13-welding direction; 14-tungsten electrode; 15-welding pool; 16-tungsten electrode reflection; 17-welding pool rectangularity; 18-weld pool area.
Fig. 4 is a block diagram of a specific process of training and iteratively updating an artificial intelligence prediction model according to an embodiment of the present invention.
Detailed Description
The principles and operation of the present invention will be described in further detail below with reference to the accompanying drawings and examples.
Fig. 2 is a schematic structural diagram of a welding process monitoring and quality diagnosing system according to an embodiment of the present invention, where the system includes a device controller 1, a welding robot 2, a molten pool vision monitoring unit 3, a broadband mobile communication network 4, an industrial edge cloud server 5, a cloud artificial intelligence computing server 6, a welding quality artificial intelligence prediction model 7, a 5G/6G industrial module 8, a human-computer interaction module 9, a light source 10, a lens 11, and a database 12; the equipment controller 1 is connected with the welding robot 2 through an Ethernet, and the molten pool vision monitoring unit 3 is communicated with an industrial edge cloud server 5 through a broadband mobile communication network 4; a 5G/6G industrial module 8 is integrated in the molten pool visual monitoring unit, and the broadband mobile communication network is a 5G/6G network; the industrial edge cloud server 5 is communicated with a cloud artificial intelligence computing server 6 through a broadband mobile communication network 4; the welding quality artificial intelligence prediction model 7 is trained and iteratively updated based on historical data of the database 12 and current task data of the industrial edge cloud server 5; the human-computer interaction module 9 is connected with the industrial edge cloud server 5 through the Ethernet to visualize the welding process data and the quality diagnosis result in real time; the light source 10 is arranged in front of the welding gun and forms an angle of 60 degrees with the axial position of the welding gun; the lens 11 is arranged behind the welding gun together with the vision sensor, forms an angle of 60 degrees with the axial position of the welding gun, and keeps fixed with the position of the welding gun.
Fig. 1 is a flow chart of a welding process monitoring and quality diagnosis method based on machine vision and edge intelligence according to the present invention, which includes the following steps:
1) the industrial edge cloud server 5 downloads and deploys the welding quality artificial intelligence prediction model 7 from the cloud artificial intelligence calculation server 6; the welding quality artificial intelligence prediction model 7 is trained by a machine learning algorithm based on characteristic engineering or a light-weight deep learning algorithm, is trained in advance by the cloud artificial intelligence computing server 6 and is issued to the industrial edge cloud server 5; in this embodiment, the welding quality artificial intelligence prediction model 7 is trained according to an error inverse propagation algorithm by using principal component analysis feature dimension reduction and combining with a multi-layer feedforward neural network. Respectively adjusting the angles of the light source 10 and the lens 11 of the molten pool vision monitoring unit 3, so that the light source 10 and the lens 11 respectively align at preset angles to areas to be welded; setting the frame number of the continuous acquisition image sequence of the molten pool vision monitoring unit 3, and setting the molten pool image acquisition frame rate and the exposure time; the number of frames is not less than 1 and not more than fcontinuous collection≦ 5, in this embodiment, the number of frames f continuous collection4; sending an arc starting instruction to the welding robot 2 through the equipment controller 1, and starting welding by the welding robot 2;
2) the device controller 1 sends an image acquisition instruction to the molten pool vision monitoring unit 3, the molten pool vision monitoring unit 3 starts to acquire a molten pool image, the molten pool image is shown in fig. 3 in the embodiment of the invention, a welding gun advances along the welding direction 13, and the acquired molten pool image also comprises the tungsten electrode 14 and the tungsten electrode reflection 16 besides the welding molten pool 15; after transmission pretreatment such as region-of-interest extraction and image compression is carried out on each frame of collected molten pool images, the images are uploaded to the industrial edge cloud server 5 through the broadband mobile communication network 4 to carry out image noise reduction and molten pool contour enhancement, and then image edge storage and output are carried out; the image edge storage adopts a time sequence database technology to realize storage space saving and high-speed read-write operation; the image edge storage adopts JPEG 75% quality compressed image, which accounts for 1-10% of the total image acquisition amount;
3) after the continuous collection of the image sequence is completed, the industrial edge cloud server 5 sequentially performs image processing and image analysis on the molten pool image sequence output in the step 2), outputs a current frame molten pool image processing result in a human-computer interaction module, and then extracts, outputs and stores a characteristic vector for describing the transient morphology of the molten pool and the dynamic behavior of the molten pool, wherein the characteristic vector is composed of a spatial characteristic value and an image sequence time domain characteristic value of each frame of molten pool image; in the embodiment of the invention, the change values of the welding pool rectangularity 17 and the welding pool area 18 along with time jointly form a characteristic vector for describing the transient morphology of the welding pool and the dynamic behavior of the welding pool; the time domain curves and the statistical analysis results of the characteristic values are synchronously subjected to data visualization output in a man-machine interaction module, and the time domain curves and the statistical analysis results of the characteristic values are stored in the industrial edge cloud server 5; classifying and labeling the weld pool image sequence by adopting the artificial intelligent welding quality prediction model 7 in the step 1); uploading the molten pool image judged to be abnormal and the annotation information thereof to the cloud artificial intelligence calculation server 6; the marking information at least comprises a defect type and a defect generation time; in this embodiment, the molten pool image sequence is classified in two steps: detecting an abnormal image, labeling the abnormal image, predicting a weld joint area where a suspected defect is located by combining a robot kinematics model according to the frame number and interval time of the abnormal image, and performing data visualization output on a prediction result in a man-machine interaction module; identifying abnormal images or defect types to which the abnormal images belong, and performing information management and man-machine interaction operation on identification results;
4) the equipment controller 1 judges whether the current welding operation task of the welding robot 2 is finished or not by a program, if the current welding operation task is finished, an arc extinguishing instruction is sent to the welding robot 2, and the cloud artificial intelligence computing server 6 sequentially performs data preparation, model training, model testing, iterative model updating and cloud deployment on data received in the welding operation task; if not, returning to the step 2), and repeating the steps 2) to 4); the data preparation comprises four steps of data preprocessing, data marking verification and data enhancement; the data preprocessing comprises the steps of cleaning data, screening out misjudged images and storing new defect images into a typical defect image database 12 of the cloud artificial intelligence computing server 6; the data marking adopts manual marking or semi-automatic marking; the model training comprises the following steps: determining the type of a training data set of the welding quality artificial intelligence prediction model 7; collecting marked training data; thirdly, splitting the data set; determining input features of the training data set; determining a proper algorithm of the training data set; executing an algorithm on the training data set; in this embodiment, the model effect evaluation of the model test adopts a chaotic matrix, a 0-1 loss, an ROC curve or an accuracy, recall and F measurement method.
Fig. 4 is a block diagram illustrating a specific process of training and iteratively updating an artificial intelligence prediction model according to an embodiment of the present invention, which includes the following steps:
1) the stored historical data is recalled from the database 12 as a training data set for the artificial intelligence prediction model 7.
2) The training data set with the labeled information is divided into a training set, a verification set and a test set.
3) Judging the type of the artificial intelligence prediction model 7, and if the type is a light-weight deep learning algorithm, executing the step 4); if the machine learning model is based on the feature engineering, step 5) is executed.
4) And carrying out ROI extraction and image enhancement pretreatment on the molten pool image of the training data set.
5) And performing image analysis and image processing, extracting corresponding characteristic vectors, and performing preprocessing such as data cleaning, data dimension reduction and the like on the characteristic vectors.
6) And training the artificial intelligent prediction model 7 by taking the preprocessed molten pool image/feature vector as model input.
7) The model performance is evaluated by chaotic matrix, 0-1 loss, ROC curve or by accuracy, recall and F metric methods.
8) Deploying the trained artificial intelligence prediction model 7 to the industrial edge cloud server 5.
9) And performing welding process monitoring and real-time quality diagnosis tasks based on the artificial intelligence prediction model 7.
10) After the welding task is finished, the abnormal image/feature vector data collected in the welding task is marked and uploaded to the cloud artificial intelligence computing server 6 through the broadband mobile communication network 4 to iteratively update the artificial intelligence prediction model 7.
11) And marking the abnormal image data collected in the welding task and uploading the abnormal image data to the typical defect image database 12 through the broadband mobile communication network 4 for storage.
12) And the industrial edge cloud server 5 downloads and deploys the updated welding quality artificial intelligence prediction model 7 from the cloud artificial intelligence calculation server 6, and executes the next welding process monitoring and real-time quality diagnosis tasks.
Claims (10)
1. A welding process monitoring and quality diagnosis method based on machine vision and edge intelligence is characterized by comprising the following steps:
1) the industrial edge cloud server downloads and deploys the welding quality artificial intelligence prediction model from the cloud artificial intelligence computing server; respectively adjusting the angles of a light source and a lens of the molten pool vision monitoring unit to enable the light source and the lens to be respectively aligned to the areas to be welded; setting the frame number of the continuous acquisition image sequence; the welding robot starts welding after receiving an arc starting command sent by the equipment controller;
2) the molten pool visual monitoring unit starts to acquire molten pool images after receiving an image acquisition instruction sent by the equipment controller; after transmission pretreatment of each frame of acquired molten pool image, uploading the image to an industrial edge cloud server through a broadband mobile communication network for image pretreatment, and then performing image edge storage and output;
3) after the continuous collection image sequence is collected, the industrial edge cloud server sequentially performs image processing and image analysis on the molten pool image sequence output in the step 2), and then extracts and outputs characteristic vectors describing the transient morphology of the molten pool and the dynamic behavior of the molten pool; classifying and labeling the weld pool image sequence by adopting the artificial intelligent welding quality prediction model in the step 1); uploading the molten pool image judged to be abnormal and the annotation information thereof to a cloud artificial intelligence calculation server;
4) the equipment controller judges whether the current welding operation task of the welding robot is finished or not by a program, if the current welding operation task of the welding robot is finished, an arc extinguishing instruction is sent to the welding robot, and the cloud artificial intelligence computing server sequentially performs data preparation, model training, model testing, iterative model updating and cloud deployment on data received in the current welding operation task; if not, returning to the step 2), and repeating the steps 2) to 4).
2. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: the welding quality artificial intelligence prediction model in the step 1) is trained by a machine learning algorithm or a light-weight deep learning algorithm based on characteristic engineering, is trained in advance by a cloud artificial intelligence computing server and is issued to an industrial edge cloud server.
3. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: the 5G/6G industrial module is integrated in the molten pool visual monitoring unit in the step 2), and the broadband mobile communication network in the step 3) is a 5G/6G network.
4. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: and the image processing in the step 2) comprises the steps of image noise reduction and molten pool contour enhancement.
5. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: the image edge storage in the step 2) adopts a time sequence database technology to realize storage space saving and high-speed read-write operation; the image edge storage adopts JPEG 75% quality compressed image, which accounts for 1-10% of the total image acquisition amount.
6. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: and 3) the characteristic vector in the step 3) is composed of space domain characteristic values and image sequence time domain characteristic values of each frame of molten pool image.
7. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: the marking information in the step 3) at least comprises the defect type and the defect generation time.
8. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 1, wherein: and 4) the data preparation in the step 4) comprises data preprocessing, data annotation verification and data enhancement.
9. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 8, wherein: the data preprocessing comprises the steps of cleaning data, screening out misjudged images and storing new defect images into a typical defect image database.
10. The welding process monitoring and quality diagnostic method based on machine vision and edge intelligence as claimed in claim 8, wherein: and the data marking adopts manual marking or semi-automatic marking.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111529178.1A CN114167830A (en) | 2021-12-14 | 2021-12-14 | Welding process monitoring and quality diagnosis method based on machine vision and edge intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111529178.1A CN114167830A (en) | 2021-12-14 | 2021-12-14 | Welding process monitoring and quality diagnosis method based on machine vision and edge intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114167830A true CN114167830A (en) | 2022-03-11 |
Family
ID=80486539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111529178.1A Withdrawn CN114167830A (en) | 2021-12-14 | 2021-12-14 | Welding process monitoring and quality diagnosis method based on machine vision and edge intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114167830A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117086519A (en) * | 2023-08-22 | 2023-11-21 | 江苏凯立达数据科技有限公司 | Networking equipment data analysis and evaluation system and method based on industrial Internet |
-
2021
- 2021-12-14 CN CN202111529178.1A patent/CN114167830A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117086519A (en) * | 2023-08-22 | 2023-11-21 | 江苏凯立达数据科技有限公司 | Networking equipment data analysis and evaluation system and method based on industrial Internet |
| CN117086519B (en) * | 2023-08-22 | 2024-04-12 | 京闽数科(北京)有限公司 | Networking equipment data analysis and evaluation system and method based on industrial Internet |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4132742A1 (en) | In-situ inspection method based on digital data model of weld | |
| CN107471028B (en) | Grinding method and system | |
| CN109676135A (en) | A kind of laser gain material manufacture vision grey value difference on-line monitoring and bug repairing apparatus | |
| CN111199072B (en) | All-aluminum vehicle body riveting system based on online process library self-learning and implementation method thereof | |
| CN111299763A (en) | Anti-noise-interference laser visual welding seam automatic tracking method and system | |
| CN114167830A (en) | Welding process monitoring and quality diagnosis method based on machine vision and edge intelligence | |
| CN116186946B (en) | Hydraulic system fault diagnosis method and system based on diagnosis model | |
| CN114453709A (en) | Robot welding site intelligent monitoring system based on edge calculation | |
| CN108067714A (en) | A kind of thin-walled circumferential weld termination quality on-line monitoring and defect positioning system and method | |
| CN111426699A (en) | Boiler heating surface appearance defect online detection and identification device and method | |
| CN113570571A (en) | Industrial edge end power battery defect detection method and system | |
| Wang et al. | Online detection of weld surface defects based on improved incremental learning approach | |
| CN113298806A (en) | Intelligent detection method and device for welding defects, welding equipment and storage medium | |
| CN114083091A (en) | Narrow and deep space internal welding early warning monitoring method based on cloud edge and end cooperation | |
| CN112801984B (en) | Weld joint positioning method based on countermeasure learning under laser vision system | |
| Pasinetti et al. | In-line monitoring of laser welding using a smart vision system | |
| CN207528206U (en) | A kind of guide rod detection device | |
| CN206132666U (en) | X -ray production apparatus flexible electric actuator of machinery and because device's detecting system | |
| CN117409332B (en) | Long wood shaving appearance data detection system and method based on big data processing | |
| CN219121327U (en) | Online sorting system for anode guide rods | |
| CN117733439B (en) | Welding robot with weld joint forming quality real-time prediction function | |
| CN113421252B (en) | Improved convolutional neural network-based vehicle body welding defect detection method | |
| CN117620345B (en) | Data recording system of vacuum reflow oven | |
| CN117269168B (en) | New energy automobile precision part surface defect detection device and detection method | |
| CN116778152A (en) | Roof intelligent infrared camera photovoltaic module fault identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220311 |