CN113579545A - Intelligent self-decision-making molten pool monitoring system - Google Patents
Intelligent self-decision-making molten pool monitoring system Download PDFInfo
- Publication number
- CN113579545A CN113579545A CN202110952487.3A CN202110952487A CN113579545A CN 113579545 A CN113579545 A CN 113579545A CN 202110952487 A CN202110952487 A CN 202110952487A CN 113579545 A CN113579545 A CN 113579545A
- Authority
- CN
- China
- Prior art keywords
- molten pool
- image
- decision
- welding
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 23
- 238000003466 welding Methods 0.000 claims abstract description 96
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 19
- 230000009466 transformation Effects 0.000 claims abstract description 10
- 230000002787 reinforcement Effects 0.000 claims abstract description 7
- 230000003993 interaction Effects 0.000 claims abstract description 5
- 230000009471 action Effects 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 14
- 230000002194 synthesizing effect Effects 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 4
- 238000011217 control strategy Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 230000000638 stimulation Effects 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 abstract description 3
- 238000003908 quality control method Methods 0.000 abstract description 2
- 238000013486 operation strategy Methods 0.000 abstract 1
- 239000011159 matrix material Substances 0.000 description 11
- 230000006870 function Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000003786 synthesis reaction Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000010891 electric arc Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000008571 general function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000001308 synthesis method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B23—MACHINE TOOLS; METAL-WORKING NOT OTHERWISE PROVIDED FOR
- B23K—SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM
- B23K31/00—Processes relevant to this subclass, specially adapted for particular articles or purposes, but not covered by only one of the preceding main groups
- B23K31/12—Processes relevant to this subclass, specially adapted for particular articles or purposes, but not covered by only one of the preceding main groups relating to investigating the properties, e.g. the weldability, of materials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30152—Solder
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Mechanical Engineering (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent self-decision molten pool monitoring system which comprises a front-end sensing module, a molten pool image processing module, a molten pool state analysis module, a decision learning module and a control module. The invention designs hardware image acquisition equipment required to synchronously acquire high dynamic range images of a molten pool image, integrates and realizes a gradient image reconstruction method based on Fourier transformation, utilizes the molten pool state and welding parameters represented by molten pool depth characteristics, takes the molten pool state as signal feedback, performs SAC reinforcement learning, takes the maximum target image matching degree as a target, can imitate trial and error experience learning of manual welding, performs optimal welding strategy learning of interaction of a welding operation strategy and a molten pool state environment, is suitable for online monitoring of fusion welding quality control under different parameter environments, and can execute a welding strategy of optimal welding quality.
Description
Technical Field
The invention relates to the technical field of welding line control, in particular to an intelligent self-decision molten pool monitoring method.
Background
Welding plays an indispensable role in modern industrial manufacturing production, is an important business process, and the judgment of welding quality is also important for the control of the whole welding process. An important effective means for reflecting welding quality is to judge the state of a molten pool during welding, the existing information for representing the state of the molten pool is various, and visual information for sensing the state of the molten pool in a non-contact mode can intuitively and effectively reflect the state of the molten pool in the welding process, so that the visual monitoring of the fusion welding process based on the visual information is one of the future development directions.
In the welding process, a welder usually needs to estimate the welding quality according to the state information of a molten pool, an electric arc and the like and the past experience, so that the parameters are adjusted to realize real-time welding control and ensure the welding quality. Namely, in long-term welding operation, a welder forms a set of experience through continuously adjusting strategies according to the molten pool state during welding and the welding quality evaluation after welding in the molten pool state, so that the high-grade welder can judge the welding quality level in advance by observing the molten pool state. However, under the driving of the development of modern automation and intelligent industrial manufacturing, the intelligent and automatic welding equipment gradually replaces manual welding, and efficient and stable welding operation of additive manufacturing is realized. Therefore, an intelligent molten pool monitoring system is urgently needed in the field of welding quality on-line sensing and intelligent control, molten pool monitoring in the welding process of automatic welding equipment such as a welding robot can be realized, welding quality can be timely evaluated according to the molten pool state, and parameters are adjusted to realize welding quality control.
Disclosure of Invention
The invention aims to provide an intelligent self-decision molten pool monitoring system.
In order to achieve the purpose, the technical scheme of the invention is as follows:
an intelligent self-decision molten pool monitoring system comprises a front end sensing module, a molten pool image processing module, a molten pool state analysis module, a decision learning module and a control module.
The front-end sensing module is used for collecting different light and shade image information in a molten pool and preprocessing the light and shade image information.
And the molten pool image processing module is used for synthesizing the acquired and processed image data into a molten pool high dynamic range image so as to display the detailed characteristics of the brightness of the molten pool.
And the molten pool state analysis module is used for extracting the characteristics of the obtained molten pool image, acquiring a molten pool state data set to form input parameters, and giving out rewarding stimulation of the action of the welding machine according to the target molten pool state.
The decision learning module is used for outputting an execution strategy according to the molten pool state and the welding machine parameters, receiving a molten pool state feedback reward signal, autonomously judging the quality of molten pool change caused by outputting the welding strategy, and executing a strategy capable of generating the maximum value in the current state.
The control module is used for receiving the control strategy of the decision learning module and controlling the welding machine to execute corresponding welding operation according to the strategy.
Preferably, the front-end sensing module adopts a hardware camera device suitable for the molten pool image processing module, and the hardware camera device comprises a power supply system, a clock system, a protection system, an image sensor and an HDR image signal processor.
The power supply system is used for outputting voltages of all levels to provide power for hardware, outputting the voltages of all levels through the switching power supply or the low-dropout linear voltage-stabilized power supply, adapting to the standard level of each chip, meeting the power-on and power-off time sequence and ensuring the normal work of each hardware part.
The clock system provides internal clock signals for other hardware parts and is used as control signals for the coordination work of all parts in the chip.
The protection system is used for protecting the damage of hardware, and has the main functions of informing HOST when a certain part of the hardware is short-circuited or the temperature of an internal chip is overhigh, and taking corresponding measures such as: cutting off the power supply; stopping the chip to work, protecting the safety of the device, and resuming the normal work when the fault is removed.
The image sensor is used for collecting image information and transmitting the information to the HDR image signal processor.
The HDR image signal processor is used to synthesize a molten pool HDR image and perform advanced noise reduction.
Preferably, the image sensor is controlled by two shutter pointers which control integration of odd and even line pairs in a single frame of data, capture the odd and even line pairs of the single frame of data with the shutter pointers at different times, and then integrate and output the frame.
Preferably, the HDR image signal processor is communicated with the molten pool state analysis module, the decision learning module and the control module through a two-wire system synchronous serial bus.
Preferably, the molten pool image processing module synthesizes a high dynamic range image of the molten pool by a gradient image reconstruction method based on Fourier transformation, can embody the brightness details of the molten pool image, and avoids the loss of characteristics caused by over-brightness or over-darkness of a local area due to higher molten pool temperature, thereby more accurately analyzing and processing the molten pool state.
Preferably, the step of synthesizing the high dynamic range image of the molten pool from the gradient reconstruction image method based on Fourier transform comprises the following steps: loading a high exposure picture and a low exposure picture, respectively calculating the gradient vector field of each picture, then taking the maximum gradient point of the partial derivative picture in the direction corresponding to the gradient vector field of each picture to synthesize a new gradient, obtaining an algorithm of a closed form solution by using Fourier transformation, and synthesizing a brightness range to obtain an expanded HDR image.
Preferably, the weld pool state data set obtained in the weld pool state analysis module includes weld pool state information and corresponding welding parameters.
Preferably, the decision learning module adopts deep reinforcement learning and learns the strategy through interaction between the welding machine and the state of the monitored molten pool.
Preferably, the decision learning module performs continuous trial and error action feedback according to the characteristics of the molten pool image, and performs decision control on the welding quality through a reward and punishment mechanism to form a set of optimal execution strategy in each state in the welding process.
The molten pool image processing module realizes a gradient image reconstruction method based on Fourier transformation, synthesizes a high dynamic range image of a molten pool, can embody the brightness details of the molten pool image, avoids losing characteristics due to over-brightness or over-darkness of a local area caused by higher temperature of the molten pool, more accurately analyzes and processes the molten pool state, can acquire and process an image with remarkable image characteristics by depending on a high dynamic camera designed by a front-end sensing module, monitors the molten pool state in real time, performs depth CNN image algorithm processing of the molten pool state analysis module, and extracts the depth characteristics of the molten pool image; the decision learning module adopts deep reinforcement learning, learns strategies from experiences through interaction of the machine and the monitored molten pool state, influences the environmental molten pool state and obtains feedback information through executing welding actions, and enables the machine to learn the response to the stimulus step by step through rewarding and punished stimuli, so that the action capable of making the maximum benefit according to the state is generated. And (3) adopting a mode of obtaining a sample and learning at the same time, synthesizing a molten pool image in a high dynamic range as sample input, updating the model and executing welding operation to obtain a new molten pool state, obtaining feedback according to the welding quality corresponding to the molten pool state, further updating the model, and continuously iterating until learning from experience of continuous welding, molten pool state change and welding quality feedback to form an optimal welding execution strategy. Training depth features of a high dynamic range synthetic image of a molten pool which is collected and preprocessed, realizing feature extraction, performing continuous trial and error action feedback according to the features, realizing decision control of welding quality by a reward and punishment mechanism, and forming a set of optimal execution strategies in each state in the welding process; the control module is an action execution mechanism and controls the welding machine to execute welding according to the strategy provided by the decision learning module.
Compared with the prior art, the invention has the beneficial effects that:
(1) the invention realizes the image reconstruction method from gradient based on Fourier transformation, synthesizes the high dynamic range image of the molten pool, and can avoid the loss of the detailed characteristics of the molten pool image due to the high temperature of the molten pool.
(2) The invention integrates the high dynamic image synthesis algorithm of the molten pool into the hardware of the front-end camera, can realize high-efficiency and rapid high dynamic range image acquisition of the molten pool, senses the state of the molten pool in real time and lays a foundation for follow-up intelligent decision learning.
(3) Compared with the method that the weld pool image is obtained by welding, the welding quality is judged in advance, and the condition that the welding is not good and can not be recovered possibly occurs is adopted, the weld pool state is monitored on line and decided in real time, the welding is carried out while monitoring is carried out, the weld pool state is obtained according to the weld pool image, and the welding is controlled to achieve good welding quality.
(4) The invention can realize welder-like intelligence by adopting reinforcement learning, realizes training and learning of a welding execution strategy from continuous trial and error experience of the molten pool state with the best welding quality, does not need to manually mark a sample, enables a machine to perform self deduction, and realizes the optimal decision by taking a high dynamic range synthetic molten pool image as a feedback state. The method can execute welding according to the current molten pool state decision, and can be suitable for automatically adjusting parameters according to the molten pool state under the condition of any welding parameter to achieve the best welding quality.
(5) The invention is suitable for different materials, and can form an optimal welding execution strategy according to different temperature environments and corresponding variable molten pool states and an empirical learning strategy.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention.
In the drawings:
FIG. 1 is a hardware block diagram of a front-end sensing module according to the present invention;
FIG. 2 is a schematic diagram of a front-end sensing module image sensor acquiring odd and even row pairs;
FIG. 3 is a flow chart of an algorithm for high dynamic range weld puddle image synthesis;
fig. 4 is a schematic diagram of the overall system framework.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
As shown in fig. 1-4, the present embodiment discloses an intelligent self-decision molten pool monitoring system for controlling a fusion welding machine to perform molten pool monitoring of high quality welding operation, which includes a front-end sensing module, a molten pool image processing module, a molten pool state analyzing module, a decision learning module, and a control module.
The front-end sensing module is used for collecting different light and shade image information in a molten pool and preprocessing the light and shade image information. And the molten pool image processing module is used for synthesizing the acquired and processed image data into a molten pool high dynamic range image so as to display the detailed characteristics of the brightness of the molten pool. The molten pool state analysis module is used for extracting the characteristics of the obtained molten pool image, acquiring a molten pool state data set to form input parameters, wherein the molten pool state data set specifically comprises molten pool state information and corresponding welding parameters, and simultaneously gives out rewarding stimulation of the action of the welding machine according to the target molten pool state. The decision learning module is used for outputting an execution strategy according to the molten pool state and the welding machine parameters, receiving a molten pool state feedback reward signal, autonomously judging the quality of molten pool change caused by outputting the welding strategy, and executing a strategy capable of generating the maximum value in the current state. The control module is used for receiving the control strategy of the decision learning module and controlling the welding machine to execute corresponding welding operation according to the strategy.
Specifically, the front-end sensing module adopts hardware camera equipment suitable for the molten pool image processing module, and the hardware camera equipment comprises a power supply system, a clock system, a protection system, an image sensor and an HDR image signal processor.
The power supply system is used for outputting voltages of all levels to provide power for hardware, outputting the voltages of all levels through the switching power supply or the low-dropout linear voltage-stabilized power supply, adapting to the standard level of each chip, meeting the power-on and power-off time sequence and ensuring the normal work of each hardware part.
The clock system provides internal clock signals for other hardware parts and is used as control signals for the coordination work of all parts in the chip.
The protection system is used for protecting the damage of hardware, and has the main functions of informing HOST when a certain part of the hardware is short-circuited or the temperature of an internal chip is overhigh, and taking corresponding measures such as: cutting off the power supply; stopping the chip to work, protecting the safety of the device, and resuming the normal work when the fault is removed.
The image sensor is used for collecting image information and transmitting the information to the HDR image signal processor. Image sensors enable HDR still and video applications through interleaved multi-exposure reading techniques. As shown in fig. 2, the HDR image sensor is controlled by two shutter pointers that control the integration of odd and even row pairs in a single frame of data. Odd and even line pairs of a single frame of data are captured at different times using a shutter pointer and then integrated to output a frame. The output data can be matched with the implemented Fourier transform-based reconstruction from gradient image algorithm.
The HDR image signal processor is used to synthesize a molten pool HDR image and perform advanced noise reduction. Wherein the processing of images and colors is implemented by a hardware logic encoded image stream processor. The HDR image signal processor and the molten pool state analysis module, the decision learning module and the control module of the main controller are communicated through a two-wire system synchronous serial bus to modify the control and state registers of the signal processor to meet the actual requirements of application.
Preferably, the molten pool image processing module synthesizes a high dynamic range image of the molten pool by a gradient image reconstruction method based on Fourier transformation, can embody the brightness details of the molten pool image, and avoids the loss of characteristics caused by over-brightness or over-darkness of a local area due to high temperature of the molten pool, thereby analyzing and processing the molten pool state more accurately.
In order to acquire low-order detail feature information of an image, a first derivative vector field, namely a gradient field, of the image is formed by using the first derivative, and features can be fused by reconstructing an original image from the gradient image to obtain a high-quality image. The gradient reconstruction problem is converted into an inhomogeneous Poisson solving problem by selecting a proper energy minimization model mode, the characteristic that a differential operator can be converted into complex multiplication through Fourier transformation is utilized, the gradient reconstruction algorithm is simplified, a closed form solution input into a gradient vector field can be obtained, and therefore the gradient image reconstruction can be realized rapidly and effectively, and the high-dynamic-range molten pool image synthesis is realized.
Energy minimization model for gradient field reconstruction, i.e. when a gradient field v ═ v has been obtained1,v2)TIn the process of reconstructing an image, an image of a gradient map which is the same as a previously obtained gradient map in the square integrable norm (two-norm) sense is searched, and the formula of an optimization problem is represented as follows:
wherein
(| | | purple hair)2Respectively, a gradient operator and a squared integrable norm on a two-dimensional rectangular region, wherein the meaning of the formula is to obtain a target solution u with the minimum two norms of the difference value between the target solution u and a given vector field v*。
The process of transforming the poisson equation is:
the energy general function is
Using in Hilbert space H2The dual operator of the upper gradient operator is the property of the negative divergence operator, and the first order variation of the formula (2) is:
where div is the divergence operator, expressed as
Delta is the Laplacian operator, denoted as
And it is easy to deduce that the laplacian has the property:
solution of u*Satisfying the condition that the first order variation calculated by the formula (3) is 0, and recording div v: v ═ v*Then, there are:
-2(△u*-v*)＝0 (4)
the Poisson (Poisson) equation is thus obtained: delta u*＝v*. Since the image is specifically processed, it can be assumed that the Newman boundary condition is satisfied.
Fourier transform is used to obtain closed form solution, and Fourier transform on the torus is used to perform spatial domain differential operator to change into frequency domain complex multiplication
Wherein ^ represents Fourier transform on the torus, ξ is a first component on a two-dimensional frequency domain, corresponding to x on a spatial domain, and a second component of the frequency domain corresponding to y on the spatial domain, and is represented by eta; i is a pure imaginary unit. Using this property, we apply an up-fourier transform to both sides of equation (4), approximating the constants, to yield:
-4π2(ξ2+η2)(u*)^＝(v*)^ (6)
implementing a decoupled differential operator to obtain u*The closed form of (c) is solved as follows:
where V is the inverse fourier transform on the torus.
Since the algorithm needs to be deployed on hardware equipment to realize operation, the derivation is to assume that the solution u is located on the torus and has a space H with two continuous weak derivatives2Implemented, using actual processing of digital imagesAnd (3) discrete matrix representation, in order to realize hardware programming to obtain a closed form solution efficiently, a coefficient matrix is formed by point-by-point division of the frequency domain obtained in the formula (7), when the size of an image to be reconstructed is determined, the coefficient matrix is also uniquely determined, when repeated reconstruction operation is needed, only one discrete coefficient matrix needs to be constructed, and matrix multiplication is called to be carried out during image reconstruction, so that the efficiency can be greatly improved. The coefficient matrix P is defined as:
where i and j are the abscissa and ordinate, respectively, in the discrete case.
Obtaining v with respect to the vector field on the divergence operator's contribution*The key is how to make differential representations of different directions on discrete scenes, using-101]The matrix is convolved with the image matrix to obtain partial derivatives in the y direction in the continuous case and the down direction in the discrete case, and the matrix is used for [ -101 ] similarly]TAnd performing convolution operation on the matrix and the image matrix to obtain partial derivatives in the x direction in the continuous case and the following directions in the discrete case. And (3) satisfying the programming realization of the formula (7) in a discrete case.
An HDR image is an image with a very wide luminance range, and it directly records luminance information, unlike a conventional image that compresses luminance information into an 8-bit or 16-bit color space. In general, when a computer is used for image representation, 8 bits (256 levels) are generally used for distinguishing image brightness, the brightness level range is too small to truly reflect the ambient brightness condition, the general image representation is very inaccurate, overexposure results in a white piece, details cannot be revealed, dimming only reduces the white pixel value, and details are not improved. If HDR image recording is used, the detail features will be presented. According to the high dynamic range molten pool image synthesis method, one or more pictures which are shot in the same scene and exposed differently are synthesized by using a gradient image reconstruction method based on Fourier transformation, as shown in figure 3, a high exposure picture and a low exposure picture are loaded, the gradient vector field of each picture is respectively calculated, then the maximum gradient point of the partial derivative picture in the direction corresponding to the gradient vector field of each picture is taken, new gradients with more details are synthesized, a closed form solution algorithm is obtained by using Fourier transformation, and the synthesized brightness range is greatly expanded to obtain an HDR picture. More weld pool image details are displayed through the HDR image, so that feature extraction and accurate weld pool state judgment are facilitated, and reliable basis is provided for accurate state feedback and welding strategy learning similar to continuous groping and rewarding mechanism feedback of people.
Furthermore, the decision learning module in the embodiment adopts deep reinforcement learning and learns the strategy through interaction between the welding machine and the state of the monitored molten pool. And the reinforcement learning SAC is adopted to enable the model to automatically explore how to adjust the welding parameters for the best welding quality, so that the welding quality in any state is ensured to the maximum extent. The welder carries out some actions through intelligent decision by monitoring the state of the molten pool, the actions further cause the state change of the molten pool, and the welder obtains feedback about previous actions from the changed state so as to guide subsequent action decision. The specific state of the molten pool in the optimal welding quality range is used as a target image, the molten pool image which is processed and synthesized is obtained by an image-based method and is used as a feedback state and a target image of a reward function, and the reward function is defined as the matching degree of a feature map of the current molten pool image and the target image after depth feature extraction. In addition, in order to improve the learning efficiency of the model algorithm, a supervised learning method simulating learning is added, and the motion flow is demonstrated for the monitoring system manually, so that the machine is in the optimization direction of system learning.
And the decision learning module performs continuous trial and error action feedback according to the characteristics of the molten pool image and performs decision control on the welding quality through a reward and punishment mechanism to form a set of optimal execution strategy in each state in the welding process. Specifically, the machine influences the state of the environmental molten pool and obtains feedback information by executing the welding action, and the machine gradually learns the response to the stimulus by rewarding and punishing the stimulus, so that the action of making the optimal welding quality according to the state is generated. And (3) adopting a mode of obtaining a sample and learning at the same time, synthesizing a molten pool image in a high dynamic range as sample input, updating the model and executing welding operation to obtain a new molten pool state, obtaining feedback according to the welding quality corresponding to the molten pool state, further updating the model, and continuously iterating until learning from experience of continuous welding, molten pool state change and welding quality feedback to form an optimal welding execution strategy.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be able to cover the technical scope of the present invention and the equivalent alternatives or modifications according to the technical solution and the inventive concept of the present invention within the technical scope of the present invention.
Claims (9)
1. An intelligent self-decision molten pool monitoring system is characterized by comprising a front-end sensing module, a molten pool image processing module, a molten pool state analysis module, a decision learning module and a control module;
the front-end sensing module is used for acquiring different light and shade image information in a molten pool and preprocessing the light and shade image information;
the molten pool image processing module is used for synthesizing the acquired and processed image data into a molten pool high dynamic range image so as to display the detailed characteristics of the brightness of the molten pool;
the molten pool state analysis module is used for extracting the characteristics of the obtained molten pool image, acquiring a molten pool state data set to form input parameters, and giving out rewarding stimulation of the action of the welding machine according to the target molten pool state;
the decision learning module is used for outputting an execution strategy according to the molten pool state and the welding machine parameters, receiving a molten pool state feedback reward signal, autonomously judging the quality of molten pool change caused by outputting the welding strategy, and executing a strategy capable of generating the maximum value in the current state;
the control module is used for receiving the control strategy of the decision learning module and controlling the welding machine to execute corresponding welding operation according to the strategy.
2. The intelligent self-decision molten pool monitoring system according to claim 1, characterized in that the front-end sensing module adopts a hardware camera device suitable for a molten pool image processing module, and the hardware camera device comprises a power supply system, a clock system, a protection system, an image sensor and an HDR image signal processor;
the power supply system is used for outputting voltages of all levels to provide power for hardware;
the clock system provides an internal clock signal for hardware as a control signal for coordination work;
the protection system is used for protecting the damage of hardware;
the image sensor is used for acquiring image information and transmitting the information to the HDR image signal processor;
the HDR image signal processor is used to synthesize a molten pool HDR image and perform advanced noise reduction.
3. The intelligent self-decision molten pool monitoring system according to claim 2, wherein the image sensor is controlled by two shutter pointers, the two shutter pointers control integration of odd and even row pairs in a single frame of data, the shutter pointers are used to capture the odd and even row pairs of the single frame of data at different times, and then the integration and output frames are performed.
4. The intelligent self-decision molten pool monitoring system as claimed in claim 3, wherein the HDR image signal processor communicates with the molten pool state analysis module, the decision learning module and the control module through a two-wire system synchronous serial bus.
5. The intelligent self-decision molten pool monitoring system of claim 1, wherein the molten pool image processing module adopts a Fourier transform-based method of reconstructing an image from gradients to synthesize a high dynamic range image of the molten pool.
6. The intelligent self-decision molten pool monitoring system according to claim 5, wherein the step of synthesizing the high dynamic range image of the molten pool from the gradient reconstruction image method based on Fourier transform comprises: loading a high exposure picture and a low exposure picture, respectively calculating the gradient vector field of each picture, then taking the maximum gradient point of the partial derivative picture in the direction corresponding to the gradient vector field of each picture to synthesize a new gradient, obtaining an algorithm of a closed form solution by using Fourier transformation, and synthesizing a brightness range to obtain an expanded HDR image.
7. The intelligent self-decision molten pool monitoring system according to claim 1, wherein the molten pool state data set obtained in the molten pool state analysis module comprises molten pool state information and corresponding welding parameters.
8. The intelligent self-decision molten pool monitoring system according to claim 1, wherein the decision learning module employs deep reinforcement learning to learn strategy through interaction of the welding machine itself and the state of the monitored molten pool.
9. The system of claim 1, wherein the decision learning module performs constant trial and error action feedback according to characteristics of the molten pool image, and performs decision control on the welding quality by a reward and punishment mechanism to form an optimal execution strategy in each state in the welding process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110952487.3A CN113579545A (en) | 2021-08-19 | 2021-08-19 | Intelligent self-decision-making molten pool monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110952487.3A CN113579545A (en) | 2021-08-19 | 2021-08-19 | Intelligent self-decision-making molten pool monitoring system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113579545A true CN113579545A (en) | 2021-11-02 |
Family
ID=78238251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110952487.3A Pending CN113579545A (en) | 2021-08-19 | 2021-08-19 | Intelligent self-decision-making molten pool monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113579545A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI795282B (en) * | 2022-04-29 | 2023-03-01 | 陳健如 | A robotic welding method |
-
2021
- 2021-08-19 CN CN202110952487.3A patent/CN113579545A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI795282B (en) * | 2022-04-29 | 2023-03-01 | 陳健如 | A robotic welding method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hu et al. | v2e: From video frames to realistic DVS events | |
| WO2021164234A1 (en) | Image processing method and image processing device | |
| CN112686928B (en) | Moving target visual tracking method based on multi-source information fusion | |
| CN112188088B (en) | Underwater self-photographing system | |
| CN113579545A (en) | Intelligent self-decision-making molten pool monitoring system | |
| WO2020228346A1 (en) | Transformer substation three-dimensional digital modeling method, system and device and storage medium | |
| CN204929109U (en) | Real -time image processing circuit | |
| CN215699047U (en) | Intelligent self-decision-making molten pool monitoring system | |
| CN117351216B (en) | Image self-adaptive denoising method based on supervised deep learning | |
| CN114298942A (en) | Image deblurring method and device, computer readable medium and electronic equipment | |
| CN117576153A (en) | Target tracking method and system | |
| Gehrig et al. | Video to events: Bringing modern computer vision closer to event cameras | |
| CN106162131A (en) | A kind of real time image processing | |
| CN113112482A (en) | PCB defect detection method based on attention mechanism network | |
| CN107004132A (en) | Eye tracking device and its auxiliary light source control method and relevant apparatus | |
| CN116612263A (en) | Method and device for sensing consistency dynamic fitting of latent vision synthesis | |
| Bhat et al. | Fast Under Water Image Enhancement for Real Time Applications | |
| CN105657259B (en) | A kind of two points of contractings step shifting method camera parameter adjustment method | |
| Zhang et al. | Single image dehazing via reinforcement learning | |
| KR101012758B1 (en) | Three-dimensional body measurement system and method | |
| Eisert et al. | Speech driven synthesis of talking head sequences | |
| CN116152128A (en) | High dynamic range multi-exposure image fusion model and method based on attention mechanism | |
| Nan et al. | An image enhancement method in coal mine underground based on deep retinex network and fusion strategy | |
| CN114029952A (en) | Robot operation control method, device and system | |
| CN112132923A (en) | Two-stage digital image style transformation method and system based on style thumbnail high-definition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |