KR20230003997A - A control system and method for melten pool recognition and penetration depth - Google Patents

Abstract

The present invention provides a molten pool recognition and penetration depth control system and method for extracting feature points for a plurality of first welding images by supervising and learning a plurality of labeled first welding images, matching the feature points for the plurality of first welding images to a plurality of additionally photographed second welding images to recognize and extract a second molten pool image, and according to the result of comparing the penetration depth of the second molten pool derived by artificial intelligence learning of the shape information measured from the second molten pool image and the preset penetration depth, controlling the current and welding speed of a welding part.

Description

Melten pool recognition and penetration depth control system and method {A control system and method for melten pool recognition and penetration depth}

The present invention relates to a system and method for recognizing a molten pool and controlling penetration depth, and more particularly, by accurately recognizing a molten pool from a welding image, deriving the penetration depth of the recognized molten pool, and then determining the penetration depth derived from the penetration depth and the preset penetration depth. It relates to a molten pool recognition and penetration depth control system and method for controlling the current and welding speed of a welded part according to the result of comparing the

Arc welding is to join at least two base materials by supplying heat necessary for welding through an electric arc generated between a welding rod and a part to be welded.

In general, poor penetration causes various defects. In this way, if the penetration control fails and a defect occurs, the time required to repair the defect is 5 to 10 times longer than the total welding time required for penetration.

Therefore, penetration control is very important in welding automation technology. Conventionally, as a welding automation technology, a technology that applies voltage signals, arc images, and digital welders according to welding waveform control has been developed. to be.

In order to solve the above problem, there is a technique of controlling penetration according to a measured voltage signal.

In the case of the conventional commercialized welding penetration control device, penetration is predicted by using the feature that the molten pool increases as the welding current increases and the vibration frequency due to the welding arc pressure decreases as the molten pool increases.

In the case of a molten pool where arc pressure acts, the molten pool vibrates up and down due to the pressure, and as a result, the length of the arc increases and decreases, and as the length of the arc increases and decreases, the voltage value due to the arc length varies.

In the case of a constant current welding machine, V (voltage) = I (current) R (resistance), and since the current is constant, when the arc length decreases, the resistance decreases (voltage decreases), and when the arc length increases, the resistance increases (voltage increases), so the voltage Penetration control can be performed by estimating the frequency through value measurement.

However, in the prior art described above, when the welding wire is additionally supplied, the above phenomenon is broken, and thus it is impossible to perform penetration control by predicting the frequency through voltage measurement, so there is a limit to accurate penetration control.

Accordingly, research and development on technology for controlling penetration using image processing technology is being actively conducted.

As shown in (a), (b), (c), and (d) of FIG. 1, the prior art using the above image processing controls each sensitivity for the algorithm used for image processing for each welding condition. variables need to be adjusted.

However, the prior art described above has a problem in that the number of sensitivity variables to be adjusted according to various welding condition environments is too large, so that it is not easy to adjust, and accordingly, the recognition accuracy of the molten pool is lowered.

(Patent Document 1) Patent Registration No. 10-2187764 (2020.12.01.)

(Patent Document 2) Registered Patent Publication No. 1 0-2155053 (2020.09.07.)

An object of the present invention for solving the above problems is that the image segmentation unit learns a plurality of first welding images taken by the image capturing unit to learn about the first arc light image, the first molten pool image, and the first background image. Recognizing and extracting a second molten pool image by extracting feature points and matching feature points for the first arc light image, the first molten pool image, and the first background image to a plurality of second welding images additionally photographed by the imaging unit It is to provide a melt pool recognition and penetration depth control system and method.

In addition, an object of the present invention for solving the above problem is that the shape information measuring unit measures the width (w), length (l), width (s) and width of the second image pool image based on the second melt pool image. (w) The length ratio (r) is measured, and the penetration depth derivation unit measures the width (w), length (l), area (s), width (w) to length ratio (r) and the thickness (t) of the base material by artificial It is to provide a melt pool recognition and penetration depth control system and method that derives the penetration depth (p) of the second melt pool image in real time by intelligent learning.

In addition, an object of the present invention for solving the above problem is to determine the current of the welding part 110 according to the result of the derived control unit comparing the penetration depth (p) of the second molten pool image and the preset penetration depth (p) And to provide a molten pool recognition and penetration depth control system and method for controlling the welding speed.

The technical problem to be achieved by the present invention is not limited to the above-mentioned technical problem, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below. There will be.

The configuration of the present invention for achieving the above object is a welding part for supplying heat generated by discharging the arc light to a junction where at least two base materials are in contact; A plurality of first welding images including a first arc light image, a first molten pool image formed at the junction, and a background image located around the first molten pool image are obtained in real time by photographing the tip of the welding portion. a video recording unit; The first arc light image, the first molten pool image, and the first background image manually labeled by the user are supervised and learned to obtain the first arc light image, the first molten pool image, and the first background image. Set feature points for the first arc light image, the first molten pool image, and the second molten pool from a plurality of second welding images additionally obtained from the imaging unit based on the feature points for the first background image. a deep learning unit that extracts an image, processes the second molten pool image, measures shape information of the second molten pool image, and derives a penetration depth by artificial intelligence learning of the measured shape information; And a control unit for controlling the current and welding speed of the welding part according to the result of comparing the derived penetration depth and the preset penetration depth. It provides a molten pool recognition and penetration depth control system comprising a.

In an embodiment of the present invention, the plurality of first welding images are big data for setting the feature point, which is a criterion for recognizing the second molten pool image, and the first arc light image composed of a plurality, the An image dataset composed of a first molten pool image and the first background image, and the deep learning unit performs supervised learning on the labeled first arc light image, the first molten pool image, and the first background image and an image segmentation unit configured to set feature points for the labeled first arc light image, the first molten pool image, and the first background image, wherein the image segmentation unit sets feature points for the first molten pool image as references. It may be characterized by matching the plurality of second welding images transmitted from the imaging unit to separate the second molten pool image from the plurality of second welding images.

In an embodiment of the present invention, the image segmentation unit applies a VGG 16 net-based Convolutional Neural Network (CNN) algorithm as an image segmentation technique, and the feature point for the first melt pool image is the first arc light It may be characterized by including an image and a unique brightness, saturation, color, and shape pattern of the first molten pool image different from the first background image.

In an embodiment of the present invention, the deep learning unit further comprises a shape information measuring unit for measuring shape information of the second molten pool image by image processing the second molten pool image transmitted from the image segmentation unit. The shape information measurement unit extracts only pixels corresponding to the second molten pool image, and then extracts a second molten pool boundary line connected by pixels located at the outermost part of the second molten pool image, and extracts the extracted second molten pool image. It may be characterized by measuring the width (w), length (l), width (s) and the width (w) to length ratio (r) of the second molten pool image based on the pool boundary line.

In an embodiment of the present invention, the shape information measuring unit applies an image processing algorithm to the width (w), length (l), width (s), width (w) to length ratio (w) for the second molten pool image shape information including r) and the thickness (t) of the base material is measured, and the width (w), length (l), width (s), width (w) to length ratio (r) of the second melt pool image And it may be characterized in that the thickness (t) of the base material is converted into feature vector data of a 5x1 matrix.

In an embodiment of the present invention, the deep learning unit artificial intelligence learns the feature vector data of the 5x1 matrix transmitted from the shape information measuring unit to derive the penetration depth (p) of the second melt pool image. It may be characterized in that it further comprises;

In an embodiment of the present invention, the penetration depth deriving unit transmits the penetration depth to the control unit, and the control unit controls the welding portion so that the welding portion maintains the current state when the penetration depth and the preset penetration depth are the same that can be characterized.

In an embodiment of the present invention, the penetration depth deriving unit transmits the penetration depth to the control unit, and the control unit controls the current and welding speed of the welding part when the penetration depth and the preset penetration depth are different can be done with

In an embodiment of the present invention, when the penetration depth is greater than the preset penetration depth, the control unit controls the welding portion to reduce the current of the welding portion or increase the welding speed of the welding portion. there is.

In an embodiment of the present invention, when the penetration depth is smaller than the preset penetration depth, the control unit controls the welding portion to increase the current of the welding portion or decrease the welding speed of the welding portion. there is.

In addition, an object of the present invention for solving the above problem is (a) a plurality of first welding images taken by the imaging unit are a first arc light image, a first molten pool image, and a first background image by a user Labeled as; (b) an image segmentation unit provided in the deep learning unit supervises and learns an image dataset in which the plurality of first welding images are labeled as the first arc light image, the first molten pool image, and the first background image; Setting feature points for the first arc light image, the first molten pool image, and the first background image; (c) transmitting a plurality of second welding images captured by the image capture unit to the image segmentation unit; (d) separating a second molten pool image from the plurality of second welding images based on feature points of the first arc light image, the first welding pool image, and the first background image by the image division unit; (e) measuring shape information of the second molten pool image based on the second molten pool image by a shape information measuring unit provided in the deep learning unit; (f) deriving a penetration depth by artificial intelligence learning of the shape information of the second molten pool image by the penetration depth derivation unit; And (g) determining whether or not to control the welding part according to the result of the control unit comparing the penetration depth transmitted from the penetration depth derivation unit and the preset penetration depth. Provides a depth control method.

In an embodiment of the present invention, the step (a) may include: (a1) acquiring the plurality of first welding images by photographing the tip of the welding portion by the image capture unit; (a2) labeling the first arc light image, the first molten pool image, and the first background image by the user; and (a3) transmitting the labeled first arc light image, the first molten pool image, and the first background image to the image division unit by the imaging unit.

In an embodiment of the present invention, the plurality of first welding images are big data for setting the feature point, which is a criterion for recognizing the second molten pool image, and the first arc light image composed of a plurality, the An image dataset composed of a first molten pool image and the first background image, and in step (b), (b1) the labeled first arc light transmitted from the image division unit to the image division unit Receiving an image, a first molten pool image, and a first background image; and (b2) the image segmentation unit supervises and learns the labeled first arc light image, the first molten pool image, and the first background image, and the labeled first arc light image, the first molten pool image, and the first background image. It may be characterized by including; setting a feature point for.

In an embodiment of the present invention, the step (c) may include: (c1) acquiring the plurality of second welding images by photographing the end of the welding part again by the image capture unit; (c2) transmitting the plurality of second welding images to the image dividing unit by the image capturing unit;

In an embodiment of the present invention, the step (d) may include: (d1) the image division unit receiving the plurality of second welding images transmitted from the image capture unit; (d2) The image segmentation unit matches the plurality of second welding images based on the feature points of the labeled first arc light image, first molten pool image, and first background image to obtain information from the plurality of second welding images. Separating the second molten pool image; and (d3) transmitting the second molten pool image to the shape information measurement unit by the image segmentation unit.

In an embodiment of the present invention, the step (e) may include: (e1) receiving the second molten pool image transmitted from the image dividing unit by the shape information measuring unit; (e2) extracting only pixels corresponding to the second molten pool image by the shape information measuring unit; (e3) extracting a second molten pool boundary where the outermost pixels of the second molten pool image are connected by the shape information measuring unit; (e4) The width (w), length (l), width (s) of the second molten pool image based on the extracted second molten pool boundary line by the shape information measuring unit, and the width (w) to length ratio ( and measuring r), wherein the shape information measurement unit applies an image processing algorithm to compare width (w), length (l), width (s), and width (w) of the second molten pool image. Shape information including the length ratio (r) and the thickness (t) of the base material is measured, and the ratio of length to width (w), length (l), width (s), and width (w) of the second molten pool image (r) and the thickness (t) of the base material may be converted into feature vector data of a 5x1 matrix.

In an embodiment of the present invention, in the step (f), (f1) the penetration depth derivation unit measures the width (w), length (l), and width of the second melt pool image transmitted from the shape information measurement unit. (s) receiving shape information including a width (w) to length ratio (r) and a base material thickness (t); (f2) The penetration depth derivation unit artificially calculates the width (w), length (l), width (s), width (w) to length ratio (r) and thickness (t) of the base material for the second molten pool image. Deriving the penetration depth for the second molten pool image by intelligent learning; and (f3) transmitting the penetration depth from the penetration depth deriving unit to the control unit.

In an embodiment of the present invention, the step (g) may include: (g1) receiving, by the control unit, the penetration depth transmitted from the penetration depth deriving unit; and (g2) determining, by the control unit, whether or not to control the welding part according to a result of comparing the penetration depth with a preset penetration depth.

In an embodiment of the present invention, after the step (g2), (h) the control unit controlling the current and the welding speed of the welding part; further comprising, in the (h) step, the penetration depth is When the penetration depth is greater than the predetermined penetration depth, the controller may control the welding part to reduce the current of the welding part or increase the welding speed of the welding part.

In an embodiment of the present invention, in the step (h), when the penetration depth is smaller than the predetermined penetration depth, the control unit controls the welding portion to increase the current of the welding portion or increase the welding speed of the welding portion. It may be characterized in that it further comprises the step of reducing.

The effect of the present invention according to the configuration as described above is that the image segmentation unit supervises and learns a plurality of first welding images taken by the image capturing unit, and the feature points for the first arc light image, the first molten pool image, and the first background image of the molten pool by extracting and matching feature points for the first arc light image, the first molten pool image, and the first background image to a plurality of second welding images additionally photographed by the imaging unit to extract the second molten pool image. The recognition rate can be improved by minimizing the recognition error.

In addition, the effect of the present invention according to the above configuration is that the shape information measuring unit measures the width (w), length (l), width (s) and width ( w) Measures the length ratio (r), and the penetration depth derivation unit measures the width (w), length (l), area (s), width (w) to length ratio (r) and the thickness (t) of the base material by artificial intelligence By learning and deriving the penetration depth (p) of the second molten pool image in real time, the working state of the arc welding can be monitored in real time.

In addition, the effect of the present invention according to the configuration as described above is the current and Working errors can be minimized by controlling the welding speed.

The effects of the present invention are not limited to the above effects, and should be understood to include all effects that can be inferred from the detailed description of the present invention or the configuration of the invention described in the claims.

1 (a), (b), (c), (d) are images showing welding images processed according to the prior art.
2 is a block diagram showing a melt pool recognition and penetration depth control system according to an embodiment of the present invention.
3 is a first welding including a first arc light image, a first molten pool image, and a first background image captured by an imaging unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention This is an image showing that the image has been manually labeled by the user.
(a), (b), and (c) of FIG. 4 are images showing a dataset composed of a plurality of first arc light images, a first molten pool image, and a first background image labeled in FIG. .
5 is a conceptual diagram illustrating a convolutional neural network (CNN) algorithm based on a VGG 16 net applied to an image segmentation unit provided in a melt pool recognition and penetration depth control system according to an embodiment of the present invention.
6 (a), (b), and (c) show a process of supervising and learning a plurality of first welding images by an image segmentation unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention. It is an image.
7 to 10 are diagrams showing results for each condition in the supervised learning process of the image segmentation unit provided in the molten pool recognition and penetration depth control system according to an embodiment of the present invention.
11 is a view showing a process of extracting shape information for a first molten pool image using an image processing algorithm in a shape information measurement unit provided in a melt pool recognition and penetration depth control system according to an embodiment of the present invention. .
12 is a diagram showing shape information obtained from a shape information measurement unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention and a penetration depth obtained from a penetration depth derivation unit.
13 is a diagram illustrating a process of deriving a penetration depth from a first melt pool image extracted from a system and method for recognizing a melt pool and controlling penetration depth according to an embodiment of the present invention.
14 is a flowchart illustrating a method for recognizing a molten pool and controlling a penetration depth according to an embodiment of the present invention.

Hereinafter, the present invention will be described with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and, therefore, is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected (connected, contacted, combined)" with another part, this is not only "directly connected", but also "indirectly connected" with another member in between. "Including cases where In addition, when a part "includes" a certain component, it means that it may further include other components without excluding other components unless otherwise stated.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1. Melt Pool Recognition and Penetration Depth Control System (100)

Hereinafter, a melt pool recognition and penetration depth control system according to an embodiment of the present invention will be described with reference to FIGS. 2 to 13.

2 is a block diagram showing a melt pool recognition and penetration depth control system according to an embodiment of the present invention.

The molten pool recognition and penetration depth control system 100 according to an embodiment of the present invention includes a welding unit 110, an image capturing unit 120, a deep learning unit 130 and a control unit 140.

The welding part 110 supplies heat generated by discharging the arc light to a junction where at least two base materials are in contact.

The welding part 110 for this may be a welding rod that transmits arc light to a junction where at least two base materials come into contact. In addition, a welding electrode holder for a user to grip is connected to the welding part 110 .

In addition, in order for the welding part 110 to operate, an electric line electrically connected to the welding rod holder, a grounding clamp electrically connected to the base material, a grounding line electrically connected to the grounding clamp, and an electric line and the grounding line are electrically connected and generate an arc light. An arc welder may be provided.

The ends of the welded part 110 arc-weld at least two base materials as they move along the joint in a state of being separated by a predetermined distance toward the joint where at least two base materials come into contact.

3 is a first welding including a first arc light image, a first molten pool image, and a first background image captured by an imaging unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention This is an image showing that the image has been manually labeled by the user. (a), (b), and (c) of FIG. 4 are images showing a dataset composed of a plurality of first arc light images, a first molten pool image, and a first background image labeled in FIG. .

The image capturing unit 120 photographs the end of the welding portion 110, and as shown in FIGS. 3 (a) and 4 (b), the first arc light image, the first molten pool image formed at the junction, and A plurality of first welding images including a background image located around the first molten pool image are acquired in real time.

Here, the plurality of first welding images are big data for setting feature points that are criteria for recognizing the second molten pool image, and are composed of a plurality of first arc light images, first molten pool images, and first background images. It is an image dataset that is constructed.

The obtained plurality of first welding images are manually labeled by the user as shown in (b) of FIG. 3 .

Specifically, as shown in FIGS. 3(b) and 4(a), the labeling marks the first arc light image as arc light, displays the first melt pool as a melt pool, and displays the first arc light image as a melt pool. By marking it as a background, it indicates what the image is about.

Here, the first arc light image, the first molten pool image, and the first background image are images for the deep learning unit 130 to derive and select feature points for the arc light, the molten pool, and the background through supervised learning. These are the plurality of first welding images first photographed by the photographing unit 120 .

The first welding image includes a first arc light image, a first molten pool image, and a first background image, as shown in (c) of FIG. 4, and is composed of a plurality of data sets.

On the other hand, the second arc light image, the second molten pool image, and the second background image to be described later are images to be recognized as molten pools in the present invention, and are captured by the image capture unit 110 and used by the deep learning unit 130. It is an image that is matched with feature points for the arc light, molten pool, and background.

The above image capture unit 120 transmits the labeled first arc light image, the first molten pool image, and the first background image to the image segmentation unit 131 provided in the deep learning unit 130, and then additionally second An arc light image, a second molten pool image, and a second background image are acquired and transmitted to the image segmentation unit 130 .

The deep learning unit 130 performs supervised learning on the first arc light image, the first molten pool image, and the first background image manually labeled by the user to obtain the first arc light image, the first molten pool image, and Extracting and setting feature points for the first background image, extracting second molten pool images from a plurality of second welding images additionally obtained from the imaging unit 120 based on the feature points for the first molten pool image, and 2 Image processing of the molten pool image measures the shape information of the second molten pool image, and artificial intelligence learns the measured shape information to derive the penetration depth.

The deep learning unit 130 for this includes an image segmentation unit 131, a shape information measurement unit 132, and a penetration depth derivation unit 133.

5 is a conceptual diagram illustrating a convolutional neural network (CNN) algorithm based on a VGG 16 net applied to an image segmentation unit provided in a melt pool recognition and penetration depth control system according to an embodiment of the present invention. 6 (a), (b), and (c) show a process of supervising and learning a plurality of first welding images by an image segmentation unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention. It is an image. 7 to 10 are diagrams showing results for each condition in the supervised learning process of the image segmentation unit provided in the molten pool recognition and penetration depth control system according to an embodiment of the present invention.

The image segmentation unit 131 supervises and learns the labeled first arc light image, the first molten pool image, and the first background image, and performs feature points on the labeled first arc light image, the first molten pool image, and the first background image. Extract and set.

Specifically, as shown in FIG. 5, the image segmentation unit 131 applies a VGG 16 net-based Convolutional Neural Network (CNN) algorithm as an image segmentation technique, and since the CNN algorithm corresponds to a known technology, specific The description is omitted.

Illustratively, as shown in (a), (b), and (c) of FIG. 6 , first welding images according to various welding conditions may be supervised and learned in the image segmentation unit 131 .

As for the learning variables used in supervised learning, as shown in (a), (b), and (c) of FIG. 6, there are a total of three variables that are controlled in learning: learning rate, batch size, and epoch.

Specifically, FIG. 6 (a) shows the first welding image when learning rate = 0.001, batch size = 30, and epoch = 300, and FIG. 6 (b) shows learning rate = 0.001, The first welding image when batch size = 50 and epoch = 500 is shown, and FIG. 6 (c) shows the first welding image when learning rate = 0.001, batch size = 50, and epoch = 1000. .

The learning rate is a variable for determining the next point through multiplication with a scalar, and is used in the gradient descent algorithm.

In addition, the learning rate must be appropriately set so that it is neither too large nor too small to efficiently reach the local minimum.

If the learning rate is large, the data deviate randomly and cannot converge to the lowest point.

On the other hand, when the learning rate is small, the learning time takes a very long time and the lowest point is not reached.

Such a learning rate allows supervised learning to proceed faster or slower depending on the change.

Next, the batch size indicates the size of data, and in FIG. 4(c), the batch size is 16 (= the number of data).

Epoch is a dataset that learns multiple batch sizes at once, and if the batch size is 16 in FIG. 4 (c), the epoch is 1.

In addition, in (c) of FIG. 4, if the batch size is 4 (including only horizontal columns among 16), the epoch is 4.

Accordingly, the results of learning while changing only one variable among the above three variables (learning rate, batch size, and epoch) are shown in FIGS. 7 to 10 .

7 and 8 show the results of learning by changing the learning rate and epoch to 0.001 and 300 and changing the batch size to 20 and 30, respectively.

9 and 10 show the result of learning by changing the learning rate and batch size to 0.001 and 50, and changing only the epoch to 500 and 1000, respectively.

As a result of supervised learning while changing the three variables for supervised learning as described above, learning rate = 0.001, batch size = 50, epoch = 1000 as shown in FIG. It was derived that it is the optimal variable for doing and recognizing.

The criteria for deriving the optimal parameters are the clearness of the boundary of the extracted molten pool image and the increase in epoch (according to the increase in epoch, the accuracy of the algorithm recognizing the molten pool image generally increases).

However, if the epoch is unconditionally increased, the time required for learning increases, and learning is performed in a form suitable only for the learning data, so the actual arc welding result and the result of the learning data are different. there is.

Accordingly, the above cases were comprehensively considered and learning rate = 0.001, batch size = 50, and epoch = 1000 were derived as optimal variables.

Next, when a plurality of second welding images are additionally transmitted from the image capture unit 120 after extracting and setting feature points for the first arc light image, the first molten pool image, and the first background image, the image segmentation unit 131 Extracts and separates the second molten pool image from the plurality of second welding images by matching the plurality of second welding images transmitted from the image capturing unit 120 based on the feature points of the first molten pool image.

Characteristic points of the first molten pool image for this purpose may include unique brightness, chroma, color, and shape patterns of the first molten pool image different from the first arc light image and the first background image, but are not limited thereto. .

The image segmentation unit 131 transmits the separated second molten pool image to the shape information measuring unit 132 .

11 is a view showing a process of extracting shape information for a first molten pool image using an image processing algorithm in a shape information measurement unit provided in a melt pool recognition and penetration depth control system according to an embodiment of the present invention. . 12 is a diagram showing shape information obtained from a shape information measurement unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention and a penetration depth obtained from a penetration depth derivation unit.

The shape information measuring unit 132 processes the second molten pool image transmitted from the image dividing unit 131 to measure the shape information of the second molten pool image.

Specifically, as shown in FIG. 11, the shape information measurement unit 132 extracts only the pixels corresponding to the second melt pool image (applying the convex hull image processing algorithm), and then the pixels located at the outermost part of the second melt pool image Extracts the second molten pool boundary followed by the second molten pool boundary, and based on the extracted second molten pool boundary, the width (w), length (l), width (s), and width (w) to length ratio of the second molten pool image ( r) is measured.

That is, the above-described shape information measurement unit 132 applies an image processing algorithm to determine the width (w), length (l), width (s), and width (w) of the second molten pool image as shown in FIG. ) Measure the shape information including the length ratio (r) and the thickness (t) of the base material, and the width (w), length (l), width (s), width (w) of the second molten pool image The ratio (r) and the thickness (t) of the base material are converted into vector data of a 5x1 matrix. Here, the thickness (t) of the base material is information known in advance.

In summary, the image segmentation unit 131 recognizes the molten pool shape extracted through the first deep learning algorithm, and the shape information measuring unit 132 determines the outermost boundary line surrounding the molten pool shape through the convex hull image processing algorithm. Through this, the length, width, area, and width-to-length ratio (rate) of the molten pool shape are measured.

At this time, the measured melt pool shape data is combined with the thickness (t) information of the base material and converted into feature vector data, and the converted feature vector data represents the melt pool shape currently captured in the melt pool monitoring.

The shape information measuring unit 132 transmits the feature vector data obtained by converting the shape information to the penetration depth deriving unit 133 .

13 is a diagram illustrating a process of deriving a penetration depth from a first melt pool image extracted from a system and method for recognizing a melt pool and controlling penetration depth according to an embodiment of the present invention.

Referring to FIG. 13, the penetration depth derivation unit 133 derives the penetration depth p of the second melt pool image by artificial intelligence learning of the feature vector data of the 5x1 matrix transmitted from the shape information measurement unit 132.

In summary, the secondary learning performed in the penetration depth derivation unit 133 is performed in conjunction with the penetration depth at the location where the current melt pool is captured, and inputs the feature vector data for the melt pool to the deep learning model to derive the penetration depth It is made by an algorithm that

Thereafter, the penetration depth derivation unit 133 converts the width (w), length (l), area (s), width (w) to length ratio (r), and the thickness (t) of the base material into vector data of a 5x1 matrix. The penetration depth is derived in real time by using an algorithm that derives the penetration depth inputted.

The penetration depth deriving unit 133 transmits the penetration depth to the control unit.

The control unit 140 controls the current and welding speed of the welding part 110 according to the result of comparing the derived penetration depth with the preset penetration depth.

Specifically, the control unit 140 controls the welded portion 110 so that the welded portion 110 maintains its current state when the penetration depth is the same as the preset penetration depth.

That is, in the above case, since the bead of the arc welding to be implemented is well formed, the welding part 110 is controlled to maintain the current current and welding speed.

On the other hand, the control unit controls the current and welding speed of the welding part 110 when the penetration depth is different from the preset penetration depth.

Specifically, when the penetration depth is greater than the predetermined penetration depth, the control unit 140 controls the welding portion 110 to reduce the current of the welding portion 110 or increase the welding speed of the welding portion 110 .

In the above case, since the base material and the end of the welded portion 110 are close to each other and are formed larger and deeper than the bead of arc welding to be realized, the current of the welded portion 110 is reduced or the welding speed of the welded portion 110 is increased. Arc welding can be performed with a predetermined penetration depth by making the base material melt less and arc welding is performed.

Meanwhile, when the penetration depth is smaller than the predetermined penetration depth, the control unit 140 controls the welding portion 110 to increase the current of the welding portion 110 or decrease the welding speed of the welding portion 110 .

In the above case, since the base material and the end of the welded part 110 are farther apart and formed smaller and shallower than the bead of arc welding to be realized, the current of the welded part 110 is increased or the welding speed of the welded part 110 is reduced. Arc welding can be performed at a preset depth of penetration by allowing the base material to melt more so that arc welding is performed.

2. Melt Pool Recognition and Penetration Depth Control Method

Hereinafter, a method for recognizing a molten pool and controlling a penetration depth according to an embodiment of the present invention will be described with reference to FIGS. 2 to 14 .

14 is a flowchart illustrating a method for recognizing a molten pool and controlling a penetration depth according to an embodiment of the present invention.

Referring to FIG. 14, in the method for recognizing the molten pool and controlling the penetration depth according to an embodiment of the present invention, (a) a plurality of first welding images taken by the imaging unit 120 are first arc light images by a user. , Step of labeling the first molten pool image and the first background image (S100), (b) the image segmentation unit 131 provided in the deep learning unit 130 displays a plurality of first welding images as a first arc light image , Setting feature points for the first arc light image, the first molten pool image, and the first background image by supervising the image dataset labeled with the first molten pool image and the first background image (S200), (c ) Transmitting a plurality of second welding images captured by the image capture unit 120 to the image segmentation unit 131 (S300), (d) the image segmentation unit 131 transmits the first arc light image, the first welding image Separating a second molten pool image from a plurality of second welding images based on feature points of the pool image and the first background image (S400), (e) a shape information measuring unit provided in the deep learning unit 130 ( 132) measuring the shape information of the second molten pool image based on the second molten pool image, (f) the penetration depth derivation unit 132 learns the shape information of the second molten pool image by artificial intelligence to determine the penetration depth Step of deriving (S500) and (g) the control unit 140 determining whether to control the welding part 110 according to the result of comparing the penetration depth transmitted from the penetration depth derivation unit 133 with the preset penetration depth includes

Specifically, the step (a) includes: (a1) acquiring a plurality of first welding images by photographing the tip of the welding portion 110 by the image capture unit 120; (a2) a first arc light image by a user; , The step of labeling the first molten pool image and the first background image, and (a3) the first arc light image, the first molten pool image, and the first background labeled by the image capturing unit 120 with the image dividing unit 131 It includes transmitting the image.

Here, the plurality of first welding images are big data for setting feature points for the plurality of first welding images, which are references for recognizing the second molten pool image. It is an image dataset composed of a full image and a first background image.

In addition, in the step (a2), the labeled first arc light image, first molten pool image, and first background image are expressed as characters as shown in (b) of FIG. 3 or shown in (b) of FIG. 4 As shown, it may be expressed in color.

Next, in the step (b), (b1) the image division unit 131 receiving the labeled first arc light image, the first molten pool image, and the first background image transmitted from the image capturing unit 120 and (b2) the image division unit 131 supervises and learns the labeled first arc light image, the first molten pool image, and the first background image, and the labeled first arc light image, the first molten pool image, and the first background image. and extracting and setting feature points for the image.

Next, in the step (c), (c1) the image capturing unit 120 re-photographs the end of the welding portion 110 to obtain a plurality of second welding images, (c2) the image capturing unit 120 and transmitting a plurality of second welding images to the image segmentation unit 131.

Next, in the step (d), (d1) the image segmentation unit 131 receives a plurality of second welding images transmitted from the image capture unit 120, (d2) the image segmentation unit 120 labels Extracting and separating the second molten pool image from the plurality of second welding images by matching with a plurality of second welding images based on the feature points of the first arc light image, the first molten pool image, and the first background image. and (d3) transmitting, by the image segmentation unit 131, the second molten pool image to the shape information measurement unit 132.

Next, in the step (e), (e1) the shape information measuring unit 132 receives the second molten pool image transmitted from the image dividing unit 131, (e2) the shape information measuring unit 132 Extracting only pixels corresponding to the second molten pool image, (e3) extracting, by the shape information measurement unit 132, the second molten pool boundary line connecting the outermost pixels of the second molten pool image, ( e4) Width (w), length (l), width (s), and width (w) to length ratio (r) of the second molten pool image based on the second molten pool boundary line extracted by the shape information measurement unit 132 ) and measuring the

Here, the shape information measurement unit 132 applies an image processing algorithm to determine the width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image, and base material Measure the shape information including the thickness (t) of, and the width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image, and the thickness of the base material ( t) into vector data of a 5x1 matrix.

Next, in the step (f), (f1) the penetration depth deriving unit 133 determines the width (w), length (l), and width (s) of the second molten pool image transmitted from the shape information measurement unit 132. ), receiving shape information including the length ratio (r) to the width (w) and the thickness (t) of the base material, (f2) the penetration depth deriving unit 132 determines the width (w) of the second melt pool image ), length (l), width (s) and width (w) to length ratio (r), and artificial intelligence learning of the thickness (t) of the base material to derive the penetration depth for the second melt pool image, and (f3 ) The penetration depth deriving unit 133 transmits the penetration depth to the control unit.

Next, in the step (g), (g1) the control unit 140 receives the penetration depth transmitted from the penetration depth deriving unit 133, and (g2) the control unit 140 calculates the penetration depth and the preset penetration depth. and determining whether to control the welding part 110 according to the comparison result.

In addition, the present invention further includes, after the step (g2), when the penetration depth and the preset penetration depth are different, (h) the control unit 140 controls the current and welding speed of the welding portion 110.

Specifically, in the step (h), when the penetration depth is greater than the predetermined penetration depth, the controller 140 controls the welding portion 110 to reduce the current of the welding portion 110 or increase the welding speed of the welding portion 110. Include an increase in

Meanwhile, in step (h), when the penetration depth is smaller than the predetermined penetration depth, the control unit 140 controls the welding portion 110 to increase the current of the welding portion 110 or increase the welding speed of the welding portion 110. A step of reducing is further included.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present invention.

Hereinafter, the present invention will be described with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and, therefore, is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.
Throughout the specification, when a part is said to be "connected (connected, contacted, combined)" with another part, this is not only "directly connected", but also "indirectly connected" with another member in between. "Including cases where In addition, when a part "includes" a certain component, it means that it may further include other components without excluding other components unless otherwise stated.
Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
1. Melt Pool Recognition and Penetration Depth Control System (100)
Hereinafter, a melt pool recognition and penetration depth control system according to an embodiment of the present invention will be described with reference to FIGS. 2 to 13.
2 is a block diagram showing a melt pool recognition and penetration depth control system according to an embodiment of the present invention.
The molten pool recognition and penetration depth control system 100 according to an embodiment of the present invention includes a welding unit 110, an image capturing unit 120, a deep learning unit 130 and a control unit 140.
The welding part 110 supplies heat generated by discharging the arc light to a junction where at least two base materials are in contact.
The welding part 110 for this may be a welding rod that transmits arc light to a junction where at least two base materials come into contact. In addition, a welding electrode holder for a user to grip is connected to the welding part 110 .
In addition, in order for the welding part 110 to operate, an electric line electrically connected to the welding rod holder, a grounding clamp electrically connected to the base material, a grounding line electrically connected to the grounding clamp, and an electric line and the grounding line are electrically connected and generate an arc light. An arc welder may be provided.
The ends of the welded part 110 arc-weld at least two base materials as they move along the joint in a state of being separated by a predetermined distance toward the joint where at least two base materials come into contact.
3 is a first welding including a first arc light image, a first molten pool image, and a first background image captured by an imaging unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention This is an image showing that the image has been manually labeled by the user. (a), (b), and (c) of FIG. 4 are images showing a dataset composed of a plurality of first arc light images, a first molten pool image, and a first background image labeled in FIG. .
The image capturing unit 120 photographs the end of the welding portion 110, and as shown in FIGS. 3 (a) and 4 (b), the first arc light image, the first molten pool image formed at the junction, and A plurality of first welding images including a background image located around the first molten pool image are acquired in real time.
Here, the plurality of first welding images are big data for setting feature points that are criteria for recognizing the second molten pool image, and are composed of a plurality of first arc light images, first molten pool images, and first background images. It is an image dataset that is constructed.
The obtained plurality of first welding images are manually labeled by the user as shown in (b) of FIG. 3 .
Specifically, as shown in FIGS. 3(b) and 4(a), the labeling marks the first arc light image as arc light, displays the first melt pool as a melt pool, and displays the first arc light image as a melt pool. By marking it as a background, it indicates what the image is about.
Here, the first arc light image, the first molten pool image, and the first background image are images for the deep learning unit 130 to derive and select feature points for the arc light, the molten pool, and the background through supervised learning. These are the plurality of first welding images first photographed by the photographing unit 120 .
The first welding image includes a first arc light image, a first molten pool image, and a first background image, as shown in (c) of FIG. 4, and is composed of a plurality of data sets.
On the other hand, the second arc light image, the second molten pool image, and the second background image to be described later are images to be recognized as molten pools in the present invention, and are captured by the image capture unit 110 and used by the deep learning unit 130. It is an image that is matched with feature points for the arc light, molten pool, and background.
The above image capture unit 120 transmits the labeled first arc light image, the first molten pool image, and the first background image to the image segmentation unit 131 provided in the deep learning unit 130, and then additionally second An arc light image, a second molten pool image, and a second background image are acquired and transmitted to the image segmentation unit 130 .
The deep learning unit 130 performs supervised learning on the first arc light image, the first molten pool image, and the first background image manually labeled by the user to obtain the first arc light image, the first molten pool image, and Extracting and setting feature points for the first background image, extracting second molten pool images from a plurality of second welding images additionally obtained from the imaging unit 120 based on the feature points for the first molten pool image, and 2 Image processing of the molten pool image measures the shape information of the second molten pool image, and artificial intelligence learns the measured shape information to derive the penetration depth.
The deep learning unit 130 for this includes an image segmentation unit 131, a shape information measurement unit 132, and a penetration depth derivation unit 133.
5 is a conceptual diagram illustrating a convolutional neural network (CNN) algorithm based on a VGG 16 net applied to an image segmentation unit provided in a melt pool recognition and penetration depth control system according to an embodiment of the present invention. 6 (a), (b), and (c) show a process of supervising and learning a plurality of first welding images by an image segmentation unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention. It is an image. 7 to 10 are diagrams showing results for each condition in the supervised learning process of the image segmentation unit provided in the molten pool recognition and penetration depth control system according to an embodiment of the present invention.
The image segmentation unit 131 supervises and learns the labeled first arc light image, the first molten pool image, and the first background image, and performs feature points on the labeled first arc light image, the first molten pool image, and the first background image. Extract and set.
Specifically, as shown in FIG. 5, the image segmentation unit 131 applies a VGG 16 net-based Convolutional Neural Network (CNN) algorithm as an image segmentation technique, and since the CNN algorithm corresponds to a known technology, specific The description is omitted.
Illustratively, as shown in (a), (b), and (c) of FIG. 6 , first welding images according to various welding conditions may be supervised and learned in the image segmentation unit 131 .
As for the learning variables used in supervised learning, as shown in (a), (b), and (c) of FIG. 6, there are a total of three variables that are controlled in learning: learning rate, batch size, and epoch.
Specifically, FIG. 6 (a) shows the first welding image when learning rate = 0.001, batch size = 30, and epoch = 300, and FIG. 6 (b) shows learning rate = 0.001, The first welding image when batch size = 50 and epoch = 500 is shown, and FIG. 6 (c) shows the first welding image when learning rate = 0.001, batch size = 50, and epoch = 1000. .
The learning rate is a variable for determining the next point through multiplication with a scalar, and is used in the gradient descent algorithm.
In addition, the learning rate must be appropriately set so that it is neither too large nor too small to efficiently reach the local minimum.
If the learning rate is large, the data deviate randomly and cannot converge to the lowest point.
On the other hand, when the learning rate is small, the learning time takes a very long time and the lowest point is not reached.
Such a learning rate allows supervised learning to proceed faster or slower depending on the change.
Next, the batch size indicates the size of data, and in FIG. 4(c), the batch size is 16 (= the number of data).
Epoch is a dataset that learns multiple batch sizes at once, and if the batch size is 16 in FIG. 4 (c), the epoch is 1.
In addition, in (c) of FIG. 4, if the batch size is 4 (including only horizontal columns among 16), the epoch is 4.
Accordingly, the results of learning while changing only one variable among the above three variables (learning rate, batch size, and epoch) are shown in FIGS. 7 to 10 .
7 and 8 show the results of learning by changing the learning rate and epoch to 0.001 and 300 and changing the batch size to 20 and 30, respectively.
9 and 10 show the result of learning by changing the learning rate and batch size to 0.001 and 50, and changing only the epoch to 500 and 1000, respectively.
As a result of supervised learning while changing the three variables for supervised learning as described above, learning rate = 0.001, batch size = 50, epoch = 1000 as shown in FIG. It was derived that it is the optimal variable for doing and recognizing.
The criteria for deriving the optimal parameters are the clearness of the boundary of the extracted molten pool image and the increase in epoch (according to the increase in epoch, the accuracy of the algorithm recognizing the molten pool image generally increases).
However, if the epoch is unconditionally increased, the time required for learning increases, and learning is performed in a form suitable only for the learning data, so the actual arc welding result and the result of the learning data are different. there is.
Accordingly, the above cases were comprehensively considered and learning rate = 0.001, batch size = 50, and epoch = 1000 were derived as optimal variables.
Next, when a plurality of second welding images are additionally transmitted from the image capture unit 120 after extracting and setting feature points for the first arc light image, the first molten pool image, and the first background image, the image segmentation unit 131 Extracts and separates the second molten pool image from the plurality of second welding images by matching the plurality of second welding images transmitted from the image capturing unit 120 based on the feature points of the first molten pool image.
Characteristic points of the first molten pool image for this purpose may include unique brightness, chroma, color, and shape patterns of the first molten pool image different from the first arc light image and the first background image, but are not limited thereto. .
The image segmentation unit 131 transmits the separated second molten pool image to the shape information measuring unit 132 .
11 is a view showing a process of extracting shape information for a first molten pool image using an image processing algorithm in a shape information measurement unit provided in a melt pool recognition and penetration depth control system according to an embodiment of the present invention. . 12 is a diagram showing shape information obtained from a shape information measurement unit provided in a molten pool recognition and penetration depth control system according to an embodiment of the present invention and a penetration depth obtained from a penetration depth derivation unit.
The shape information measuring unit 132 processes the second molten pool image transmitted from the image dividing unit 131 to measure the shape information of the second molten pool image.
Specifically, as shown in FIG. 11, the shape information measurement unit 132 extracts only the pixels corresponding to the second melt pool image (applying the convex hull image processing algorithm), and then the pixels located at the outermost part of the second melt pool image Extracts the second molten pool boundary followed by the second molten pool boundary, and based on the extracted second molten pool boundary, the width (w), length (l), width (s), and width (w) to length ratio of the second molten pool image ( r) is measured.
That is, the above-described shape information measurement unit 132 applies an image processing algorithm to determine the width (w), length (l), width (s), and width (w) of the second molten pool image as shown in FIG. ) Measure the shape information including the length ratio (r) and the thickness (t) of the base material, and the width (w), length (l), width (s), width (w) of the second molten pool image The ratio (r) and the thickness (t) of the base material are converted into vector data of a 5x1 matrix. Here, the thickness (t) of the base material is information known in advance.
In summary, the image segmentation unit 131 recognizes the molten pool shape extracted through the first deep learning algorithm, and the shape information measuring unit 132 determines the outermost boundary line surrounding the molten pool shape through the convex hull image processing algorithm. Through this, the length, width, area, and width-to-length ratio (rate) of the molten pool shape are measured.
At this time, the measured melt pool shape data is combined with the thickness (t) information of the base material and converted into feature vector data, and the converted feature vector data represents the melt pool shape currently captured in the melt pool monitoring.
The shape information measuring unit 132 transmits the feature vector data obtained by converting the shape information to the penetration depth deriving unit 133 .
13 is a diagram illustrating a process of deriving a penetration depth from a first melt pool image extracted from a system and method for recognizing a melt pool and controlling penetration depth according to an embodiment of the present invention.
Referring to FIG. 13, the penetration depth derivation unit 133 derives the penetration depth p of the second melt pool image by artificial intelligence learning of the feature vector data of the 5x1 matrix transmitted from the shape information measurement unit 132.
In summary, the secondary learning performed in the penetration depth derivation unit 133 is performed in conjunction with the penetration depth at the location where the current melt pool is captured, and inputs the feature vector data for the melt pool to the deep learning model to derive the penetration depth It is made by an algorithm that
Thereafter, the penetration depth derivation unit 133 converts the width (w), length (l), area (s), width (w) to length ratio (r), and the thickness (t) of the base material into vector data of a 5x1 matrix. The penetration depth is derived in real time by using an algorithm that derives the penetration depth inputted.
The penetration depth deriving unit 133 transmits the penetration depth to the control unit.
The control unit 140 controls the current and welding speed of the welding part 110 according to the result of comparing the derived penetration depth with the preset penetration depth.
Specifically, the control unit 140 controls the welded portion 110 so that the welded portion 110 maintains its current state when the penetration depth is the same as the preset penetration depth.
That is, in the above case, since the bead of the arc welding to be implemented is well formed, the welding part 110 is controlled to maintain the current current and welding speed.
On the other hand, the control unit controls the current and welding speed of the welding part 110 when the penetration depth is different from the preset penetration depth.
Specifically, when the penetration depth is greater than the predetermined penetration depth, the control unit 140 controls the welding portion 110 to reduce the current of the welding portion 110 or increase the welding speed of the welding portion 110 .
In the above case, since the base material and the end of the welded portion 110 are close to each other and are formed larger and deeper than the bead of arc welding to be realized, the current of the welded portion 110 is reduced or the welding speed of the welded portion 110 is increased. Arc welding can be performed with a predetermined penetration depth by making the base material melt less and arc welding is performed.
Meanwhile, when the penetration depth is smaller than the predetermined penetration depth, the control unit 140 controls the welding portion 110 to increase the current of the welding portion 110 or decrease the welding speed of the welding portion 110 .
In the above case, since the base material and the end of the welded part 110 are farther apart and formed smaller and shallower than the bead of arc welding to be realized, the current of the welded part 110 is increased or the welding speed of the welded part 110 is reduced. Arc welding can be performed at a preset depth of penetration by allowing the base material to melt more so that arc welding is performed.
2. Melt Pool Recognition and Penetration Depth Control Method
Hereinafter, a method for recognizing a molten pool and controlling a penetration depth according to an embodiment of the present invention will be described with reference to FIGS. 2 to 14 .
14 is a flowchart illustrating a method for recognizing a molten pool and controlling a penetration depth according to an embodiment of the present invention.
Referring to FIG. 14, in the method for recognizing the molten pool and controlling the penetration depth according to an embodiment of the present invention, (a) a plurality of first welding images taken by the imaging unit 120 are first arc light images by a user. , Step of labeling the first molten pool image and the first background image (S100), (b) the image segmentation unit 131 provided in the deep learning unit 130 displays a plurality of first welding images as a first arc light image , Setting feature points for the first arc light image, the first molten pool image, and the first background image by supervising the image dataset labeled with the first molten pool image and the first background image (S200), (c ) Transmitting a plurality of second welding images captured by the image capture unit 120 to the image segmentation unit 131 (S300), (d) the image segmentation unit 131 transmits the first arc light image, the first welding image Separating a second molten pool image from a plurality of second welding images based on feature points of the pool image and the first background image (S400), (e) a shape information measuring unit provided in the deep learning unit 130 ( 132) measuring the shape information of the second molten pool image based on the second molten pool image, (f) the penetration depth derivation unit 132 learns the shape information of the second molten pool image by artificial intelligence to determine the penetration depth Step of deriving (S500) and (g) the control unit 140 determining whether to control the welding part 110 according to the result of comparing the penetration depth transmitted from the penetration depth derivation unit 133 with the preset penetration depth includes
Specifically, the step (a) includes: (a1) acquiring a plurality of first welding images by photographing the tip of the welding portion 110 by the image capture unit 120; (a2) a first arc light image by a user; , The step of labeling the first molten pool image and the first background image, and (a3) the first arc light image, the first molten pool image, and the first background labeled by the image capturing unit 120 with the image dividing unit 131 It includes transmitting the image.
Here, the plurality of first welding images are big data for setting feature points for the plurality of first welding images, which are references for recognizing the second molten pool image. It is an image dataset composed of a full image and a first background image.
In addition, in the step (a2), the labeled first arc light image, first molten pool image, and first background image are expressed as characters as shown in (b) of FIG. 3 or shown in (b) of FIG. 4 As shown, it may be expressed in color.
Next, in the step (b), (b1) the image division unit 131 receiving the labeled first arc light image, the first molten pool image, and the first background image transmitted from the image capturing unit 120 and (b2) the image division unit 131 supervises and learns the labeled first arc light image, the first molten pool image, and the first background image, and the labeled first arc light image, the first molten pool image, and the first background image. and extracting and setting feature points for the image.
Next, in the step (c), (c1) the image capturing unit 120 re-photographs the end of the welding portion 110 to obtain a plurality of second welding images, (c2) the image capturing unit 120 and transmitting a plurality of second welding images to the image segmentation unit 131.
Next, in the step (d), (d1) the image segmentation unit 131 receives a plurality of second welding images transmitted from the image capture unit 120, (d2) the image segmentation unit 120 labels Extracting and separating the second molten pool image from the plurality of second welding images by matching with a plurality of second welding images based on the feature points of the first arc light image, the first molten pool image, and the first background image. and (d3) transmitting, by the image segmentation unit 131, the second molten pool image to the shape information measuring unit 132.
Next, in the step (e), (e1) the shape information measuring unit 132 receives the second molten pool image transmitted from the image dividing unit 131, (e2) the shape information measuring unit 132 Extracting only pixels corresponding to the second molten pool image, (e3) extracting, by the shape information measurement unit 132, the second molten pool boundary line connecting the outermost pixels of the second molten pool image, ( e4) Width (w), length (l), width (s), and width (w) to length ratio (r) of the second molten pool image based on the second molten pool boundary line extracted by the shape information measurement unit 132 ) and measuring the
Here, the shape information measurement unit 132 applies an image processing algorithm to determine the width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image, and base material Measure the shape information including the thickness (t) of, and the width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image, and the thickness of the base material ( t) into vector data of a 5x1 matrix.
Next, in the step (f), (f1) the penetration depth deriving unit 133 determines the width (w), length (l), and width (s) of the second molten pool image transmitted from the shape information measurement unit 132. ), receiving shape information including the length ratio (r) to the width (w) and the thickness (t) of the base material, (f2) the penetration depth deriving unit 132 determines the width (w) of the second melt pool image ), length (l), width (s) and width (w) to length ratio (r), and artificial intelligence learning of the thickness (t) of the base material to derive the penetration depth for the second melt pool image, and (f3 ) The penetration depth deriving unit 133 transmits the penetration depth to the control unit.
Next, in the step (g), (g1) the control unit 140 receives the penetration depth transmitted from the penetration depth deriving unit 133, and (g2) the control unit 140 calculates the penetration depth and the preset penetration depth. and determining whether to control the welding part 110 according to the comparison result.
In addition, the present invention further includes, after the step (g2), when the penetration depth and the preset penetration depth are different, (h) the control unit 140 controls the current and welding speed of the welding portion 110.
Specifically, in step (h), when the penetration depth is greater than the preset penetration depth, the control unit 140 controls the welding portion 110 to reduce the current of the welding portion 110 or increase the welding speed of the welding portion 110. Include an increase in
Meanwhile, in step (h), when the penetration depth is smaller than the predetermined penetration depth, the control unit 140 controls the welding portion 110 to increase the current of the welding portion 110 or increase the welding speed of the welding portion 110. A step of reducing is further included.
The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.
The scope of the present invention is indicated by the following claims, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof should be construed as being included in the scope of the present invention.

Claims (20)

a welding part supplying heat generated by discharging the arc light to a junction where at least two base materials are in contact;
A plurality of first welding images including a first arc light image, a first molten pool image formed at the junction, and a background image located around the first molten pool image are obtained in real time by photographing the tip of the welding portion. a video recording unit;
The first arc light image, the first molten pool image, and the first background image manually labeled by the user are supervised and learned to obtain the first arc light image, the first molten pool image, and the first background image. Set feature points for the first arc light image, the first molten pool image, and the second molten pool from a plurality of second welding images additionally obtained from the imaging unit based on the feature points for the first background image. a deep learning unit that extracts an image, processes the second molten pool image, measures shape information of the second molten pool image, and derives a penetration depth by artificial intelligence learning of the measured shape information; and
Melting pool recognition and penetration depth control system comprising a; control unit for controlling the current and welding speed of the welding part according to the result of comparing the derived penetration depth and the preset penetration depth.
According to claim 1,
The plurality of first welding images are big data for setting the feature point, which is a criterion for recognizing the second molten pool image, and consists of a plurality of first arc light images, the first molten pool image, and the first molten pool image. 1 is an image dataset consisting of a background image,
The deep learning unit performs supervised learning on the labeled first arc light image, the first molten pool image, and the first background image, and performs feature points on the labeled first arc light image, the first molten pool image, and the first background image. Including; an image segmentation unit that sets a setting;
The image segmentation unit matches the plurality of second welding images transmitted from the image capturing unit based on feature points of the first molten pool image to separate the second molten pool image from the plurality of second welding images Melting pool recognition and penetration depth control system, characterized in that.
According to claim 2,
The image segmentation unit applies a VGG 16 net-based Convolutional Neural Network (CNN) algorithm as an image segmentation technique,
Characteristic points of the first molten pool image include unique brightness, chroma, color, and shape patterns of the first molten pool image different from the first arc light image and the first background image. and penetration depth control system.
According to claim 2,
The deep learning unit further includes a shape information measuring unit configured to measure shape information of the second molten pool image by image processing the second molten pool image transmitted from the image segmentation unit,
The shape information measurement unit extracts only pixels corresponding to the second molten pool image, extracts a second molten pool boundary line where pixels positioned at the outermost part of the second molten pool image are connected, and determines the extracted second molten pool boundary line. Melt pool recognition and penetration depth control system, characterized in that for measuring the width (w), length (l), width (s) and the width (w) to length ratio (r) of the second melt pool image based on .
According to claim 4,
The shape information measurement unit applies an image processing algorithm to the width (w), length (l), width (s), width (w) to length ratio (r) and thickness (t) of the base material for the second molten pool image. ), and the width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image and the thickness (t) of the base material Melting pool recognition and penetration depth control system, characterized in that for converting to feature vector data of a 5x1 matrix.
According to claim 4,
The deep learning unit further includes a penetration depth derivation unit for deriving a penetration depth (p) of the second molten pool image by artificial intelligence learning of the feature vector data of the 5x1 matrix transmitted from the shape information measurement unit. molten pool recognition and penetration depth control system.
According to claim 6,
The penetration depth derivation unit transmits the penetration depth to the control unit,
When the penetration depth and the preset penetration depth are equal to the control unit, the molten pool recognition and penetration depth control system, characterized in that for controlling the welding portion to maintain the current state.
According to claim 6,
The penetration depth derivation unit transmits the penetration depth to the control unit,
The control unit is a molten pool recognition and penetration depth control system, characterized in that for controlling the current and welding speed of the welding part when the penetration depth and the preset penetration depth are different.
According to claim 8,
When the penetration depth is greater than the predetermined penetration depth, the control unit controls the weld pool to reduce the current of the weld portion or increase the welding speed of the weld pool recognition and penetration depth control system.
According to claim 8,
When the penetration depth is smaller than the preset penetration depth, the control unit controls the welding portion to increase the current of the welding portion or reduce the welding speed of the welding portion. Melting pool recognition and penetration depth control system.
(a) labeling a plurality of first welding images taken by an imaging unit as a first arc light image, a first molten pool image, and a first background image by a user;
(b) an image segmentation unit provided in the deep learning unit supervises and learns an image dataset in which the plurality of first welding images are labeled as the first arc light image, the first molten pool image, and the first background image; Setting feature points for the first arc light image, the first molten pool image, and the first background image;
(c) transmitting a plurality of second welding images captured by the image capture unit to the image segmentation unit;
(d) separating a second molten pool image from the plurality of second welding images based on feature points of the first arc light image, the first welding pool image, and the first background image by the image division unit;
(e) measuring shape information of the second molten pool image based on the second molten pool image by a shape information measuring unit provided in the deep learning unit;
(f) deriving a penetration depth by artificial intelligence learning of the shape information of the second molten pool image by the penetration depth derivation unit; and
(g) determining whether or not to control the welding part according to a result of the control unit comparing the penetration depth transmitted from the penetration depth derivation unit and the preset penetration depth; molten pool recognition and penetration depth, characterized in that it comprises control method.
According to claim 11,
In step (a),
(a1) acquiring the plurality of first welding images by photographing an end of the welding portion by the image capture unit;
(a2) labeling the first arc light image, the first molten pool image, and the first background image by the user; and
(a3) transmitting the labeled first arc light image, the first molten pool image, and the first background image to the image dividing unit by the image capture unit; melting pool recognition and penetration depth control comprising Way.
According to claim 12,
The plurality of first welding images are big data for setting the feature point, which is a criterion for recognizing the second molten pool image, and consists of a plurality of first arc light images, the first molten pool image, and the first molten pool image. 1 is an image dataset consisting of a background image,
In step (b),
(b1) receiving the labeled first arc light image, the first molten pool image, and the first background image transmitted from the image capture unit by the image segmentation unit; and
(b2) The image division unit supervises and learns the labeled first arc light image, the first molten pool image, and the first background image to obtain the labeled first arc light image, the first molten pool image, and the first background image. Setting a feature point for; molten pool recognition and penetration depth control method comprising the.
According to claim 11,
In step (c),
(c1) acquiring the plurality of second welding images by photographing the end of the welding part again by the image capture unit;
(c2) transmitting the plurality of second welding images to the image dividing unit by the image capture unit; molten pool recognition and penetration depth control method comprising the.
According to claim 14,
In step (d),
(d1) receiving, by the image division unit, the plurality of second welding images transmitted from the image capturing unit;
(d2) The image segmentation unit matches the plurality of second welding images based on the feature points of the labeled first arc light image, first molten pool image, and first background image to obtain information from the plurality of second welding images. Separating the second molten pool image; and
(d3) transmitting the second molten pool image by the image segmentation unit to the shape information measurement unit; molten pool recognition and penetration depth control method comprising the.
According to claim 11,
In step (e),
(e1) receiving the second molten pool image transmitted from the image dividing unit by the shape information measuring unit;
(e2) extracting only pixels corresponding to the second molten pool image by the shape information measuring unit;
(e3) extracting a second molten pool boundary where the outermost pixels of the second molten pool image are connected by the shape information measuring unit;
(e4) The width (w), length (l), width (s) of the second molten pool image based on the extracted second molten pool boundary line by the shape information measuring unit, and the width (w) to length ratio ( Including; measuring r);
The shape information measurement unit applies an image processing algorithm to the width (w), length (l), width (s), width (w) to length ratio (r) and thickness (t) of the base material for the second molten pool image. ), and the width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image and the thickness (t) of the base material Melting pool recognition and penetration depth control method, characterized in that for converting to feature vector data of a 5x1 matrix.
According to claim 16,
In step (f),
(f1) Width (w), length (l), width (s), width (w) to length ratio (r) of the second molten pool image transmitted from the shape information measurement unit by the penetration depth derivation unit, and Receiving shape information including the thickness (t) of the base material;
(f2) The penetration depth derivation unit artificially calculates the width (w), length (l), width (s), width (w) to length ratio (r) and thickness (t) of the base material for the second molten pool image. Deriving the penetration depth for the second molten pool image by intelligent learning; and
(f3) transmitting the penetration depth to the control unit by the penetration depth deriving unit; melt pool recognition and penetration depth control system comprising a.
According to claim 11,
In step (g),
(g1) receiving, by the control unit, the penetration depth transmitted from the penetration depth deriving unit; and
(g2) determining, by the control unit, whether or not to control the welding part according to a result of comparing the penetration depth and the preset penetration depth; molten pool recognition and penetration depth control method comprising the.
According to claim 18,
After the step (g2),
(H) the control unit controlling the current and welding speed of the welding part; further comprising,
In step (h),
When the penetration depth is greater than the predetermined penetration depth, the control unit controls the welding portion to reduce the current of the welding portion or increase the welding speed of the welding portion. Depth control method.
According to claim 19,
In step (h),
When the penetration depth is smaller than the predetermined penetration depth, the control unit controls the welding portion to increase the current of the welding portion or decrease the welding speed of the welding portion. Melting pool recognition and Penetration depth control method.

Images