KR102230886B1 - Projection welding defect detection system based on image recognition - Google Patents

Abstract

The present invention is to provide a defect detection system capable of easily detecting a defect without the need for individual visual confirmation by a user. According to an embodiment of the present invention, a projection welding defect detection system may comprise: a data acquisition unit which acquires welding data for each point output through projection welding by a welding machine; an image conversion unit for acquiring image data through image conversion for the welding data; and a judging unit which determines whether or not a defect occurs due to projection welding from the image data using a detection model learned in advance according to a deep learning algorithm.

Description

Projection welding defect detection system based on image recognition {PROJECTION WELDING DEFECT DETECTION SYSTEM BASED ON IMAGE RECOGNITION}

The present invention relates to a projection welding defect detection system based on image recognition. More specifically, based on image recognition to determine whether a defect due to welding has occurred from the image data of welding data acquired during welding using a detection model created by learning based on a convolutional neural network. It relates to a projection welding defect detection system of.

Deep learning, deep learning, is a machine learning that attempts a high level of abstraction (summarizing key content or functions in a large amount of data or complex data) through a combination of several nonlinear transducers. machine learning) is defined as a set of algorithms, and it can be said to be a branch of machine learning that teaches computers how people think in a large framework. When there is any data, it is represented in a form that can be understood by a computer (e.g., in the case of images, pixel information is expressed as a column vector, etc.), and a lot of research (how to do better expression techniques) to apply it to learning. And how to create a model to learn them), and as a result of these efforts, various deep learning techniques such as DNN (deep neural network), CNN (convolutional neural network), and DBN (deep belief network) They are applied in fields such as computer vision, speech recognition, natural language processing, and voice/signal processing to show cutting-edge results.

On the other hand, projection welding is one of the representative welding methods of resistance welding, and is a method of welding by applying pressure to the resistance heat generated in the process of transferring current to the contact part of the base material to be joined. 1 is an exemplary diagram showing a resistance welding method. In the resistance welding, as shown in (a) of FIG. 1, a large current is passed through the welding base material and the welding base material is heated by heat generated by the contact resistance of the joint. It is a method of welding by making it melted and applying mechanical pressure. That is, projection welding corresponds to resistance welding such as welding in terms of joining using energy generated when a welding current is applied as a heat source while the materials to be welded are in close contact with each other. In general resistance welding, a predetermined electrode is used to pass current to the welded part on a plane, but projection welding is a point in that projection welding processes a protrusion on one or both sides of the material to be welded before welding and concentrates the current through the protrusion. It is distinguished from welding or seam welding. Whereas spot welding uses a round rod-shaped electrode, in projection welding, a welding current and a pressing force are applied with a block-shaped electrode above and below the material to be welded with a protrusion.

In performing resistance welding including projection welding, it is a very difficult task to determine welding defects, and the accuracy of defect determination is not high. As shown in Fig. 1(b), there are various types of welding defects. In particular, welding defects are caused by various causes such as working conditions, shape of welding material, type of welding material, foreign matter on the surface of welding material, performance of welding machine, welding condition, etc. Since it can occur, it is not time- and cost-effective for an operator to identify these causes individually. In the past, the inspector and the inspector track the welding line one by one in order to inspect the weldability, and the inspector reads the state of the weld with the naked eye, and then determines the presence or absence of a defect in the joint. However, in the above inspection method, the subjective judgment factor of the inspector is strongly reflected, resulting in a problem that the inspection reliability decreases due to a decrease in accuracy.

In addition, data such as current, voltage, pneumatic pressure, and temperature measured from the welding machine are monitored and patterns are analyzed. FIG. 2 is an exemplary view showing a process of data monitoring to detect defects that may occur during resistance welding. As shown in FIG. 2, data such as measured current is analyzed to determine whether a defect in welding exists through waveform analysis. It is the situation that is being judged. However, the reference value, which is the basis for determining normal and defective, is directly determined by a person through data collection, and the user must directly find out what is the category of normal or defective data, and it is difficult to use data in a non-linear relationship. .

In order to prevent these problems, the development of a welding defect detection system using the above-described deep learning technique is continuously required.

Republic of Korea Patent Publication No. 10-2006-0076562 (published date: 2006.07.04)

The present invention is to solve the above-described problem, by imaging the welding data to learn a convolutional neural network (CNN), and automatically determine whether a welding defect according to the learning result by the user individually visually. The purpose of this is to provide a defect detection system capable of easily detecting defects without the need for confirmation.

As an embodiment of the present invention, a projection welding defect detection system based on image recognition capable of interlocking with a welding machine may be provided.

The projection welding defect detection system according to an embodiment of the present invention includes a data acquisition unit for obtaining welding data for each point of time output through projection welding by a welding machine, and an image conversion for obtaining image data through image conversion of the welding data. A determination unit for determining whether or not a defect occurs due to projection welding from image data may be included using a detection model learned in advance according to sub- and deep learning algorithms.

In the projection welding defect detection system according to an embodiment of the present invention, the welding data may include voltage data, current data, and pressure data.

In the projection welding defect detection system according to an embodiment of the present invention, the image conversion unit may acquire image data using a frequency conversion technique.

In the projection welding defect detection system according to an embodiment of the present invention, a data storage unit for storing welding data and image data for each viewpoint may be further included.

In the projection welding defect detection system according to an embodiment of the present invention, an alarm generator for generating an alarm signal according to determination of whether a defect has occurred may be further included.

In the projection welding defect detection system according to an exemplary embodiment of the present invention, a control unit for controlling a protrusion current applied from a welding machine may be further included according to determination of whether or not a defect occurs.

As an embodiment of the present invention, a method for detecting a projection welding defect using a projection welding defect detection system based on image recognition capable of interlocking with a welding machine may be provided.

Projection welding defect detection method according to an embodiment of the present invention includes the steps of obtaining welding data for each point of time output through projection welding by a welding machine, obtaining image data through image conversion of welding data, and a deep learning algorithm. According to this, a step of determining whether or not a defect occurs due to projection welding from the image data using the pre-learned detection model may be included.

As an embodiment of the present invention, a computer-readable recording medium in which a program for implementing the above-described method is recorded may be provided.

According to the image recognition-based projection welding defect detection system of the present invention, it is possible to reduce the hassle of an inspector to check with the naked eye, and to accurately detect defects according to a learning result of a convolutional neural network (CNN).

In addition, since welding data is converted into image data and input to the convolutional neural network, even if welding failure occurs due to any cause such as electrode oxidation, welding machine failure, material problem, etc., it is possible to accurately determine.

1 is an exemplary view showing a resistance welding method.
2 is an exemplary view showing a process in which data monitoring is performed to detect a defect that may occur during resistance welding.
3 is an exemplary view showing a projection welding defect detection system based on image recognition according to an embodiment of the present invention.
4 is a conceptual diagram of a convolutional neural network (CNN).
5 is an exemplary view showing a discrimination model in a projection welding defect detection system based on image recognition according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

The terms used in the present specification will be briefly described, and the present invention will be described in detail.

Terms used in the present invention have selected general terms that are currently widely used as possible while taking functions of the present invention into consideration, but this may vary according to the intention or precedent of a technician working in the field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. . In addition, when a part is said to be "connected" with another part throughout the specification, this includes not only the case of being "directly connected", but also the case of being connected "with another element in the middle."

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

As an embodiment of the present invention, a projection welding defect detection system based on image recognition capable of interlocking with the welding machine 20 may be provided. In the present specification, the welding machine 20 refers to a welding device operating in a projection welding method, and the database stores data received from the projection welding defect detection system, or a data management server for transferring the stored data to the detection system. Refers to the device of. In addition, projection welding is one of the representative welding methods of resistance welding, and refers to a method of welding by applying pressure to resistance heat generated in the process of transferring current to the contact part of the base material to be joined.

3 is an exemplary view showing a projection welding defect detection system based on image recognition according to an embodiment of the present invention.

3, the projection welding defect detection system according to an embodiment of the present invention is a data acquisition unit 100 for obtaining welding data for each point of time output through projection welding by a welding machine 20, for welding data. An image conversion unit 200 for acquiring image data through image conversion and a determination unit 300 that determines whether a defect occurs due to projection welding from the image data using the detection model 400 learned in advance according to a deep learning algorithm. ) May be included.

The data acquisition unit 100 may acquire welding data for each time point according to the progress of the projection welding. The welding data may include voltage data, current data, and pressure data, and the welding data may be time-specific or time-series data acquired at predetermined time intervals. To this end, the welding machine 20 may be provided with a sensor module for measuring current, voltage, and pressure according to projection welding. The voltage data may indicate a voltage value applied to the electrode of the welding machine 20 during welding, or may indicate a voltage value applied between the electrode and the welding base material. The current data may represent a current value output from the welding machine 20 to the welding base material. In addition, the pressure data may represent a pressing force value applied to the welding base material according to projection welding. That is, the sensor module may include a current measurement module, a voltage measurement module, and a pressure measurement module for measuring the current data, voltage data, and pressure data, respectively.

The welding data for each viewpoint acquired by the data acquisition unit 100 may be filtered by a filter unit (not shown) before being converted into image data by the image conversion unit 200. The filter unit may include at least one of a high-pass filter, a low-pass filter, a band-pass filter, a matching filter, or the filters, and filtering may be performed by a combined filter.

In addition, the data acquisition unit 100 may additionally perform sampling. That is, the welding data may be sampled in units of a predetermined time interval, and the time interval may be set to 0.1 [sec] as a time for the image conversion unit 200 to convert to image data suitable. have. However, since this is only an example, the time interval may be set to various values according to actual implementation.

The image conversion unit 200 may generate image data for welding data acquired by the data acquisition unit 100 according to an image conversion process. Welding data including current data, voltage data, and pressure data is a kind of raw data and, as described above, corresponds to point-in-time data or time series data. That is, the welding data may be converted into image data of a predetermined size by applying an image conversion technique. In addition, image data may be generated from the above-described sampled signal.

The image data 250 may be an image implemented in a two-dimensional array of welding data acquired by the data acquisition unit 100, and welding data values (voltage, current, pressure) over time are converted into pixel values. It can be an image.

Also, the image data 250 may be an image obtained using a frequency conversion technique. Various methods may be used for the frequency transformation of the welding data, but it is preferable that a Fast Fourier Transform (FFT) method is used for the frequency transformation. That is, as the welding data is frequency-converted according to the FFT method, the intensity of each frequency component may be expressed as a frequency spectrum, and the image data 250 may be generated by converting the intensity into an image form. Hereinafter, a description will be made based on the image data 250 generated according to the result of the FFT transformation.

The image conversion unit 200 may adjust the size of the acquired image data 250. For example, image data 250 having a size of 333 X 215 may be converted to a size of 90 X 50. The size of the converted image data 250 may be set in various ways. That is, the image conversion unit 200 may be adjusted to a size of 110 X 70 or a size of 55 X 35 as well as a size of 90 X 50 as described above. The size of the image data 250 to be adjusted may be set differently according to a learning target, a structure of the detection model 400, and the like.

In addition, the image conversion unit 200 may perform a shifting process on the image data 250. That is, the image conversion unit 200 may augment the image data 250 through image shifting in the horizontal and vertical directions. Image shifting can be performed in various directions in addition to the horizontal and vertical directions, but the characteristics of the image data 250 generated according to the frequency conversion (for example, the frequency intensity based on the x-axis is displayed on the y-axis). It is desirable to perform image shifting only in the direction and vertical direction. That is, in order to improve image analysis performance, image shifting processing may be performed only in the horizontal direction and the vertical direction as described above. The ranges in the horizontal direction and the vertical direction to be shifted may be set differently according to a learning target, a structure of the detection model 400, and the like.

In the determination unit 300, the image data 250 is input to the detection model 400, and whether a defect occurs due to projection welding may be determined by the detection model 400. Defect detection can be classified into two types, normal or abnormal, and results can be derived. Alternatively, the classification result may be derived by dividing the degree of defect occurrence into a plurality of classes.

The detection model 400 is generated according to a result of pre-learning the relationship between the image data 250 and the defect information according to a deep learning algorithm, and the deep learning algorithm may include a convolutional neural network (CNN). In addition, the detection model 400 may be generated according to a learned result by combining with a recurrent neural network (RNN) in addition to the CNN. Hereinafter, a description will be made based on the detection model 400 formed according to the CNN-based learning result. The defect information relates to the presence or absence of defects or defects according to the welding obtained in advance with respect to the image data 250, and may be classified as normal or abnormal for each image data 250.

First, briefly describing CNN, CNN is a type of artificial neural network used to analyze input data (mainly, visual images). As shown in FIG. 4, the CNN 15 includes the feature extraction layer 17 and A classification layer 18 may be included. That is, a feature from the input data is extracted in the feature extraction layer 17, and a class to which the input data corresponds may be classified based on the feature extracted in the classification layer 18. The feature extraction layer 17 may include at least one convolutional layer and a pooling layer, and the classification layer is a fully connected layer including one hidden layer ( It may be a fully connected layer).

The convolution layer may correspond to a filter that generates a feature map representing features of an object through a convolution operation. That is, the CNN detects the characteristics of the image data 250 by detecting in what form a local pattern forming the image data 250 exists with respect to the image data 250 input to the convolutional layer. can do. The pattern may correspond to a feature existing between adjacent pixels, such as a shape, wave height, and period of a waveform in the image data 250. A stride may be performed to find a feature in the image data 250, and a stride size used in the convolution layer may be variously set.

In the pooling layer, a pooling operation may be performed to reduce the size of the output data of the convolution layer or to emphasize specific data. The pooling layer may include a max pooling layer and an average pooling layer.

The fully connected layer may correspond to a classfier for classifying input data based on the extracted feature information.

5 is an exemplary view showing a discrimination model in a projection welding defect detection system based on image recognition according to an embodiment of the present invention.

The discrimination model is preferably a first CNN 410 for extracting a first feature from the image data 250 and a second CNN for extracting a second feature from the image data 250 as shown in FIG. 5. 420, a classification layer 440 for determining whether a defect occurs based on the extracted first and second features, and the outputs of the first CNN 410 and the second CNN 420 are fused to each other, and the classification layer ( A fusion layer 430 formed to be input to 440 may be further included.

The first CNN 410 may correspond to a neural network formed based on a convolutional neural network (CNN) including a single convolutional layer. That is, the first CNN 410 is formed in a shallow structure with only the first convolution layer and the first pooling layer, as shown in FIG. 5, and the first characteristic from the input image data 250 Can be extracted.

Unlike this, the second CNN 420 may correspond to a neural network formed based on a convolutional neural network including a plurality of convolutional layers. That is, the second CNN 420 is formed in a deep structure with second to fourth convolution layers and a second pooling layer as shown in FIG. 5 and inputted from the image data 250. The second feature can be extracted.

A neural network including only one convolutional layer may correspond to a shallow neural network as described above, and a neural network including a plurality of convolutional layers may correspond to a deep neural network. Accordingly, the first feature extracted from the shallow structure of the first CNN 410 may be a shallow feature, and the second feature extracted from the deep structure of the second CNN 420 may be a deep feature.

The fusion layer 430 may be a layer that combines the outputs of the first CNN 410 and the second CNN 420 and connects them to become an input of the classification layer 440. The fusion layer 430 may be formed of a pooling layer.

The classification layer 440 is a fully connected layer described above, and the classification layer 440 includes at least one hidden layer and an output layer, as well as for changing dimensions. A flatten layer may be additionally included. The softmax activation function is used in the output layer, so that whether a defect occurs due to welding can be determined.

In more detail, as shown in FIG. 3, in the first CNN 410, a single first convolution layer 411 and a first pooling layer 412 are formed together to form a shallow convolutional neural network. have. Unlike this, in the second CNN 420, three convolution layers corresponding to the second convolution layer 421, the third convolution layer 422, and the fourth convolution layer 423, and a second pooling layer ( 424) can be connected to form a deep convolutional neural network. The number of filters and the filter size of the first to fourth convolution layers may be the same or may be set differently.

When the structure of the discrimination model is determined based on the features extracted from the shallow and deep neural networks, respectively, as shown in FIG. 5, the result is more accurate than the discrimination result according to the convolutional neural network of the general structure. Can be obtained. That is, after the feature extraction from the image data 250 was separately performed through the two convolutional neural networks, it was determined whether or not a defect occurred more accurately according to the result of fusion of it in the fusion layer 430.

In addition, a communication unit may be further included in the projection welding defect detection system based on image recognition according to an embodiment of the present invention. In the communication unit, as well as communication between components in the projection welding defect detection system, communication with other devices such as a welding machine or other systems may be performed. The communication method of the communication unit may be various wired or wireless communication methods, and is not limited to a specific communication method.

In the projection welding defect detection system according to an embodiment of the present invention, a data storage unit for storing welding data and image data 250 for each viewpoint may be further included. The data storage unit may be a database management system (DBMS) for interworking with a database and accessing data in the database.

In the projection welding defect detection system according to an embodiment of the present invention, an alarm generator for generating an alarm signal according to determination of whether a defect has occurred may be further included.

That is, when it is determined that a defect has occurred in the determination unit 300, an alarm signal is generated by the alarm generation unit of the detection system and is output by an alarm output unit provided in advance in the detection system, or the alarm signal is sent to the welding machine 20. By transmitting the alarm signal may be output from the welding machine (20). The alarm signal may be output in various forms. For example, the alarm signal may include a voice signal, a vibration signal, a display signal, and the like, but these are only exemplary and may be output in any form to notify the user of the occurrence of a welding defect.

In the projection welding defect detection system according to an exemplary embodiment of the present invention, a control unit for controlling a protrusion current applied from the welding machine 20 may be further included according to determination of whether or not a defect occurs.

The protrusion current relates to a current applied to a welding base material (a welding target) during projection welding, and may be data for controlling the operation of the welding machine 20. That is, as the protrusion current is adjusted, the welding strength (degree) may be different. In addition, the protrusion current may be consistent with the current data acquired from the welding machine 20, but may be processed separately for controlling the operation of the welding machine 20, so it is different from the current data obtained from the welding machine 20. It may be a different value. In other words, the protrusion current corresponds to a kind of target current value to be applied to the welding base material by the welding machine 20, and the above-described current data may correspond to the actually measured current value.

Preferably, the welding machine 20 includes a sensor module for measuring current, voltage, and pressure according to the progress of the projection welding, and a control module for controlling the protruding current, and the control module includes a defect in the projection welding defect detection system. When it is determined that is generated, the protrusion current is adjusted within a preset reference current range according to a control signal transmitted from the control signal of the projection welding defect detection system, and a defect occurs from the defect system for a predetermined reference time after the adjustment. Depending on whether or not the operation of the welding machine 20 may be stopped.

That is, in the control module capable of controlling the overall operation of the welding machine 20 as the real-time defect detection according to welding proceeds as described above, the operation of the welding machine is performed according to a control signal from the control unit of the projection welding defect detection system. It can be controlled, and in more detail, the protrusion current is primarily controlled within the reference current range to prevent additional defects, and real-time control so that the operation of the welding machine 20 is temporarily stopped when additional defects occur after a predetermined reference time. Can be performed. Accordingly, it is possible to quickly detect defects during welding, stop work immediately after detection, or prevent additional defects.

As an embodiment of the present invention, a method for detecting a projection welding defect using a projection welding defect detection system based on image recognition capable of interlocking with the welding machine 20 may be provided. The projection welding defect detection method of the present invention can be performed by the above-described projection welding defect detection system, and hereinafter, the same contents as those of the projection welding defect detection system are omitted.

Projection welding defect detection method according to an embodiment of the present invention is the step of obtaining welding data for each point of time output through projection welding by the welding machine 20, and obtaining image data 250 through image conversion of the welding data. It may include a step of determining whether or not a defect occurs due to projection welding from the image data 250 using the detection model 400 learned in advance according to the deep learning algorithm.

As an embodiment of the present invention, a computer-readable recording medium in which a program for implementing the above-described method is recorded may be provided.

That is, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. Further, the structure of data used in the above-described method can be recorded on a computer-readable medium through various means. A recording medium for recording executable computer programs or codes for performing the various methods of the present invention should not be understood as including temporary objects such as carrier waves or signals. The computer-readable medium may include a storage medium such as a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), and an optical reading medium (eg, CD-ROM, DVD, etc.).

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

Although the preferred embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also present. It belongs to the scope of rights of That is, the scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

20: welder
100: data acquisition unit
200: image conversion unit
300: discrimination unit
400: detection model

Claims (8)

In the image recognition-based projection welding defect detection system that can be linked with a welding machine,
A data acquisition unit for obtaining welding data for each time point output through projection welding by the welding machine;
An image conversion unit for obtaining image data through image conversion of the welding data;
A data storage unit for storing welding data and image data for each point in time;
A discrimination unit for determining whether or not a defect occurs due to the projection welding from the image data using a detection model learned in advance according to a deep learning algorithm;
A control unit for controlling a protruding current applied from the welding machine according to the determination of whether the defect has occurred; And
Includes; an alarm generator for generating an alarm signal according to the determination of whether or not the defect has occurred,
The welding data includes voltage data, current data, and pressure data,
The image conversion unit acquires the image data using a frequency conversion technique,
The detection model,
A first CNN for extracting a first feature from the image data;
A second CNN for extracting a second feature from the image data;
A classification layer for determining whether a defect has occurred based on the first feature and the second feature; And
A fusion layer formed such that the outputs of the first CNN and the second CNN are fused to be input to the classification layer; and
The first CNN includes a first convolution layer and a first pooling layer,
The second CNN is a projection welding defect detection system comprising a second convolution layer to a fourth convolution layer and a second pooling layer. The method of claim 1,
The classification layer,
A fully connected layer, comprising at least one hidden layer and an output layer, and a platen layer for dimensional change. The method of claim 2,
Wherein the first feature is a shallow feature, and the second feature is a deep feature. The method of claim 3,
The image conversion unit,
A projection welding defect detection system for performing image shifting processing in a horizontal direction and a vertical direction on the image data. The method of claim 4,
The data acquisition unit,
In order to obtain the welding data for each time point, the projection welding defect detection system is to sample the welding data based on a predetermined section unit. The method of claim 5,
The control unit,
The projection current is controlled within a preset reference current range according to the determination of whether or not the defect occurs, and control to stop the operation of the welding machine according to determination of the occurrence of the defect for a predetermined time after the adjustment Welding defect detection system.
delete delete

Images