CN115908292A

CN115908292A - Method and device for detecting defects of welding workpiece and storage medium

Info

Publication number: CN115908292A
Application number: CN202211404058.3A
Authority: CN
Inventors: 吴振廷; 林宗儒; 李若乔; 李剑光; 吴珈瑜; 干斌; 向飞
Original assignee: Fulian Yuzhan Technology Shenzhen Co Ltd
Current assignee: Fulian Yuzhan Technology Shenzhen Co Ltd
Priority date: 2022-11-10
Filing date: 2022-11-10
Publication date: 2023-04-04

Abstract

The embodiment of the application provides a method, a device and a storage medium for detecting the defects of a welding workpiece, wherein the method comprises the following steps: acquiring a welding image of a welding workpiece and preset welding position information of the welding workpiece; inputting the welding image of the welding workpiece into a first detection model trained in advance to obtain the outline of the welding workpiece in the welding image output by the first detection model; the first detection model is trained in advance and used for extracting the outline characteristics of the input welding image and outputting a model of the outline of the welding workpiece in the welding image; determining whether the welding position of the welding workpiece is abnormal or not according to the contour of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece; and if the position deviation is abnormal, determining that the position deviation defect of the welding workpiece exists. The method is used for improving the defect detection efficiency of the welding workpiece.

Description

Method and device for detecting defects of welding workpiece and storage medium

Technical Field

The present disclosure relates to the field of defect detection technologies, and in particular, to a method and an apparatus for detecting defects of a welding workpiece, and a storage medium.

Background

In order to improve the product quality, it is usually necessary to detect whether welding defects such as missing welding, missing mounting, welding spot climbing wall, etc. exist in a welding workpiece. When detecting the defects of the welding workpiece, the method aims at the detection of welding defects such as missing welding, wall climbing of welding spots and the like and is established on the basis that the welding workpiece does not have the defect of welding position degree deviation. If the welding position degree of the welding workpiece deviates, the deviation of the welding position of the welding workpiece is shown, and at the moment, even if other defects such as welding spot wall climbing, welding missing and the like do not exist in the welding workpiece, the welding workpiece cannot be continuously used, so that the product of the welding workpiece cannot be used due to the deviation of the welding position degree. Therefore, when detecting a welding defect of a welding workpiece, it is generally necessary to detect whether or not there is a deviation in the welding position degree of the welding workpiece. At present, the detection method of welding defects such as welding position degree deviation of a welding workpiece generally adopts a traditional image algorithm and a visual method. The traditional image algorithm has the defect of insufficient robustness, and the traditional image algorithm has high relevance of products and images and low detection efficiency. The visual inspection method needs manual participation, has low detection efficiency and has the condition of missing detection.

Disclosure of Invention

In view of this, the present application provides a method, an apparatus and a storage medium for detecting defects of a welding workpiece, so as to solve the problem of low detection efficiency of the welding workpiece in the prior art.

In a first aspect, an embodiment of the present application provides a method for detecting defects of a welded workpiece, including:

acquiring a welding image of a welding workpiece and preset welding position information of the welding workpiece;

inputting the welding image of the welding workpiece into a first detection model trained in advance to obtain the outline of the welding workpiece in the welding image output by the first detection model; the first detection model is trained in advance and used for extracting the outline characteristics of the input welding image and outputting a model of the outline of the welding workpiece in the welding image;

determining whether the welding position of the welding workpiece is abnormal or not according to the contour of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece;

if the position deviation is abnormal, determining that the position deviation defect of the welding workpiece exists;

if the welding position of the welding workpiece is not abnormal, acquiring a target welding image; the target welding image is a welding image at a welding position of the welding workpiece;

inputting the target welding image into a pre-trained second detection model to obtain welding information of a welding workpiece in the target welding image output by the second detection model; the second detection model is trained in advance and is used for performing semantic cutting on the input target welding image, extracting multi-channel welding characteristic information and outputting welding information of a welding workpiece in the target welding image;

and determining whether the welding workpiece has welding defects according to the welding information of the welding workpiece.

Preferably, the determining whether the welding position of the welding workpiece is abnormal according to the profile of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece includes:

determining the welding position of the welding workpiece in the welding image according to the contour of the welding workpiece output by the first detection model;

calculating the spacing distance between the welding position of the welding workpiece in the welding image and the preset welding position of the welding workpiece according to the welding position of the welding workpiece in the welding image and the preset welding position information of the welding workpiece;

acquiring a preset distance threshold value, and judging whether the spacing distance is greater than the preset distance threshold value;

and if the spacing distance is greater than a preset distance threshold value, determining that the welding position of the welding workpiece is abnormal.

Preferably, the first detection model includes: a first encoder, a first decoder and a first classifier;

the first encoder is used for extracting outline characteristic information in the input welding image;

the first decoder is used for performing deconvolution and feature splicing on the contour feature information extracted by the first encoder to obtain first feature information;

and the first classifier is used for classifying each pixel in the welding image based on first characteristic information to obtain the welding position information of the welding workpiece.

Preferably, the second detection model includes: a second encoder, a second decoder and a second classifier;

the second encoder is used for performing semantic cutting on the input target welding image and extracting multi-channel welding characteristic information;

the second decoder is used for respectively carrying out up-sampling and feature fusion on the multi-channel welding feature information to obtain second feature information;

and the second classifier is used for classifying the second characteristic information to obtain the welding information of the welding workpiece.

Preferably, the welding information includes: at least one of a weld spot profile, a first workpiece profile, a second workpiece profile, a weld spot center, a defect spot.

Preferably, the determining whether the welding workpiece has the welding defect according to the welding information of the welding workpiece comprises:

when the welding information contains a welding spot center, determining whether the defects of welding missing, welding missing or welding spot lack exist according to the welding spot center; alternatively, the first and second electrodes may be,

when the welding information comprises a first workpiece outline and a welding spot outline, determining whether the defect of welding spot deviation exists according to the first workpiece outline and the welding spot outline; alternatively, the first and second electrodes may be,

when the welding information comprises a welding spot outline and a second workpiece outline, determining whether the defect of wall climbing of the welding spot exists according to the second workpiece outline and the welding spot outline; alternatively, the first and second electrodes may be,

when the welding information comprises a first workpiece outline, determining a first workpiece angle according to the first workpiece outline, and determining whether a defect of the first workpiece angle exists according to the first workpiece angle; alternatively, the first and second electrodes may be,

when the welding information comprises a second workpiece outline, detecting whether the second workpiece outline accords with a preset rule, and determining whether the second workpiece has a deformed defect; alternatively, the first and second electrodes may be,

and when the welding information comprises the welding spot center and the defect point, determining whether the defect of the gap exists according to the welding spot center and the defect point.

Preferably, said determining said first workpiece angle from said first workpiece profile comprises:

determining the position of the first workpiece contour in a first coordinate system according to the first workpiece contour;

the first workpiece angle is determined based on a position of the first workpiece profile in a first coordinate system.

Preferably, the step of detecting whether the second workpiece contour meets a preset rule comprises the following steps:

determining the position of the second workpiece contour in a second coordinate system according to the second workpiece contour;

determining the shape corresponding to the second workpiece outline according to the position of the second workpiece outline;

and judging whether the shape corresponding to the outline of the second workpiece meets a preset rule or not.

Preferably, the acquiring the target welding image includes:

and cutting the welding position area of the welding workpiece in the welding image according to the contour of the welding workpiece to obtain the target welding image.

Preferably, the detection method further comprises:

when the welding defect of the welding workpiece is determined, display information of the welding defect of the welding workpiece is formed, and the appearance or the characteristic value of the welding defect is displayed according to the display information.

In a second aspect, an embodiment of the present application provides a defect detection apparatus for a welding workpiece, including:

the communicator is used for receiving a welding image of a welding workpiece from an image acquisition module and preset welding position information of the welding workpiece from an input device;

the processor is coupled with the communicator and used for inputting the welding image of the welding workpiece into a first detection model trained in advance to obtain the outline of the welding workpiece in the welding image output by the first detection model; the first detection model is trained in advance and used for extracting the outline characteristics of the input welding image and outputting a model of the outline of the welding workpiece in the welding image;

Preferably, the first detection model includes: a first encoder, a first decoder and a first classifier; the first encoder is used for extracting outline characteristic information in the input welding image; the first decoder is used for performing deconvolution and feature splicing on the contour feature information extracted by the first encoder to obtain first feature information; and the first classifier is used for classifying each pixel in the welding image based on first characteristic information to obtain the welding position information of the welding workpiece.

Preferably, the second detection model includes: a second encoder, a second decoder, and a second classifier; the second encoder is used for performing semantic cutting on the input target welding image and extracting multi-channel welding characteristic information; the second decoder is used for respectively carrying out up-sampling and feature fusion on the multi-channel welding feature information to obtain second feature information; the second classifier is used for classifying the second characteristic information to obtain welding information of the welding workpiece;

wherein the welding information comprises: at least one of a weld spot profile, a first workpiece profile, a second workpiece profile, a weld spot center, a defect spot.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and when the program runs, a device on which the computer-readable storage medium is located is controlled to execute the method described in any one of the foregoing first aspects.

By adopting the scheme provided by the embodiment of the application, the welding image of the welding workpiece can be input into the first detection model trained in advance, and the outline of the welding workpiece in the welding image is output through the first detection model, so that whether the welding position of the welding workpiece is deviated or not is determined according to the outline of the welding workpiece output by the first detection model and the preset welding position information of the welding workpiece, and if the deviation is determined, the defect of deviation of the position degree of the welding workpiece is determined; for the welding workpiece with no abnormal welding position, the corresponding welding image is further input into the second detection model, and the second detection model extracts different welding characteristic information in the welding image, so that the detection of different welding defects of the welding workpiece can be further realized, and the accuracy of the detection of the defects of the welding workpiece is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.

FIG. 1 is a schematic structural diagram of a defect detection apparatus for a welded workpiece according to an embodiment of the present disclosure;

fig. 2a is a schematic view of a scene of defect detection of a welded workpiece according to an embodiment of the present application;

FIG. 2b is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 2c is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 3 is a schematic flowchart of a method for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of a training method for a first detection model according to an embodiment of the present disclosure;

FIG. 5a is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 5b is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 5c is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 6 is a schematic flowchart of another method for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 7a is a schematic view of a scene of defect detection of another welding workpiece according to an embodiment of the present application;

FIG. 7b is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

fig. 8 is a schematic flowchart of a training method for a second detection model according to an embodiment of the present disclosure;

FIG. 9a is a schematic view of a scene of another defect detection of a welded workpiece according to an embodiment of the present application;

FIG. 9b is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

FIG. 9c is a schematic view of a scene of another defect detection for a welded workpiece according to an embodiment of the present application;

FIG. 9d is a schematic view of another exemplary scenario for detecting defects of a welded workpiece according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of another defect detection apparatus for a welding workpiece according to an embodiment of the present application.

Detailed Description

For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.

It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be understood that the term "and/or" as used herein is merely one type of associative relationship that describes an associated object, meaning that three types of relationships may exist, e.g., A and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

In order to improve the product quality, it is usually necessary to detect whether welding defects such as missing welding, missing mounting, wall climbing of welding spots, and the like exist in a welding workpiece. When detecting the defects of the welding workpiece, the method aims at the detection of welding defects such as missing welding, wall climbing of welding spots and the like and is established on the basis that the welding workpiece does not have the defect of welding position degree deviation. If the welding position degree of the welding workpiece deviates, the deviation of the welding position of the welding workpiece is shown, and at the moment, even if other defects such as welding spot wall climbing, welding missing and the like do not exist in the welding workpiece, the welding workpiece cannot be continuously used, so that the product of the welding workpiece cannot be used due to the deviation of the welding position degree. Therefore, when detecting a welding defect of a welding workpiece, it is generally necessary to first detect whether or not there is a deviation in the welding position degree of the welding workpiece. At present, the detection method of welding defects such as welding position degree deviation of a welding workpiece generally adopts a traditional image algorithm and a visual method. The traditional image algorithm has the defect of insufficient robustness, and the traditional image algorithm has high relevance of products and images and low detection efficiency. The visual inspection method needs manual participation, has low detection efficiency and has the condition of missing detection.

In view of the above problems, embodiments of the present application provide a method and an apparatus for detecting a defect of a welding workpiece, and a storage medium, which can acquire a welding image at a welding position of a welding workpiece when a deviation detection needs to be performed on the welding position of the welding workpiece. Inputting the welding image into a first detection model trained in advance, outputting welding position information of the welding image in the welding workpiece through the first detection model, determining whether the welding position of the welding workpiece deviates according to the welding position information of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece, and determining that the position deviation defect of the welding workpiece exists if the deviation is determined. In the application, aiming at different welding workpieces, the welding position information of the welding workpiece can be output through the first detection model, and then the welding position information is compared with the preset welding position to determine whether the welding position of the welding workpiece deviates or not, the robustness is enhanced, and in the process of detecting whether the welding position of the welding workpiece has the deviation defect or not, manual participation is not needed, and the detection efficiency is improved. The details will be described below.

Referring to fig. 1, a hardware architecture diagram of a defect detection apparatus for a welded workpiece according to an embodiment of the present application is shown. The embodiment of the application provides a defect detection device 100 for welding workpieces, which comprises a communicator 101 and a processor 102.

A communicator 101 for establishing a communication channel so that the welding workpiece defect detecting apparatus 100 can communicate with other devices. Receiving data sent by other devices or sending data to other devices. In the embodiment of the present application, the communicator 101 may receive a welding image of a welding workpiece collected by an image collecting module and preset welding position information of the welding workpiece from an input device. The image acquisition module is used for acquiring images of welded workpieces, and can be a CCD (charge coupled device) camera or other imaging devices. The image acquisition module can adopt high angle, for example 70 degrees, and the collection of welding image is carried out to cyclic annular mode of polishing, can guarantee like this that the welding image of gathering is more clear, and the shadow is more balanced, as shown in fig. 2a, fig. 2b and fig. 2 c. The welded workpiece comprises at least two workpieces welded together. For example, studs and flanges may be included. The welding images of the stud and the flange welded together are acquired through the image acquisition module, and the welding images are sent to the communicator 101. The communicator 101 transmits the received welding image and the preset welding position information to the processor 102, calls the first detection model through the processor 102, realizes analysis and extraction of the welding position in the welding image, and finally determines whether the defect of position degree deviation of the welding workpiece exists or not.

The processor 102 may be the control center of the defect detection apparatus 100 for the welding workpiece, various portions of the defect detection apparatus 100 connecting the entire welding workpiece using various interfaces and lines, and executing various functions of the defect detection apparatus 100 for the welding workpiece and/or processing data by running or executing software programs and/or modules stored in the memory 103 and calling data stored in the memory. The processor 102 may be formed of an Integrated Circuit (IC), for example, a single packaged IC, or a plurality of packaged ICs with the same or different functions connected. For example, the processor 102 may include only a Central Processing Unit (CPU). In the embodiment of the present invention, the CPU may be a single operation core, or may include multiple operation cores.

Referring to fig. 3, a flowchart of a method for detecting defects of a welded workpiece according to an embodiment of the present application is shown. The method is applied to a defect detecting apparatus for a welded workpiece (hereinafter simply referred to as a defect detecting apparatus) shown in fig. 1. As shown in fig. 3, the method for detecting defects of a welded workpiece includes:

and S301, acquiring a welding image of the welding workpiece and preset welding position information of the welding workpiece.

In the embodiment of the application, in order to ensure the welding quality, the welding position of the welding workpiece needs to be subjected to defect detection so as to determine whether the defects of welding position degree deviation, welding missing, welding spot wall climbing and the like exist. When the defect detection is carried out on the welding position of the welding workpiece, the detection device is determined to carry out image acquisition on the welding position of the welding workpiece, so that a welding image is obtained. Of course, the welding image of the welding position of the welding workpiece can be acquired in advance through other devices, and at the moment, the defect detection device can directly acquire the welding image from other devices without acquiring by itself.

When welding workpieces, various welding parameters are preset so as to ensure that corresponding welding is carried out at corresponding positions. The welding parameters are preset according to actual welding requirements, and at the moment, the defect detection device of the welding workpiece can acquire preset welding position information of the welding workpiece in order to detect whether the welding workpiece has the defect of welding position degree deviation.

It should be understood that the defect detection apparatus may obtain the pre-welding position information from a memory therein, or from other external devices, and may also receive the pre-welding position information input by the user, which is not limited in this application.

Further, the welding image of the welding workpiece is an image after welding the first workpiece and the second workpiece. The first workpiece may be a flange and the second workpiece may be a stud. That is, the welding workpiece includes a first workpiece and a second workpiece, the first workpiece is a flange, and the second workpiece is a stud. The welding image is an image of the first workpiece and the second workpiece after welding, namely an image formed after welding the flange and the stud.

Step S302, inputting the welding image of the welding workpiece into a first detection model trained in advance, and obtaining the outline of the welding workpiece in the welding image output by the first detection model.

The first detection model is trained in advance and used for extracting the outline features of the input welding image and outputting a model of the outline of the welding workpiece in the welding image.

In the embodiment of the application, in order to conveniently, accurately and quickly determine the position information of the welding workpiece in the welding image, the first detection model capable of outputting the outline of the welding workpiece in the welding image is trained in advance. In this way, the defect detection device can input the welding image to the first detection model as an input of the first detection model after acquiring the welding image. The first detection model performs processing such as welding feature extraction and stitching on the input welding image to obtain the contour of the welding workpiece in the welding image, and outputs the contour of the welding workpiece in the welding image, as shown in fig. 5 a. After the defect detection device receives the contour of the welding workpiece in the welding image output by the first detection model, the position of the welding workpiece can be determined according to the contour of the welding workpiece in the welding image.

It should be understood that the welded workpiece includes a first workpiece and a second workpiece. In this case, the contour of the welded workpiece output by the first detection model may be the contour of the first workpiece, the contour of the second workpiece, or the contours of the first workpiece and the second workpiece, which may be set according to actual requirements, but the present application is not limited thereto. The defect detection model can detect the position of the first workpiece or the position of the second workpiece in the welding image, so that whether the welding workpiece in the welding image has the defect of welding position deviation can be determined. Therefore, in order to reduce the training complexity of the first detection model, the profile of the welding workpiece output by the first detection model may be only the profile of the first workpiece, or the profile of the second workpiece.

As a possible implementation, as shown in fig. 4, the first detection model may be trained by:

step S401, a first image is obtained.

In the embodiment of the present application, a first welding sample image may be acquired, and the contour information of the welding workpiece of the first welding sample image is marked in advance by a user. After a first welding sample image is obtained, the first welding sample image is cut, rotated, turned and the like to form more images so as to form a training sample set of a first preset network model. A first image is identified in a training sample set. Therefore, more images are formed by cutting, rotating, overturning and the like of the first welding sample image, the training sample set of the first preset network model can be expanded, the accuracy of the training of the first preset network model is improved, the manual participation in the formation of the training sample set of the first preset network model can be reduced, the training efficiency of the first preset network model is improved, and the training cost is reduced.

Step S402, inputting the first image into a first preset network model.

In the embodiment of the present application, the first preset network model needs to be trained to form the first detection model. At this time, the acquired first image may be input to the first preset network model.

Step S403, the first preset network model performs convolution processing and pooling processing on the input first image, and extracts feature information of each contour in the first image. And performing deconvolution and feature splicing processing on the feature information of each contour in the first image to obtain first feature information. And classifying each pixel in the first image by a classifier based on the first characteristic information to obtain the contour of the welding workpiece in the first image.

In the embodiment of the application, the first image is transmitted to the first preset network model, the first preset network model performs convolution processing and pooling processing on the first image, multi-channel feature information of the first image is extracted, and feature information of each contour in the first image is formed. And the first preset network model performs deconvolution processing on the feature information of each contour in the first image to form a deconvolution result. Since the first image is subjected to the pooling process a plurality of times, the size of the first image becomes small. While the deconvolution process increases the size of the image. However, the image after deconvolution processing is only increased in size and cannot restore the first image, so in order to reduce data loss, the convolution result after deconvolution processing is usually cut into deconvolution size-first images, and then the deconvolution size-first images are directly spliced together, so that the feature information of the images is increased. Namely, the deconvolution result and the convolution result after the convolution processing are subjected to feature splicing processing to obtain first feature information. The first feature information includes contour information of the welding workpiece of the first image. And classifying the first characteristic information by utilizing the two classification convolution layers, and outputting an image and a background image of the outline of the welding workpiece.

And S404, acquiring the welding workpiece outline mark information of the first image.

Wherein the welding workpiece contour mark information of the first image is the contour information of the welding workpiece in the first image marked in advance.

Step S405, calculating a first loss value according to the welding workpiece outline output by the first preset network model and the welding workpiece outline marking information of the first image by using a first preset loss function.

In this embodiment, after the first preset network model outputs the welding position information of the welding workpiece in the first image, the difference between the welding position information of the welding workpiece in the first image and the welding workpiece outline mark information of the first image may be calculated by using the first preset loss function, so as to obtain the first loss value.

Step S406, judging whether the first loss value is greater than a first preset threshold value, if so, adjusting model parameters of the first preset network model, and re-executing the steps S401 to S406 until the first loss value is less than the first preset threshold value, so as to obtain a first detection model.

In this embodiment of the application, when the first loss value is greater than the first preset threshold, it indicates that the welding position information error of the welding workpiece in the first image output by the first preset network model is large, and at this time, the model parameters of the first preset network model, such as the parameters of the convolution kernel, the pooling kernel, the deconvolution kernel, and the like, may be adjusted to reduce the contour error of the welding workpiece in the first image output by the first preset network model. After the model parameters of the first preset network model are adjusted, the steps S401 to S406 may be executed again, and the model parameters of the first preset network model are continuously adjusted, so that the result output by the first preset network model is more and more accurate until the first loss value is smaller than the first preset threshold value, and at this time, the training of the first preset network model may be considered to be completed, and the first detection model is obtained.

It should be noted that the first preset network model is a preset neural network model, and may be a U-Net neural network model, or may be another neural network model, which is not limited in this application.

It should be noted that the first preset loss function is a preset function for calculating a loss value, and the function may be a cross entropy loss function or other loss functions, which is not limited in this application. The first preset threshold is preset according to actual requirements and is used for balancing the threshold for judging whether the training of the first preset network model is completed. When the first loss value is larger than the first preset threshold value, the error of the result output by the first preset network model is considered to be larger, and the first preset network model needs to be trained continuously. When the first loss value is smaller than the first preset threshold, the result output by the first preset network model is considered to be more accurate, and at this time, the training of the first preset network model can be finished.

The training of the first preset network model can be completed through the steps S401 to S406 to obtain the first detection model, so that the first detection model can be directly used when the welding position of the welding workpiece is subjected to abnormal detection, and the detection complexity is greatly reduced. In addition, the first detection model can be suitable for various welding workpieces, and the robustness of welding position abnormity detection is improved.

As a possible implementation, the first detection model includes: the device comprises a first encoder, a first decoder and a first classifier. Wherein the content of the first and second substances,

the first encoder is used for extracting contour characteristic information in the input welding image.

And the first decoder is used for performing deconvolution and feature splicing on the contour feature information extracted by the first encoder to obtain first feature information.

And the first classifier is used for performing convolution processing on the first characteristic information to obtain the profile of the welding workpiece.

In the embodiment of the present application, the first detection model includes a first encoder, a first decoder, and a first classifier. After the welding image is input into the first detection model, a first encoder of the first detection model performs multi-channel convolution processing and pooling processing on the input welding image by using a preset convolution kernel and a preset pooling kernel to extract feature information of each contour in the welding image. And the first decoder performs characteristic splicing on the characteristic information obtained by performing deconvolution on the contour characteristic information extracted by the first encoder and the characteristic information in the multi-channel welding image extracted by performing convolution processing in the first encoder to obtain first characteristic information of the size of the welding image. And inputting the first characteristic information into a first classifier, classifying the welding workpiece and the background in the welding image by the first classifier, and outputting the outline of the welding workpiece in the welding image.

That is to say, the first detection model can extract the welding information of various channels from the welding image input into the first detection model through the cooperation between different devices to obtain the contour of the welding workpiece in the welding image, so that the first detection model can accurately extract the contour of each welding workpiece in the welding image, and the accuracy of the defect detection of the subsequent welding workpiece is improved.

Step S303, determining whether the welding position of the welding workpiece is abnormal according to the contour of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece.

In the embodiment of the application, after the first detection model outputs the contour of the welding workpiece in the welding image, the defect detection device can position the welding position of the welding workpiece according to the output contour of the welding workpiece, so as to determine whether the welding position of the welding workpiece is abnormal or not, that is, whether the welding position is deviated or not, according to the welding position of the welding workpiece and preset welding position information of the welding workpiece.

As a possible implementation manner, determining whether the welding position of the welding workpiece is abnormal according to the profile of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece includes:

determining the welding position of the welding workpiece in the welding image according to the contour of the welding workpiece output by the first detection model; calculating the spacing distance between the welding position of the welding workpiece in the welding image and the preset welding position of the welding workpiece according to the welding position of the welding workpiece in the welding image and the preset welding position information of the welding workpiece; acquiring a preset distance threshold value, and judging whether the spacing distance is greater than the preset distance threshold value; if the spacing distance is larger than the preset distance threshold value, determining that the welding position of the welding workpiece is abnormal, otherwise, determining that the welding position of the welding workpiece is not abnormal.

That is, after receiving the contour of the welding workpiece in the welding image output by the first detection model, the defect detection device may determine the welding position of the welding workpiece according to the position of the contour of the welding workpiece in the welding image. For example, the welding position of the welding workpiece may be determined based on the pixel position of the welding workpiece in the welding image. After the welding position of the welding workpiece is determined, the welding position of the welding workpiece can be compared with preset welding position information of the welding workpiece, and an offset distance is calculated, namely the spacing distance between the welding position of the welding workpiece in the welding image and the preset welding position of the welding workpiece is calculated; and acquiring a preset distance threshold, comparing the calculated spacing distance with the preset distance threshold, and judging whether the calculated spacing distance is greater than the preset distance threshold. When the calculated distance is greater than the preset distance threshold, it is determined that the welding position of the welding machine deviates from the preset welding position when the welding machine is welding the workpiece, and it may be determined that the welding position of the welding workpiece is abnormal at this time, as shown in fig. 5 b.

Or when the calculated spacing distance is not greater than the preset distance threshold, it is determined that the welding position of the welding machine does not deviate from the preset welding position when the welding machine is welding the workpiece, and it can be determined that the welding position of the welding workpiece is not abnormal.

It should be noted that the preset distance threshold is preset according to actual requirements, and is used for measuring a threshold for judging whether a welding position of a welding workpiece welded by a welding machine deviates or not. That is, when the distance between the welding position of the welding workpiece and the preset welding position of the welding workpiece is greater than the preset distance threshold, it is determined that the welding position of the welding workpiece is shifted from the preset welding position, and it may be determined that the welding position of the welding workpiece is abnormal. When the distance between the welding position of the welding workpiece and the preset welding position of the welding workpiece is not greater than the preset distance threshold, it is determined that the welding position of the welding workpiece does not deviate from the preset welding position, and it may be determined that there is no abnormality in the welding position of the welding workpiece, as shown in fig. 5 c.

That is to say, the defect detection device can confirm the welding position of the welding workpiece in the welding image through the welding profile output by the first detection model, and can detect whether the welding position of the welding workpiece deviates from the preset welding position of the welding workpiece by comparing the welding position of the welding workpiece determined by the defect detection device with the preset welding position information of the welding workpiece, so that the detection result is accurate, the detection time can be shortened, and the detection cost is increased.

And step S304, if the position degree deviation defect of the welding workpiece exists, determining that the position degree deviation defect of the welding workpiece exists.

In the embodiment of the present application, when the defect detection device determines that the welding position of the welding workpiece is abnormal, it may determine that the welding workpiece has a positional deviation defect of the welding workpiece.

Aiming at different welding workpieces, the welding position information of the welding workpiece can be output through the first detection model and then is compared with the preset welding position to determine whether the welding position of the welding workpiece deviates or not, robustness is enhanced, manual participation is not needed in the process of detecting whether the welding position of the welding workpiece has the deviation defect, and detection efficiency is improved.

Fig. 6 is a schematic flowchart of another method for detecting defects of a welded workpiece according to an embodiment of the present application. The method is applied to the defect detecting device shown in FIG. 1. Compared with the defect detection method shown in fig. 3, the defect detection method according to the embodiment of the present application adds other detection steps of welding defects. As shown in fig. 6, the method includes:

step S601, obtaining a welding image of a welding workpiece and preset welding position information of the welding workpiece.

For details, reference may be made to step S301, which is not described herein again.

Step S602, inputting the welding image of the welding workpiece to a first detection model trained in advance, and obtaining the outline of the welding workpiece in the welding image output by the first detection model.

Specifically, the step S302 is not described herein again.

Step S603, determining whether the welding position of the welding workpiece is abnormal according to the profile of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece.

Specifically, reference to step S303 is not repeated herein.

The following steps are performed by the defect detection device depending on the determination result. When it is determined that the welding position of the welding workpiece is abnormal, the following step S604 is executed. Upon determining that the welding position of the welding workpiece is not abnormal, the following step S605 is executed.

And step S604, if the position deviation is abnormal, determining that the position deviation defect of the welding workpiece exists.

Specifically, the step S304 is not described herein again.

And step S605, if the welding position of the welding workpiece is not abnormal, acquiring a target welding image.

Wherein the target welding image is a welding image at a welding position of the welding workpiece.

In the embodiment of the present application, when it is determined that the welding position of the welding workpiece is not abnormal, the defect detection device indicates that the welding position of the welding workpiece is not shifted, and at this time, it may further detect whether there is another welding defect in the welding workpiece. The defect detection device may acquire a target welding image of the welding workpiece, that is, an image at a welding position of the welding workpiece.

As one possible implementation, acquiring the target welding image includes: and cutting the welding position area of the welding workpiece in the welding image according to the outline of the welding workpiece to obtain a target welding image.

That is, in the step S602, the contour of the welding workpiece in the captured welding image is output, and at this time, the welding position of the welding workpiece may be determined according to the output contour of the welding workpiece, and further, the welding position region of the welding workpiece in the welding image may be cut, and other contents included in the welding image may be deleted, and only the part of the welding position of the welding workpiece is retained, so as to obtain the target welding image, which is shown in fig. 7 a.

In this way, the detection device cuts the welding position area of the welding workpiece in the welding image through the contour of the welding workpiece output by the first detection model to obtain the target welding image, namely, the part only reserving the welding position area of the welding workpiece, so that the complexity of the target welding image can be reduced, and the accuracy of detecting other defects of the welding workpiece in the welding image can be improved.

And step S606, inputting the target welding image into a pre-trained second detection model to obtain welding information of the welding workpiece in the target welding image output by the second detection model.

The second detection model is trained in advance and used for performing semantic cutting on the input target welding image, extracting multi-channel welding characteristic information and outputting welding information of a welding workpiece in the target welding image.

In the embodiment of the present application, in order to more conveniently and accurately detect the welding defect of the welding workpiece, a model capable of outputting the welding information of the welding workpiece in the input welding image may be trained in advance. The defect detection device can input the acquired target welding image into the second detection model, and the second detection model extracts the welding outline of the target welding image and outputs the welding information of the welding workpiece.

It should be understood that welding information is information that can characterize the welding of various welding workpieces.

As one possible implementation, the welding information includes: at least one of a weld spot profile, a first workpiece profile, a second workpiece profile, a weld spot center, a defect point. Therefore, different welding characteristic information in the target welding image can be extracted to obtain the welding information of the welding workpiece, so that when the welding information is used for detecting the defects of the welding workpiece, the detection of different welding defects of the welding workpiece can be realized, and the accuracy of the detection of the defects of the welding workpiece is improved.

As a possible implementation, the first workpiece may be a flange, the second workpiece may be a stud, and the welding information includes: at least one of a weld profile, a flange profile, a stud profile, a weld center, a defect point, as described with reference to fig. 7 b.

As a possible implementation, as shown in fig. 8, the second detection model may be trained by:

step S801, a second image is acquired.

In this embodiment, a second welding sample image may be obtained, and the user marks welding information of the welding workpiece in the second welding sample image in advance, for example, at least one of a welding spot profile, a flange profile, a stud profile, a welding spot center, and a defect point in the second welding sample image is marked. After a second welding sample image is obtained, the second welding sample image is cut, rotated, turned over and the like to form more images, so that a training sample set of a second preset network is formed. A second image is determined in the training sample set.

And S802, inputting a second image into a second preset network model.

In the embodiment of the present application, the second predetermined network model needs to be trained to form the second detection model. At this time, the acquired second image may be input to the second preset network model.

And S803, performing convolution processing and pooling processing on the input second image by using a second preset network model, and extracting the characteristic information of each welding workpiece in the second image. And performing deconvolution and feature splicing processing on the feature information of each welding workpiece in the second image to obtain second feature information. And classifying each pixel in the second image by a classifier based on the second characteristic information to obtain welding information in the second image.

In this embodiment, the second image is transmitted to the second preset network model, and the second preset network model performs convolution processing and pooling processing on the second image, and extracts multi-channel feature information of the second image, such as feature information of a welding spot, feature information of a flange profile, feature information of a stud profile, feature information of a welding spot center, and the like, to form feature information of the welding workpiece in the second image. And the second preset network model performs deconvolution processing on the characteristic information of the welding workpiece in the second image to form a deconvolution result. Since the second image is subjected to the pooling process a plurality of times, the size of the second image becomes small. While the deconvolution process increases the size of the image. However, the size of the image after the deconvolution processing is only increased, and the second image cannot be restored, so in order to reduce data loss, the convolution result after the deconvolution processing is usually cut into images with sizes communicated first by deconvolution, and then the images are directly spliced together, so that the characteristic information of the images is increased. Namely, the deconvolution result and the convolution result after the convolution processing are subjected to feature splicing processing to obtain second feature information. The second characteristic information includes characteristic information of the welding workpiece of the second image. And classifying the second characteristic information by utilizing the classified convolution layer, and outputting the welding information of the welding workpiece. The classified convolution layer may be a single classified convolution layer or a multi-layered classified convolution layer. When the classified buildup layer is a multilayer, the classified buildup layer can be classified into a flange contour classified buildup layer, a stud contour classified buildup layer, a solder joint classified buildup layer, a notch classified buildup layer, and a solder joint center classified buildup layer. After the second characteristic information is transmitted to the classified convolution layer, the characteristic classification is carried out on the second characteristic information through the classified convolution layer, and the welding information containing different welding characteristics is formed.

And step S804, acquiring welding mark information of the welding workpiece of the second image.

Wherein the welding workpiece welding mark information of the second image is welding information for the welding workpiece in the second image marked in advance.

Step S805, calculating a second loss value according to the welding information of the welding workpiece and the welding mark information of the welding workpiece of the second image output by the second preset network model by using a second preset loss function.

In this embodiment, after the second preset network model outputs the welding information of the welding workpiece in the second image, the second preset loss function may be used to calculate the difference between the welding information of the welding workpiece in the second image and the welding mark information of the welding workpiece in the first image, so as to obtain a second loss value.

Step S806, judging whether the second loss value is larger than a second preset threshold value; and when the second loss value is greater than the second preset threshold value, adjusting the model parameters of the second preset network model, and re-executing the steps S801-S806 until the second loss value is less than the second preset threshold value, so as to obtain a second detection model.

In this embodiment, when the second loss value is greater than the second preset threshold, it indicates that the error of the welding information of the welding workpiece in the second image output by the second preset network model is large, and at this time, the model parameters of the second preset network model, such as the convolution kernel, the pooling kernel, the deconvolution kernel, and other parameters, may be adjusted to reduce the error of the welding information of the welding workpiece in the second image output by the second preset network model. After the model parameters of the second preset network model are adjusted, the above steps S801 to S806 may be re-executed, and the model parameters of the second preset network model are continuously adjusted, so that the result output by the second preset network model is more and more accurate until the second loss value is smaller than the second preset threshold value, and at this time, the second preset network model may be considered to be trained completely, and the second detection model is obtained.

It should be noted that the second preset network model is a preset neural network model, and may be a U-Net neural network model, or may be another neural network model, which is not limited in this application.

It should be noted that the second preset loss function is a preset function for calculating a loss value, and the function may be a cross entropy loss function or other loss functions, which is not limited in this application. The second preset threshold is preset according to actual requirements and is used for balancing the threshold for finishing the training of the second preset network model. And when the second loss value is larger than the second preset threshold value, considering that the error of the result output by the second preset network model is larger, and needing to continue training the second preset network model. When the second loss value is smaller than the second preset threshold value, the result output by the second preset network model is considered to be more accurate, and the training of the second preset network model can be ended at this moment.

Through the above steps S801 to S806, the training of the second preset network model can be completed to obtain the second detection model, so that when the target welding image is input to the second detection model, the second detection model can extract multi-channel welding characteristic information of the target welding image, output the welding information of the welding workpiece in the target welding image, and improve the accuracy of the welding information. And when the welding information of the welding workpiece is subsequently utilized to detect whether other welding defects exist in the welding workpiece, the accuracy of defect detection can be improved, manual participation is not needed, the detection time is prolonged, and the detection cost is reduced.

As a possible implementation, the second detection model includes: a second encoder, a second decoder, and a second classifier. Wherein the content of the first and second substances,

and the second encoder is used for performing semantic cutting on the input target welding image and extracting multi-channel welding characteristic information.

And the second decoder is used for respectively carrying out up-sampling and feature fusion on the multi-channel welding feature information to obtain second feature information.

In the embodiment of the present application, the second detection model includes a second encoder, a second decoder, and a second classifier. And after the target welding image is input into the second detection model, a second encoder of the second detection model performs multi-channel convolution processing and pooling processing on the input target welding image by utilizing a preset convolution kernel and a preset pooling kernel to extract the characteristic information of the welding workpiece in the target welding image. And the second decoder performs characteristic splicing on the characteristic information of the welding workpiece extracted by the second encoder after deconvolution and the characteristic information of the multi-channel target welding image extracted by convolution processing in the second encoder to obtain second characteristic information of the size of the target welding image. And inputting the second characteristic information into a second classifier, classifying each welding characteristic of the welding workpiece in the target welding image by the second classifier, and outputting the welding information of the welding workpiece in the target welding image.

That is to say, the second detection model can extract the welding characteristic information of the welding workpiece in the welding image by extracting the welding information of various channels from the target welding image input into the second detection model through the matching among different devices, so that the second detection model can accurately extract the welding information of the welding workpiece in the welding image, and the accuracy of the defect detection of the subsequent welding workpiece is improved.

And step S607, determining whether the welding workpiece has welding defects according to the welding information of the welding workpiece.

In the embodiment of the application, after the defect detection device acquires the welding information of the welding workpiece, whether the welding workpiece has the defects of welding missing, neglected installation, few welding spots, welding spot deviation, welding spot wall climbing, flange angle defect, stud deformation, workpiece notch and the like can be detected according to the welding information.

Wherein determining whether the welding workpiece has the welding defect according to the welding information of the welding workpiece comprises:

when the welding information contains the center of the welding spot, determining whether the defects of welding missing, welding missing or few welding spots exist according to the center of the welding spot; alternatively, the first and second electrodes may be,

when the welding information comprises a welding spot outline and a second workpiece outline, determining whether the defect of wall climbing of the welding spot exists according to the second workpiece outline and the welding spot outline; alternatively, the first and second liquid crystal display panels may be,

In the embodiment of the application, when the welding information includes the center of the welding spot, the defect detection device can detect whether the defect of missing welding and missing mounting exists by detecting whether the center of the welding spot exists. That is, when the welding information acquired by the defect detection device does not include the center of the welding spot, it can be determined that the defect of missing welding or missing mounting exists. When the welding information contains the welding spot centers, the defect detection device can determine the number of the welding spots on the welding workpiece by detecting the number of the welding spot centers, and further determine whether the defects of few welding spots exist. At this time, the defect detection device needs to obtain the preset welding parameters. The welding parameters comprise the preset number of welding spots, and the defect detection device can determine that the defects of few welding spots exist according to whether the number of the welding spots on the welding workpiece is smaller than the preset number of the welding spots or not.

When the welding information includes the first workpiece profile and the welding point profile, the defect detecting device may detect whether the welding point profile exceeds the first workpiece profile, and if the welding point profile exceeds the first workpiece profile, determine that there is a defect of welding point deviation, as shown in fig. 9 a. As a possible implementation manner, the welding point information output in the second detection model is an image. That is, the first workpiece contour is an image of the first workpiece contour, the welding spot contour is an image of the welding spot contour, and the defect detection device can determine the positions of the pixel points of the first workpiece contour according to the image of the first workpiece contour. The defect detection device can also detect the positions of the pixel points of the welding spot outline according to the image of the welding spot outline, and further can detect whether the positions of the pixel points of the welding spot outline exceed the positions of the pixel points of the first workpiece outline, and if so, the defect of welding spot deviation can be determined.

When the welding information includes the welding point profile and the second workpiece profile, the defect detecting device may detect whether the second workpiece profile exceeds the welding point profile, and if the second workpiece profile exceeds the welding point profile, determine that there is a defect of welding point climbing, as shown in fig. 9 b. As a possible implementation manner, the welding point information output in the second detection model is an image. The second workpiece outline is an image of the second workpiece outline, the welding spot outline is an image of the welding spot outline, and the defect detection device can determine the positions of the pixel points of the second workpiece outline according to the image of the second workpiece outline. The defect detection device can also detect the positions of the pixel points of the welding spot outline according to the image of the welding spot outline, and further can detect whether the positions of the pixel points of the second workpiece outline exceed the positions of the pixel points of the welding spot outline, and if the positions of the pixel points of the second workpiece outline exceed the positions of the pixel points of the welding spot outline, the defect that the welding spot climbs the wall can be determined.

When the welding information includes the first workpiece profile, the defect detection device may determine the position of the first workpiece profile according to the first workpiece profile, and may further determine the first workpiece angle, obtain a preset welding parameter, where the welding parameter includes an angle threshold of the first workpiece, and when the calculated first workpiece angle exceeds the angle threshold of the first workpiece, determine that a defect of the first workpiece angle exists.

As one possible implementation, determining the first workpiece angle from the first workpiece profile includes: determining the position of the first workpiece contour in the first coordinate system according to the first workpiece contour; a first workpiece angle is determined based on a position of the first workpiece profile in the first coordinate system.

That is, the defect detecting apparatus may determine the position of the first workpiece contour in the first coordinate system when determining the angle of the first workpiece. For example, when the welding information is image information, and the first workpiece contour is the first workpiece contour, the coordinates of each pixel point of the edge contour of the first workpiece contour in the first coordinate system in the image may be determined, and at this time, the welding information may further include the image of the edge of the first workpiece contour, as shown in fig. 7 b. Furthermore, according to the position of the edge of the first workpiece contour in the first coordinate system, a straight line of the first workpiece passing through the edge of the contour in the first coordinate system at most is determined, and the angle of the first workpiece is determined according to the straight line. The defect detection device can determine the straight line of the first workpiece passing through the point of the edge of the first workpiece in the first coordinate system at the most according to the position of the first workpiece in the first coordinate system by a Hough straight line detection algorithm. At this time, the first coordinate system may be a cartesian coordinate system, a straight line in the cartesian coordinate system corresponds to one point in the hough space, and collinear points in the cartesian coordinate system intersect corresponding straight lines in the hough space. Therefore, the defect detection device is mapped into the Hough space according to the position of the first workpiece contour in the first coordinate system, so that the maximum common intersection point of straight lines corresponding to the edge of the first workpiece contour can be determined in the Hough space. I.e. the common intersection point corresponding to the first workpiece contour is determined, i.e. the straight line passing through the edge of the first workpiece contour in the cartesian coordinate system with the largest number of points can be determined, as shown in fig. 9c, so that the angle of the first workpiece can be determined from this straight line.

When the welding information includes the second workpiece contour, the defect detection device may determine the position of the second workpiece contour according to the second workpiece contour, and further may determine whether the shape corresponding to the second workpiece contour is a shape in a preset rule, and if not, may determine that there is a defect of deformation of the second workpiece.

As a possible implementation manner, the detecting whether the second workpiece contour conforms to the preset rule includes: determining the position of the second workpiece contour in a second coordinate system according to the second workpiece contour; determining the shape corresponding to the second workpiece outline according to the position of the second workpiece outline; and judging whether the shape corresponding to the profile of the second workpiece meets a preset rule or not.

The shape corresponding to the contour of the second workpiece may be a circle, and the circle is preset in the preset rule. The shape corresponding to the contour of the second workpiece may also be an ellipse, or a semicircle, etc., which is not limited in this application. In this example, the description will be given taking an example in which the shape corresponding to the contour of the second workpiece may be a circle. In this case, the defect detection device may determine the shape of the contour of the second workpiece based on the position of each point in the contour of the second workpiece. The defect detection apparatus may first determine the position of the second workpiece profile in the second coordinate system. For example, when the welding information is image information, the second workpiece contour is the second workpiece contour, and then the coordinates of each pixel point on the edge of the second workpiece contour in the second coordinate system in the image can be determined. Further, according to the position of the edge of the second workpiece contour in the second coordinate system, the circle with the most points on the edge of the second workpiece contour in the second coordinate system can be determined, and the circle can be determined as the circle corresponding to the second workpiece contour. The defect detection device can determine the circle with the most points on the edge of the second workpiece contour in the second coordinate system according to the position of the second workpiece contour in the second coordinate system through the Hough circle detection algorithm. At this time, the second coordinate system may be a cartesian coordinate system, a straight line in the cartesian coordinate system corresponds to one point in the hough circular space, and collinear points in the cartesian coordinate system intersect corresponding circles in the hough circular space. Therefore, the defect detection device maps the second workpiece contour edge to the Hough circular space according to the position of the second workpiece contour edge in the second coordinate system, so that the most common intersection points of the circles corresponding to the second workpiece contour edge can be determined in the Hough circular space. That is, the common circle intersection point corresponding to the edge of the second workpiece contour is determined, that is, the circle having the most points on the edge of the second workpiece contour in the cartesian coordinate system can be determined, as shown in fig. 9d, the circle corresponding to the second workpiece contour is determined, and thus, whether the circle corresponding to the second workpiece contour meets the preset rule or not is determined.

It should be noted that, if the second workpiece has a relatively serious profile deformation and the defect detection device cannot detect the circle corresponding to the second workpiece profile through the hough circle detection algorithm, it may be directly determined that the welded workpiece has the defect of the second workpiece deformation.

It should be noted that the preset rule is a preset size threshold of a shape corresponding to the outline of the second workpiece. Taking the shape corresponding to the outline of the second workpiece as a circle as an example, setting a size threshold range of a standard circle in a preset rule, judging whether the circle corresponding to the outline of the second workpiece is in the size threshold range of the standard circle, and if the circle corresponding to the outline of the second workpiece is not in the size threshold range of the standard circle, indicating that the welded workpiece has the defect of deformation of the second workpiece and is a defective product; if the circle corresponding to the outline of the second workpiece is within the size threshold range of the standard circle, the welded workpiece is determined to be good without the defect of deformation of the second workpiece.

When the welding information includes the welding spot center and the defect point, the defect detection device can calculate the defect area ratio according to the number of the welding spot centers and the defect point. For example, when the welding information is image information, the center of the welding spot is an image of the center of the welding spot, and the defect point is an image of the defect point, at this time, the defect detection apparatus may calculate the defect area ratio according to the number of pixels of the welding spot in the center of the welding spot and the number of pixels of the defect point. The defect detection device obtains preset welding parameters, the welding parameters comprise defect area threshold values, and the defect detection device can compare the calculated defect area proportion with the defect area threshold values to determine whether the defect area proportion exceeds the defect area threshold values. If the defect is exceeded, the defect of the gap is determined to exist.

In the embodiment of the application, the defect detection device can detect different welding defects in the welding workpiece through different contents contained in the welding information, so that the defect detection efficiency can be improved. Moreover, the process does not need manual participation, the accuracy of defect detection can be improved, and the detection cost is reduced.

And step S608, when the welding defect of the welding workpiece is determined, display information of the welding defect of the welding workpiece is formed, and the appearance or the characteristic value of the welding defect is displayed according to the display information.

In the embodiment of the application, in order to facilitate the worker to know the welding defect existing in the welding workpiece, when the defect detection device detects the welding defect existing in the welding workpiece, the display information of the welding defect of the welding workpiece can be formed according to the detected welding defect, and can be character display information or image display information, and the display information is displayed in the display device, so that the appearance or the characteristic value of the welding defect is displayed according to the display information.

The embodiment of the application provides a defect detection device 100 for a welding workpiece, and is shown by referring to fig. 1. The defect detection apparatus 100 includes:

the communicator 101 is configured to receive a welding image of a welding workpiece from an image capturing module and preset welding position information of the welding workpiece from an input device.

The processor 102, coupled to the communicator 101, is configured to input a welding image of a welding workpiece to a first detection model trained in advance, and obtain an outline of the welding workpiece in the welding image output by the first detection model. And determining whether the welding position of the welding workpiece is abnormal or not according to the outline of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece. And if the position deviation is abnormal, determining that the position deviation defect of the welding workpiece exists.

As a possible implementation manner, the processor 102 is specifically configured to determine a welding position of the welding workpiece in the welding image according to the profile of the welding workpiece output by the first detection model; calculating the spacing distance between the welding position of the welding workpiece in the welding image and the preset welding position of the welding workpiece according to the welding position of the welding workpiece in the welding image and the preset welding position information of the welding workpiece; acquiring a preset distance threshold value, and judging whether the spacing distance is greater than the preset distance threshold value; and if the spacing distance is greater than the preset distance threshold, determining that the welding position of the welding workpiece is abnormal.

As a possible implementation, the first detection model includes: the device comprises a first encoder, a first decoder and a first classifier.

And the first classifier is used for classifying each pixel in the welding image based on the first characteristic information to obtain the welding position information of the welding workpiece.

As a possible implementation manner, the processor 102 is further configured to obtain a target welding image if the welding position of the welding workpiece is not abnormal; inputting the target welding image into a pre-trained second detection model to obtain welding information of a welding workpiece in the target welding image output by the second detection model; and determining whether the welding workpiece has welding defects according to the welding information of the welding workpiece.

Wherein the target welding image is a welding image at a welding position of the welding workpiece; the second detection model is trained in advance and used for performing semantic cutting on the input target welding image, extracting multi-channel welding characteristic information and outputting welding information of a welding workpiece in the target welding image.

As a possible implementation, the second detection model includes: a second encoder, a second decoder, and a second classifier.

As one possible implementation, the welding information includes: at least one of a weld spot profile, a first workpiece profile, a second workpiece profile, a weld spot center, a defect point.

At this time, the processor 102 is specifically configured to determine whether there is a missing welding defect, a missing mounting defect, or a few welding spots defect according to the welding spot center when the welding information includes the welding spot center. Or when the welding information comprises the first workpiece outline and the welding spot outline, determining whether the defect of welding spot deviation exists according to the first workpiece outline and the welding spot outline. Or when the welding information comprises the welding spot outline and the second workpiece outline, determining whether the defect of wall climbing of the welding spot exists according to the second workpiece outline and the welding spot outline. Or when the welding information comprises the first workpiece outline, determining a first workpiece angle according to the first workpiece outline, and determining whether the defect of the first workpiece angle exists according to the first workpiece angle. Or when the welding information comprises the second workpiece outline, detecting whether the second workpiece outline accords with a preset rule, and determining whether the second workpiece has a deformed defect. Or when the welding information comprises the welding spot center and the defect point, determining whether the defect of the gap exists according to the welding spot center and the defect point.

As a possible implementation, determining the first workpiece angle from the first workpiece profile comprises: determining the position of a first workpiece contour in a first coordinate system according to the first workpiece contour; the first workpiece angle is determined based on a position of the first workpiece profile in a first coordinate system.

As a possible implementation manner, the detecting whether the second workpiece contour conforms to the preset rule includes:

determining the position of the second workpiece contour in a second coordinate system according to the second workpiece contour; determining the shape corresponding to the second workpiece outline according to the position of the second workpiece outline; and judging whether the shape corresponding to the profile of the second workpiece meets a preset rule or not.

As a possible implementation manner, the processor 102 is specifically configured to cut a welding position area of the welding workpiece in the welding image according to the contour of the welding workpiece, so as to obtain a target welding image.

As a possible implementation manner, the processor 102 is further configured to form display information of the welding defect of the welding workpiece when the welding defect of the welding workpiece is determined, so as to display the appearance or characteristic value of the welding defect according to the display information.

As a possible implementation manner, the defect detection apparatus 100 further includes a memory 103, as shown in fig. 10.

The memory 103 is used for storing various types of data, such as various databases, program codes, and the like, in the defect detecting apparatus 100 for a welded workpiece. In the embodiment of the present application, the Memory 103 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random-Access Memory (SRAM), electrically Erasable Programmable Read-Only Memory (EEPROM), erasable Programmable Read-Only Memory (EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk, and the like.

Corresponding to the above embodiments, the present application also provides a computer-readable storage medium. The computer storage medium may store a program, and the program may include some or all of the steps in the embodiments of the defect detection method provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), or the like.

Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The same and similar parts among the various embodiments in this specification may be referred to each other. In particular, as for the apparatus embodiment and the terminal embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the description in the method embodiment for relevant points.

Claims

1. A method of detecting defects in a welded workpiece, comprising:

2. The method according to claim 1, wherein the determining whether the welding position of the welding workpiece is abnormal according to the profile of the welding workpiece output by the first detection model and preset welding position information of the welding workpiece comprises:

3. The method of claim 1, wherein the first detection model comprises: a first encoder, a first decoder and a first classifier;

4. The method of claim 1, wherein the second detection model comprises: a second encoder, a second decoder and a second classifier;

5. The method of claim 1 or 4, wherein the welding information comprises: at least one of a weld spot profile, a first workpiece profile, a second workpiece profile, a weld spot center, a defect point.

6. The method of claim 5, wherein the determining whether the welding workpiece has the welding defect based on the welding information of the welding workpiece comprises:

when the welding information contains the center of the welding spot, determining whether the defects of missing welding, missing welding or few welding spots exist according to the center of the welding spot; alternatively, the first and second electrodes may be,

7. The method of claim 6, wherein said determining the first workpiece angle from the first workpiece profile comprises:

the first workpiece angle is determined based on the position of the first workpiece profile in the first coordinate system.

8. The method of claim 6, wherein detecting whether the second workpiece profile meets a predetermined rule comprises:

determining a shape corresponding to the second workpiece outline according to the position of the second workpiece outline;

and judging whether the shape corresponding to the profile of the second workpiece meets a preset rule or not.

9. The method of claim 1, wherein the obtaining a target weld image comprises:

10. The method of claim 1 or 6, further comprising:

11. A defect detecting apparatus for a welded workpiece, comprising:

and determining whether the welding workpiece has welding defects or not according to the welding information of the welding workpiece.

12. The defect detection apparatus of the welded workpiece according to claim 11, wherein the first detection model includes: a first encoder, a first decoder and a first classifier;

13. The defect detection apparatus of the welded workpiece according to claim 11, wherein the second detection model includes: a second encoder, a second decoder and a second classifier;

the second classifier is used for classifying the second characteristic information to obtain welding information of the welding workpiece;

the welding information includes: at least one of a weld spot profile, a first workpiece profile, a second workpiece profile, a weld spot center, a defect point.

14. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the method of any one of claims 1-10.