CN116051532A - Deep learning-based industrial part defect detection method and system and electronic equipment - Google Patents

Abstract

The invention provides an industrial part defect detection method, system and electronic equipment based on deep learning; the method comprises the following steps: acquiring a data set; constructing a deep neural network for detecting surface defects of a target industrial part; training the deep neural network by using the data set, and acquiring a trained target neural network so as to detect surface defects of the target industrial part based on the target neural network; the invention provides an industrial part defect detection method based on deep learning, which realizes the detection of the surface defects of industrial parts by taking a convolutional neural network and a transducer as theoretical bases, combines the advantages of the convolutional neural network and the transducer, improves the segmentation precision, adopts the design of parallel branches, and ensures the convergence speed during the training of the deep neural network and the time requirement during the reasoning test.

Description

Defect detection method, system and electronic equipment for industrial parts based on deep learning

technical field

The present invention relates to the field of physics, in particular to detection technology for surface defects of industrial parts, in particular to a method, system and electronic equipment for detecting defects of industrial parts based on deep learning.

Background technique

In the process of industrial production, the limitations of existing technology, working conditions and other factors will seriously reduce the quality of finished products; among them, surface defects are a typical manifestation of product quality reduction, therefore, in order to ensure the pass rate and reliable quality , Product surface defect detection must be carried out.

"Defect" can generally be understood as the absence, defect or area compared with normal samples; surface defect detection refers to the detection of scratches, defects, foreign matter blocking, color pollution, holes and other defects on the surface of the sample, so as to obtain the surface defects of the tested sample A series of related information such as category, outline, position, size, etc.; manual defect detection used to be the mainstream method, and workers were trained to identify complex surface defects, but this method is inefficient; detection results are easily affected by human subjective factors, and cannot To meet the requirements of real-time detection; therefore, to realize the automation of defect detection is a work of great significance and great challenge.

Traditional machine vision methods must manually extract features to adapt to specific fields, and then make decisions according to artificially formulated rules or learnable classifiers (such as SVM, decision tree, etc.), this method is very dependent on human experience, and the development cycle Long, it is difficult to keep up with the iteration speed of the product.

Contents of the invention

The object of the present invention is to provide a deep learning-based industrial part defect detection method, system and electronic equipment, which are used to solve the above-mentioned problems existing in the existing product surface defect detection technology.

In order to achieve the above object and other related objects, the present invention provides a method for detecting defects in industrial parts based on deep learning, comprising the following steps: obtaining a data set; the data set includes target surface defect images of industrial parts; A deep neural network for surface defects of industrial parts; the deep neural network includes: a fusion module, a Transformer branch, a CNN branch and a decoder; the Transformer branch, the CNN branch and the decoder are all connected to the fusion module; Wherein, the fusion module is used to fuse the first result output by the Transformer branch and the second result output by the CNN branch, and the decoder is used to decode the third result output by the fusion module, so The output of the decoder is used as the output of the deep neural network; the deep neural network is trained using the data set, and the trained target neural network is obtained, so as to perform surface defects on the target industrial part based on the target neural network detection.

In an embodiment of the present invention, the acquiring the data set includes the following steps: acquiring original surface defect images of industrial parts; performing preprocessing on the original surface defect images to acquire target surface defect images.

In an embodiment of the present invention, the fusion module is further configured to enhance the fourth result generated after the fusion of the first result and the second result to generate a fifth result, and to The fifth result, the first result and the second result are concatenated.

In one embodiment of the present invention, the number of the fusion modules is three; the decoder includes: a first attention module, a second attention module, a first convolutional layer, a second convolutional layer and a segmentation head , the process of the decoder decoding the third result includes: inputting the third result output by one of the fusion modules and the third result output by another fusion module to the first attention module; The sixth result output by the first attention module is input to the first convolution layer to obtain the seventh result; the seventh result and the third result output by another fusion module are input to the The second attention module; the eighth result output by the second attention module is input to the second convolutional layer to obtain the ninth result; the ninth result is input to the segmentation head; the segmentation The output of the header serves as the output of the decoder.

In an embodiment of the present invention, the segmentation head includes: a third convolutional layer and a bilinear difference layer; wherein, in the process of inputting the ninth result to the segmentation head, the The ninth result is input to the third convolutional layer, and then the output of the third convolutional layer is restored to the original resolution through the bilinear difference layer.

In an embodiment of the present invention, the training of the deep neural network using the data set includes the following steps: selecting training images from the data set; inputting the training images into the deep neural network to train The deep neural network; in the process of training the deep neural network, the deep neural network is trained by minimizing a loss function.

In an embodiment of the present invention, in the process of training the deep neural network, the deep neural network is trained through iterative training; the training of the deep neural network using the data set also includes the following steps: using The deep neural network is evaluated by intersection score and/or Dice score.

The present invention provides an industrial part defect detection system based on deep learning, including: an acquisition module for acquiring a data set; the data set includes target surface defect images of industrial parts; The deep neural network of parts surface defects; the deep neural network includes: a fusion module, a Transformer branch, a CNN branch and a decoder; the Transformer branch, the CNN branch and the decoder are all connected to the fusion module; wherein , the fusion module is used to fuse the first result output by the Transformer branch and the second result output by the CNN branch, and the decoder is used to decode the third result output by the fusion module, the The output of the decoder is used as the output of the deep neural network; the training module is used to use the data set to train the deep neural network, obtain the trained target neural network, and use the target neural network to analyze the target industry based on the target neural network. Parts are inspected for surface defects.

The present invention provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned method for detecting defects of industrial parts based on deep learning is realized.

The present invention provides an electronic device, including: a processor and a memory; the memory is used to store computer programs; the processor is used to execute the computer programs stored in the memory, so that the electronic device performs the above-mentioned deep learning-based Defect detection method for industrial parts.

As mentioned above, the deep learning-based industrial part defect detection method, system and electronic equipment of the present invention have the following beneficial effects:

(1) Compared with the prior art, the purpose of the present invention is to solve the problem that it is difficult to capture long-distance dependent information when only using a convolutional neural network, and propose a method for detecting defects in industrial parts based on deep learning, The introduction of Transformer and attention module improves the segmentation accuracy.

(2) The present invention provides a method for detecting defects in industrial parts based on deep learning, which realizes the detection of surface defects in industrial parts based on convolutional neural network and Transformer, and combines the advantages of convolutional neural network and Transformer, The segmentation accuracy is improved, and the design of parallel branches is adopted to ensure the convergence speed of the deep neural network training and the time requirements of the reasoning test.

Description of drawings

FIG. 1 is a flowchart of an embodiment of the method for detecting defects in industrial parts based on deep learning of the present invention.

FIG. 2 is a block diagram of a deep neural network in an embodiment of the present invention.

FIG. 3 is a schematic structural diagram of an embodiment of the fusion module of the present invention.

FIG. 4 is a schematic structural diagram of an AttentionGate module in an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of an embodiment of the SCSE module of the present invention.

FIG. 6 is a schematic diagram showing the structure of ViT in an embodiment of the present invention.

FIG. 7 is a schematic structural diagram of a ResNet34 of the present invention in an embodiment.

FIG. 8 is a flow chart of another embodiment of the method for detecting defects in industrial parts based on deep learning of the present invention.

FIG. 9 is a schematic structural diagram of an embodiment of the deep learning-based industrial part defect detection system of the present invention.

Detailed ways

The implementation of the present invention is described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

It should be noted that the diagrams provided in the following embodiments are only schematically illustrating the basic ideas of the present invention, and only the components related to the present invention are shown in the diagrams rather than the number, shape and size of the components in actual implementation Drawing, the type, quantity and proportion of each component can be changed arbitrarily during its actual implementation, and its component layout type may also be more complicated.

In the past ten years, with the development of massive data analysis and learning technology, deep neural network has been applied in many visual recognition tasks. Compared with classical machine vision methods, deep learning can learn advanced features directly from data, so it has more High ability to represent complex structures, which enables an automatic learning process to replace manual engineering of features.

The convolution operation has translation invariance, which makes it naturally suitable for image processing, but its locality makes it focus on a limited area, making it difficult to capture long-range dependencies. As Transformer shines in natural language processing in recent years, its advantage of capturing global dependencies well through the global attention mechanism has also been applied in the field of computer vision; the present invention combines the advantages of the two Complementary and combined with the attention module, a complete network structure is constructed for defect detection in the industrial field.

The method for detecting defects in industrial parts based on deep learning of the present invention will be explained below in conjunction with specific embodiments and drawings.

As shown in Figure 1, the industrial part defect detection method based on deep learning of the present invention is applied to tile surface defect detection; specifically, the industrial part defect detection method based on deep learning includes the following steps:

Step S1, using a camera to obtain images of surface defects of ceramic tiles.

Step S2, preprocessing the surface defect image to obtain a data set.

Specifically, all surface defect images are rescaled to a size of 256×256×3, and data enhancement operations such as random rotation, translation, and scaling are performed on the images to obtain a dataset.

At the same time, the above operations are synchronously applied to the segmentation label of the image to obtain the data set.

Then, the dataset is randomly divided into 385 images for training and 31 images for inference testing.

Step S3, during training, each time an image I _img with dimensions H×W×C is obtained from the data set as input.

Wherein, H and W represent the height and width of the image I _img , and C represents the number of channels of the image I _img .

Specifically, a pair of images is randomly selected from the dataset each time during training: one is the surface defect image, and the other is the corresponding segmentation label.

Step S4: Construct a deep neural network, and use the image I _img as input to train the deep neural network.

Referring to Figure 2, the image I _img is input into the network during training.

In one embodiment, the deep neural network includes: Transformer branch, CNN branch, fusion module and decoder; wherein, Transformer branch adopts ViT (Vision Transformer, as shown in Figure 6 for specific structure) as the backbone network, and CNN branch adopts residual structure (ResNet) as the backbone network.

It should be noted that the present invention combines the advantages of Transformer and CNN, that is, Transformer can capture global dependencies well through the global attention mechanism, while CNN is better at capturing local details; the Transformer branch and the CNN branch have The feature map of the same resolution is input to the corresponding fusion module, and then the output of the fusion module is input to the decoder, and the output of the deep neural network is obtained by sequentially upsampling.

As shown in FIG. 7 , in one embodiment, the CNN branch uses ResNet34 as the backbone network.

It should be noted that the Transformer branch uses ViT as the backbone network, divides the image into several 16×16×3 parts, and readjusts each part into a one-dimensional vector, and each one-dimensional vector is linearly transformed to obtain the length It is a one-dimensional vector of 384, and then input into ViT, and then reassemble several vectors output by ViT into a three-dimensional feature map, and improve the resolution of the feature map and reduce the number of channels through two upsampling, and obtain outputs of different resolutions .

For the CNN branch, ResNet is used as the backbone network; specifically, after the image is input into ResNet, it first goes through a layer of convolution with a convolution kernel size of 7 and a step size of 2, followed by a Relu activation function, and uses maximum pooling for the next step. Sampling, and then go through three Residual blocks (residual blocks), and obtain outputs of different resolutions corresponding to each Residual block.

The feature information of the image is captured through the Transformer branch and the CNN branch, and then the feature information is fused through the fusion module, and the decoder decodes the output from the fusion module.

The entire process is shown in Figure 2. This embodiment takes a two-dimensional image as an example. The input image size is 256×256×3. For the Transformer branch, reshape the Transformer output vector to 16×16×384, and then pass two double Linear interpolation and deconvolution result in output feature maps of size 32×32×128 and 64×64×64, respectively.

For the CNN branch, the first convolution block has 64 convolution kernels, the size of the convolution kernel is 7, the step size is 2, followed by the Relu activation function, and the maximum pooling is adopted; after the first convolution block, A feature map with a size of 64×64×64 is obtained; followed by three consecutive residual blocks, the size of the convolution kernel in the residual block is 3, and the step size of the first residual block is 1, and the other step sizes are is 2, the number of convolution kernels is 64, 128, and 256 respectively, and the three residual blocks obtain output feature maps with sizes of 64×64×64, 32×32×128 and 16×16×256 respectively.

After obtaining the feature maps output by the CNN branch and the Transformer branch, a pair of feature maps with the same resolution are input into the fusion module. Convolutional layer of 1, align the number of channels, add them up, and then go through SCSEBlock (SpatialSqueeze and Channel Excitation Block), and splice the output result with the feature map output by the CNN branch and the feature map output by the Transformer branch, and finally Then input the splicing result into the Residual block to get the final output of the fusion module.

Among them, the SCSEBlock structure is shown in Figure 5.

SCSEBlock includes two branches, namely the SSE branch and the CSE branch; among them, for the SSE branch, after the feature map is input, it first passes through a convolution layer with a convolution kernel size of 1 and a convolution kernel number of 1 to obtain a two-dimensional The feature map is multiplied by the input feature map according to the spatial dimension to get the output of the SSE branch; for the CSE branch, after the feature map is input, it passes through an average pooling layer to obtain a one-dimensional vector, and then passes through two convolution kernels with a size of 1 convolutional layer, the dimension is first reduced and then restored, and finally a one-dimensional vector with the same dimension as the number of channels is obtained, which is multiplied by the channel dimension and the input feature map to obtain the output of the CSE branch; then the output of the SSE branch Add the output of the CSE branch to get the final output of SCSEBlock.

It should be noted that, in order to further improve the capture of global dependencies by the Transformer branch and the capture of local details by the CNN branch, the present invention further enhances the features captured by the Transformer branch and the CNN branch by using an additional attention module in the fusion module , where the SCSE Block is used to enhance the fusion result of the output of the Transformer branch and the output of the CNN branch, and finally the enhanced feature maps are spliced and finally passed through the Residual block to obtain the output of the fusion module and input to the final decoder.

且

其中Conv为卷积层，Up为上采样，AG为注意力模块。In the decoding process of the decoder of the present invention, record the feature map output by the ith fusion module as f _i , then the feature map output by the i+1 layer of the decoder is

and

Among them, Conv is the convolutional layer, Up is upsampling, and AG is the attention module.

In one embodiment, the present invention uses the AttentionGate module (AG module for short) in the decoding process to further improve the final segmentation result. A single pixel in the deep feature map has a larger receptive field, which can better focus on the more global The module uses the information extracted from the deep feature map as an attention mechanism to apply to the shallow feature map, so that the shallow feature map can pay attention to more global information.

第i+1个融合模块输出的特征图为f_i+1，则AG模块的输出为

且

其中，Conv为卷积层，其卷积核大小为1，步长为1，Up为双线性插值上采样。Note that the feature map output by the i-th decoder is

The feature map output by the i+1th fusion module is f _i+1 , then the output of the AG module is

and

Among them, Conv is a convolutional layer with a convolution kernel size of 1 and a step size of 1, and Up is bilinear interpolation upsampling.

Specifically, the decoder consists of two AttentionGate modules, two convolutional layers, and a segmentation head. The structure of the AttentionGate module is shown in Figure 4. The shallow feature map first passes through the convolutional layer with a convolution kernel size of 1. Align the number of channels with the deep feature map, and then align the resolution with the deep feature map by downsampling, then add the result to the deep feature map, pass through the Relu layer, and pass the convolution with a convolution kernel size of 1 The layer restores the number of channels to the original number, and then passes through the sigmoid layer, and then upsamples the result back to the resolution of the shallow feature map. The result is the attention feature map, which combines the attention feature map with the original shallow feature map. The output of the AttentionGate module can be obtained by multiplying the graphs.

与f₁输入AttentionGate模块，再将输出经由一个卷积层得到

之后再将

与f₂输入另一个AttentionGate模块，再将输出经由一个卷积层得到

最后将

经过分割头得到最终的分割结果。The decoder will fuse the output of the module

Input the AttentionGate module with f ₁ , and then pass the output through a convolutional layer to obtain

later on

Input another AttentionGate module with f ₂ , and then pass the output through a convolutional layer to get

Finally will

The final segmentation result is obtained through the segmentation head.

The segmentation head includes a convolutional layer and bilinear interpolation, and the output of the convolutional layer is restored to the original resolution through bilinear interpolation to obtain the final segmentation result.

In one embodiment, the deep neural network is trained by minimizing a loss function.

It should be noted that the minimized loss function is an energy function derived from traditional registration methods:

Among them, L _total improves the final segmentation accuracy through deep supervision, α, β, γ are variable hyperparameters, G is the segmentation label, head is the prediction head, which converts the input into the segmentation result, and t _i is the output of the Transformer branch. The result after i times upsampling;

L=L _IoU +L _bce ;

in,

为深度神经网络的输出。Among them, y is the segmentation label,

is the output of the deep neural network.

In this embodiment, it is set to 200 epochs, and the Adam optimizer is used to drive network optimization. α, β, and γ in L _total are respectively 0.5, 0.3, and 0.2. After the number of iterations is completed, the final model is obtained.

In one embodiment, the deep neural network is trained iteratively.

In one embodiment, for the trained deep neural network, IOU (Intersection Over Union) score and Dice score are used as indicators of defect segmentation performance.

It should be noted that the reasoning test is performed using the trained deep neural network; specifically, an image is sequentially selected in the test set as input during the test, and the corresponding segmentation label of the image is also input.

Dice评分，表达式为：

其中y为分割标签，

为网络输出。In this embodiment, there are 31 segmentation labels for the two-dimensional image. The test network outputs the segmentation result and the segmentation evaluation index. The expression of the evaluation index is: IOU score, and the expression is:

Dice score, the expression is:

where y is the segmentation label,

output for the network.

It should be noted that the IoU coefficient and the Dice coefficient (the Dice coefficient is a set similarity measure function, which is usually used to calculate the similarity between two samples) are set similarity measures used to calculate the similarity between two samples. The value range is [0,1]. The better the segmentation effect, the closer the IoU value and Dice value are to 1.

The invention discloses a method for detecting defects of industrial parts based on deep learning. Firstly, a camera is used to obtain a picture of the surface of an industrial part, and data enhancement operations such as rotation, translation, and scaling are performed on it to obtain a preprocessed image; and then the preprocessed image is obtained. The final image is input to the Transformer network and the convolutional neural network for feature extraction and feature fusion; then the feature map is transformed back to the original size through feature upsampling technology and input into the prediction layer to obtain the final semantic segmentation result; the training process is supervised During the training process, use the IOU cost loss function and the Dice cost loss function to iteratively train and optimize parameters until the model parameters converge, and save the model parameter file; during the test process, the input image is an industrial part defect image with a size of 224×224×3, Use the trained deep neural network to test on it, and finally get the segmentation result with a size of 224×224×1; the present invention can realize more accurate defects on the industrial parts data set by combining the advantages of Transformer and CNN Segmentation to improve the accuracy of the final segmentation results.

In prior art solutions, only convolutional neural networks are usually used. The present invention introduces a Transformer on the basis of the convolutional neural network, and adds an attention module to the fusion module and the decoder, which can increase the weight of useful information, suppress the influence of noise, and realize the improvement of segmentation accuracy.

The invention relates to a method for detecting defects of industrial parts based on deep learning, and is an implementation method based on a convolutional neural network and a Transformer theory. The invention combines the advantages of the convolutional neural network and the Transformer to improve the segmentation accuracy, and simultaneously adopts the design of parallel branches to ensure the convergence speed during training and the time requirement during reasoning and testing.

As shown in Figure 8, in one embodiment, the deep learning-based industrial part defect detection method of the present invention includes the following steps:

Step H1, obtaining a data set.

It should be noted that the dataset includes target surface defect images of industrial parts.

In one embodiment, the acquiring data set includes the following steps:

Step (11), obtaining the original surface defect image of the industrial part.

Step (12), performing preprocessing on the original surface defect image to obtain a target surface defect image.

In one embodiment, the preprocessing in step (12) includes image enhancement processing; wherein, the image enhancement processing includes at least but not limited to any of the following processing methods: rotation, translation, and scaling.

Step H2, constructing a deep neural network for detecting surface defects of target industrial parts.

In this embodiment, the deep neural network includes: a fusion module, a Transformer branch, a CNN branch, and a decoder; the Transformer branch, the CNN branch, and the decoder are all connected to the fusion module; wherein, the The fusion module is used to fuse the first result output by the Transformer branch and the second result output by the CNN branch, and the decoder is used to decode the third result output by the fusion module, and the decoder The output of is used as the output of the deep neural network.

In one embodiment, the fusion module is further configured to enhance the fourth result generated after the fusion of the first result and the second result to generate a fifth result, and to generate the fifth result, The first result and the second result are concatenated.

In one embodiment, the number of the fusion modules is three; the decoder includes: a first attention module, a second attention module, a first convolutional layer, a second convolutional layer and a segmentation head, the The process for the decoder to decode the third result includes:

和另一所述融合模块输出的第三结果f₁输入至所述第一注意力模块。Step (21), the third result of a described fusion module output

and the third result _f1 output by the other fusion module are input to the first attention module.

Step (22), input the sixth result output by the first attention module to the first convolutional layer, and obtain the seventh result

和又一所述融合模块输出的第三结果f₂输入至所述第二注意力模块。Step (23), the seventh result

and a third result _f2 output by the fusion module is input to the second attention module.

Step (24), input the eighth result output by the second attention module to the second convolutional layer, and obtain the ninth result

输入至所述分割头；所述分割头的输出作为所述解码器的输出。Step (25), the ninth result

input to the segmentation header; the output of the segmentation header serves as the output of the decoder.

In one embodiment, the segmentation head includes: a third convolutional layer and a bilinear difference layer; wherein, in the process of inputting the ninth result to the segmentation head, the ninth The result is input to the third convolutional layer, and then the output of the third convolutional layer is restored to the original resolution through the bilinear difference layer.

Step H3, using the data set to train the deep neural network, and obtain a trained target neural network, so as to detect surface defects of the target industrial parts based on the target neural network.

In one embodiment, the training of the deep neural network using the data set includes the following steps:

Step (31), selecting training images from the data set.

Step (32), inputting the training image into the deep neural network to train the deep neural network.

In one embodiment, during training the deep neural network, the deep neural network is trained by minimizing a loss function.

In one embodiment, in the process of training the deep neural network, the deep neural network is trained by iterative training.

In one embodiment, the training of the deep neural network using the data set further includes the following step: evaluating the deep neural network by using an intersection ratio score and/or a Dice score.

In one embodiment, the surface defect detection of the target industrial part based on the target neural network includes: inputting the target surface defect image of the target industrial part into the target neural network to realize the target neural network The detection of the surface defects of the target industrial parts; the output of the target neural network is the result of the detection of the surface defects of the target industrial parts.

It should be noted that the working principle of the deep learning-based industrial part defect detection method provided in this embodiment can refer to the introduction of the deep learning-based industrial part defect detection method in the above specific embodiments, and will not be described in detail here.

It should be noted that the scope of protection of the deep learning-based industrial part defect detection method described in the present invention is not limited to the execution order of the steps listed in this embodiment. The solutions realized by step replacement are all included in the protection scope of the present invention.

A computer program is stored on the storage medium of the present invention, and when the computer program is executed by a processor, the above-mentioned method for detecting defects of industrial parts based on deep learning is realized. The storage medium includes: various media capable of storing program codes such as read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk, U disk, memory card or optical disk.

Any combination of one or more storage media may be employed. The storage medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connection with one or more conductors, portable computer disk, hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), fiber optics, portable compact disc read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a data signal carrying computer readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including - but not limited to - electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. .

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including - but not limited to - wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, etc., including conventional A procedural programming language—such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

The present invention is described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart and/or block diagrams, and combinations of blocks in the flowchart and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.

These computer program instructions can also be stored in a computer-readable medium, and these instructions cause a computer, other programmable data processing apparatus, or other equipment to operate in a specific way, so that the instructions stored in the computer-readable medium produce information including An article of manufacture that implements the functions/actions specified in one or more blocks in a flowchart and/or block diagram.

It is also possible to load computer program instructions onto a computer, other programmable data processing apparatus, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process, thereby Cause instructions executed on a computer or other programmable device to provide a process for implementing the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The electronic device of the present invention includes a processor and a memory.

The memory is used to store computer programs; preferably, the memory includes various media capable of storing program codes such as ROM, RAM, magnetic disk, U disk, memory card or optical disk.

The processor is connected to the memory, and is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned deep learning-based industrial part defect detection method.

Preferably, the processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (Digital Signal Processor, referred to as DSP), Application Specific Integrated Circuit (ASIC for short), Field Programmable Gate Array (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

As shown in Figure 9, in one embodiment, the industrial part defect detection system based on deep learning of the present invention includes:

The acquisition module 91 is configured to acquire a data set; the data set includes target surface defect images of industrial parts.

Construction module 92, is used for constructing the deep neural network that is used to detect the surface defect of target industrial part; Described deep neural network comprises: Fusion module, Transformer branch, CNN branch and decoder; Described Transformer branch, described CNN branch and all The decoders are all connected to the fusion module; wherein, the fusion module is used to fuse the first result output by the Transformer branch and the second result output by the CNN branch, and the decoder is used to fuse the The third result output by the fusion module is decoded, and the output of the decoder is used as the output of the deep neural network.

The training module 93 is configured to use the data set to train the deep neural network, obtain a trained target neural network, and perform surface defect detection on the target industrial parts based on the target neural network.

It should be noted that the structures and principles of the acquisition module 91, the construction module 92 and the training module 93 correspond to the steps (step H1 to step H3) in the above-mentioned deep learning-based industrial part defect detection method , so it will not be repeated here.

It should be noted that it should be understood that the division of the various modules of the above system is only a division of logical functions, and may be fully or partially integrated into a physical entity or physically separated during actual implementation. And these modules can all be implemented in the form of calling software through processing elements; they can also be implemented in the form of hardware; some modules can also be implemented in the form of calling software through processing elements, and some modules can be implemented in the form of hardware. For example, the x module can be a separate processing element, or it can be integrated into a certain chip of the above-mentioned system. In addition, it can also be stored in the memory of the above-mentioned system in the form of program code, and a certain processing element of the above-mentioned system can Call and execute the function of the above x module. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, and can also be implemented independently. The processing element mentioned here may be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or an instruction in the form of software.

For example, the above modules may be one or more integrated circuits configured to implement the above method, for example: one or more specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), or, one or more digital signal processors (Digital Signal Processor, DSP for short), or, one or more Field Programmable Gate Arrays (Field Programmable Gate Array, FPGA for short), etc. For another example, when one of the above modules is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, referred to as CPU) or other processors that can call program codes. For another example, these modules can be integrated together and implemented in the form of a System-On-a-Chip (SOC for short).

It should be noted that the deep learning-based industrial part defect detection system of the present invention can implement the deep learning-based industrial part defect detection method of the present invention, but the realization device of the deep learning-based industrial part defect detection method of the present invention includes but It is not limited to the structure of the deep learning-based industrial part defect detection system listed in this embodiment, and any structural deformation and replacement of the prior art based on the principles of the present invention are included in the scope of protection of the present invention.

In summary, compared with the prior art, the deep learning-based industrial part defect detection method, system and electronic equipment of the present invention, the purpose of the present invention is to solve the problem of difficult to capture remote For the problem of distance dependence information, a deep learning-based industrial part defect detection method is proposed, and Transformer and attention module are introduced to improve the segmentation accuracy; the present invention provides a deep learning-based industrial part defect detection method, through the Convolutional neural network and Transformer are the theoretical basis to realize the detection of surface defects of industrial parts. Combining the advantages of convolutional neural network and Transformer, the segmentation accuracy is improved. The convergence speed and the time requirement of the reasoning test; therefore, the present invention effectively overcomes various shortcomings in the prior art and has high industrial application value.

The above-mentioned embodiments only illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed in the present invention should still be covered by the claims of the present invention.

Claims (10)

1. The industrial part defect detection method based on deep learning is characterized by comprising the following steps of:

acquiring a data set; the dataset comprising a target surface defect image of an industrial part;

constructing a deep neural network for detecting surface defects of a target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network;

and training the deep neural network by using the data set, and acquiring a trained target neural network so as to detect surface defects of the target industrial part based on the target neural network.

2. The deep learning based industrial part defect detection method of claim 1, wherein the acquiring the data set comprises the steps of:

acquiring an original surface defect image of an industrial part;

and preprocessing the original surface defect image to obtain a target surface defect image.

3. The deep learning based industrial part defect detection method of claim 1, wherein the fusion module is further configured to enhance a fourth result generated after the first result and the second result are fused, generate a fifth result, and splice the fifth result, the first result, and the second result.

4. The deep learning-based industrial part defect detection method of claim 1, wherein the number of fusion modules is three; the decoder includes: the process of decoding the third result by the decoder comprises the following steps:

inputting a third result output by one fusion module and a third result output by the other fusion module into the first attention module;

inputting a sixth result output by the first attention module into the first convolution layer to obtain a seventh result;

inputting the seventh result and a third result output by the fusion module to the second attention module;

inputting an eighth result output by the second attention module to the second convolution layer to obtain a ninth result;

inputting the ninth result to the segmentation head; the output of the split head is taken as the output of the decoder.

5. The deep learning based industrial part defect detection method of claim 4, wherein the segmentation head comprises: a third convolution layer and a bilinear difference layer; in the process of inputting the ninth result to the dividing head, the ninth result is input to the third convolution layer, and then the output of the third convolution layer is restored to the original resolution through the bilinear difference layer.

6. The deep learning based industrial part defect detection method of claim 1, wherein the training the deep neural network with the data set comprises the steps of:

selecting a training image from the dataset;

inputting the training image to the deep neural network to train the deep neural network;

in the training process of the deep neural network, the deep neural network is trained by minimizing a loss function.

7. The deep learning based industrial part defect detection method of claim 1, wherein the deep neural network is trained by iterative training in training the deep neural network;

the training of the deep neural network using the data set further comprises the steps of: the deep neural network is evaluated using cross-ratio scores and/or Dice scores.

8. An industrial part defect detection system based on deep learning, comprising:

the acquisition module is used for acquiring the data set; the dataset comprising a target surface defect image of an industrial part;

the construction module is used for constructing a deep neural network for detecting the surface defects of the target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network;

and the training module is used for training the deep neural network by utilizing the data set, and acquiring a trained target neural network so as to detect the surface defects of the target industrial part based on the target neural network.

9. A storage medium having stored thereon a computer program, which when executed by a processor, implements the deep learning-based industrial part defect detection method of any one of claims 1 to 7.

10. An electronic device, comprising: a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory, so that the electronic device performs the deep learning-based industrial part defect detection method according to any one of claims 1 to 7.

Images