CN116051532A

CN116051532A - Deep learning-based industrial part defect detection method and system and electronic equipment

Info

Publication number: CN116051532A
Application number: CN202310109015.0A
Authority: CN
Inventors: 蒋学芹; 陈齐航; 周树波; 潘峰
Original assignee: Donghua University
Current assignee: Donghua University
Priority date: 2023-02-13
Filing date: 2023-02-13
Publication date: 2023-05-02

Abstract

The invention provides an industrial part defect detection method, system and electronic equipment based on deep learning; the method comprises the following steps: acquiring a data set; constructing a deep neural network for detecting surface defects of a target industrial part; training the deep neural network by using the data set, and acquiring a trained target neural network so as to detect surface defects of the target industrial part based on the target neural network; the invention provides an industrial part defect detection method based on deep learning, which realizes the detection of the surface defects of industrial parts by taking a convolutional neural network and a transducer as theoretical bases, combines the advantages of the convolutional neural network and the transducer, improves the segmentation precision, adopts the design of parallel branches, and ensures the convergence speed during the training of the deep neural network and the time requirement during the reasoning test.

Description

Deep learning-based industrial part defect detection method and system and electronic equipment

Technical Field

The invention relates to the field of physics, in particular to an industrial part surface defect detection technology, and especially relates to an industrial part defect detection method, system and electronic equipment based on deep learning.

Background

In the industrial production process, the limitations of factors such as the prior art, working conditions and the like can seriously reduce the quality of finished products; among them, surface defects are a typical manifestation of degradation of product quality, and therefore, in order to ensure yield and reliable quality, product surface defect detection is necessary.

"defect" can generally be understood as a deletion, defect or area compared to a normal sample; the surface defect detection is to detect defects such as scratches, defects, foreign matter shielding, color pollution, holes and the like on the surface of a sample, so as to obtain a series of related information such as the type, the outline, the position, the size and the like of the surface defects of the sample to be detected; human defect detection has been the mainstream method, and workers are trained to identify complex surface defects, but this method is inefficient; the detection result is easily affected by artificial subjective factors, and the requirement of real-time detection cannot be met; therefore, it is a significant and challenging task to implement defect detection automation.

Traditional machine vision methods must manually extract features to fit a particular domain, then make decisions according to manually formulated rules or learnable classifiers (e.g., SVM, decision tree, etc.), which are very dependent on human experience, and have long development cycles, which are difficult to keep up with the iterative speed of the product.

Disclosure of Invention

The invention aims to provide an industrial part defect detection method and system based on deep learning and electronic equipment, which are used for solving the problems existing in the existing product surface defect detection technology.

To achieve the above and other related objects, the present invention provides a method for detecting defects of industrial parts based on deep learning, comprising the steps of: acquiring a data set; the dataset comprising a target surface defect image of an industrial part; constructing a deep neural network for detecting surface defects of a target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network; and training the deep neural network by using the data set, and acquiring a trained target neural network so as to detect surface defects of the target industrial part based on the target neural network.

In one embodiment of the present invention, the acquiring the data set includes the steps of: acquiring an original surface defect image of an industrial part; and preprocessing the original surface defect image to obtain a target surface defect image.

In an embodiment of the present invention, the fusion module is further configured to enhance a fourth result generated after the first result and the second result are fused, generate a fifth result, and splice the fifth result, the first result, and the second result.

In an embodiment of the invention, the number of the fusion modules is three; the decoder includes: the process of decoding the third result by the decoder comprises the following steps: inputting a third result output by one fusion module and a third result output by the other fusion module into the first attention module; inputting a sixth result output by the first attention module into the first convolution layer to obtain a seventh result; inputting the seventh result and a third result output by the fusion module to the second attention module; inputting an eighth result output by the second attention module to the second convolution layer to obtain a ninth result; inputting the ninth result to the segmentation head; the output of the split head is taken as the output of the decoder.

In an embodiment of the invention, the dividing head includes: a third convolution layer and a bilinear difference layer; in the process of inputting the ninth result to the dividing head, the ninth result is input to the third convolution layer, and then the output of the third convolution layer is restored to the original resolution through the bilinear difference layer.

In one embodiment of the present invention, the training the deep neural network using the data set comprises the steps of: selecting a training image from the dataset; inputting the training image to the deep neural network to train the deep neural network; in the training process of the deep neural network, the deep neural network is trained by minimizing a loss function.

In an embodiment of the present invention, in the training process of the deep neural network, training the deep neural network through iterative training; the training of the deep neural network using the data set further comprises the steps of: the deep neural network is evaluated using cross-ratio scores and/or Dice scores.

The invention provides an industrial part defect detection system based on deep learning, which comprises: the acquisition module is used for acquiring the data set; the dataset comprising a target surface defect image of an industrial part; the construction module is used for constructing a deep neural network for detecting the surface defects of the target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network; and the training module is used for training the deep neural network by utilizing the data set, and acquiring a trained target neural network so as to detect the surface defects of the target industrial part based on the target neural network.

The present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described deep learning-based industrial part defect detection method.

The present invention provides an electronic device including: a processor and a memory; the memory is used for storing a computer program; the processor is used for executing the computer program stored in the memory so as to enable the electronic equipment to execute the deep learning-based industrial part defect detection method.

As described above, the deep learning-based industrial part defect detection method, system and electronic equipment have the following beneficial effects:

(1) Compared with the prior art, the invention aims to solve the problem that long-distance dependence information is difficult to capture under the condition that only a convolutional neural network is used, and provides an industrial part defect detection method based on deep learning, which introduces a transducer and a attention module and improves segmentation accuracy.

(2) The invention provides an industrial part defect detection method based on deep learning, which realizes the detection of the surface defects of industrial parts by taking a convolutional neural network and a transducer as theoretical bases, combines the advantages of the convolutional neural network and the transducer, improves the segmentation precision, adopts the design of parallel branches, and ensures the convergence speed during the training of the deep neural network and the time requirement during the reasoning test.

Drawings

FIG. 1 is a flow chart illustrating a deep learning-based industrial part defect detection method according to an embodiment of the invention.

Fig. 2 is a schematic diagram of a deep neural network according to an embodiment of the invention.

Fig. 3 is a schematic structural diagram of a fusion module according to an embodiment of the invention.

FIG. 4 is a schematic diagram of an AttenionGate module according to an embodiment of the present invention.

Figure 5 is a schematic diagram of an SCSE module according to an embodiment of the invention.

Fig. 6 is a schematic diagram of a ViT embodiment of the present invention.

Fig. 7 is a schematic diagram of a res net34 according to an embodiment of the present invention.

FIG. 8 is a flow chart of a deep learning-based industrial part defect detection method according to another embodiment of the invention.

FIG. 9 is a schematic diagram of a deep learning-based industrial part defect detection system according to an embodiment of the invention.

Detailed Description

The following specific examples are presented to illustrate the present invention, and those skilled in the art will readily appreciate the additional advantages and capabilities of the present invention as disclosed herein. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.

It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the illustrations, not according to the number, shape and size of the components in actual implementation, and the form, number and proportion of each component in actual implementation may be arbitrarily changed, and the layout of the components may be more complex.

In recent decades, with the development of mass data analysis and learning technology, deep neural networks have been applied in many visual recognition tasks, and deep learning can directly learn advanced features from data, compared with classical machine vision methods, thus having higher complex structural representation capability, which makes an automatic learning process replace manual engineering of features.

Convolution operation has translational invariance, which makes it naturally suitable for image processing, but its locality makes it limited in the area of interest, and it is difficult to capture remote dependencies. As Transformer has grown in natural language processing in recent years, the advantage of capturing global dependencies well through global attention mechanisms has also been applied in the field of computer vision; the invention complements the advantages of the two, combines the advantages with the attention module and constructs a complete network structure for detecting defects in the industrial field.

The deep learning-based industrial part defect detection method of the present invention will be explained with reference to specific embodiments and drawings.

As shown in fig. 1, the deep learning-based industrial part defect detection method of the present invention is applied to tile surface defect detection; specifically, the industrial part defect detection method based on deep learning comprises the following steps:

and S1, acquiring a surface defect image of the ceramic tile by using a camera.

And S2, preprocessing the surface defect image to obtain a data set.

Specifically, all the surface defect images are resized to 256×256×3, and data enhancement operations such as random rotation, translation, and scaling are performed on the images to acquire a data set.

Simultaneously, the operations are synchronously acted on the segmentation labels of the images, and a data set is acquired.

The dataset was then randomly split into 385 images for training and 31 images for reasoning testing.

Step S3, during training, each time an image I with dimensions of H multiplied by W multiplied by C is acquired from the data set _img As input.

Wherein H and W represent image I _img C represents the height and width ofImage I _img Is used for the number of channels.

Specifically, a pair of images is randomly selected from the dataset each time during training: one is a surface defect image and the other is a corresponding split label.

Step S4: build deep neural network and apply image I _img As input, a deep neural network is trained.

Referring to FIG. 2, image I is trained _img Input into the network.

In one embodiment, a deep neural network includes: a transducer branch, a CNN branch, a fusion module, and a decoder; the transducer branch adopts ViT (Vision Transformer, the specific structure is shown in fig. 6) as a backbone network, and the CNN branch adopts a residual error structure (res net) as the backbone network.

It should be noted that, in order to combine the advantages of the transducer and the CNN, that is, the transducer can well capture the global dependency through the global attention mechanism, while the CNN is better at capturing the local details; and inputting feature graphs with the same resolution in the transducer branch and the CNN branch into corresponding fusion modules, and then inputting the output of the fusion modules into a decoder, and sequentially up-sampling to obtain the output of the deep neural network.

In one embodiment, as shown in fig. 7, the CNN branch employs a res net34 as the backbone network.

It should be noted that, the transducer branch adopts ViT as a backbone network, divides an image into a plurality of 16×16×3 portions, readjust each portion into one-dimensional vectors, linearly transform each one-dimensional vector to obtain a one-dimensional vector with length of 384, input ViT, and splice a plurality of vectors output by ViT into a three-dimensional feature map again, and increase the resolution of the feature map and reduce the channel number by up-sampling twice, and obtain outputs with different resolutions.

For CNN branches, resNet is adopted as a backbone network; specifically, after the image is input into ResNet, one layer of convolution with the convolution kernel size of 7 and the step length of 2 is firstly carried out, then a Relu activation function is connected, downsampling is carried out by using maximum pooling, then three Residual blocks (Residual blocks) are carried out, and outputs with different resolutions corresponding to the Residual blocks are obtained.

Feature information of the image is captured through a transducer branch and a CNN branch, then the feature information is fused through a fusion module, and a decoder decodes output from the fusion module.

As shown in fig. 2, in the present embodiment, the input image size is 256×256×3, for the transducer branch, the transducer output vector reshape is 16×16×384, and then the output feature maps with the sizes of 32×32×128 and 64×64 are obtained through bilinear interpolation and deconvolution twice.

For CNN branches, the first convolution block has 64 convolution kernels, the size of the convolution kernel is 7, the step size is 2, the function is activated by Relu, and the maximum pooling is adopted; after the first convolution block has been passed, obtaining a characteristic diagram with the size of 64 multiplied by 64; then three continuous residual blocks are arranged, the sizes of convolution kernels in the residual blocks are all 3, the steps of the residual blocks except the first residual block step size is 1, the rest step sizes are all 2, the number of the convolution kernels is 64, 128 and 256, and the three residual blocks respectively obtain output characteristic diagrams with the sizes of 64 multiplied by 64, 32 multiplied by 128 and 16 multiplied by 256.

After obtaining the feature graphs output by the CNN branch and the Transformer branch, inputting the feature graphs with the same resolution ratio into a fusion module, wherein the structure of the fusion module is shown in fig. 3, the feature graphs output by the CNN branch and the Transformer branch pass through convolution layers with the convolution kernel size of 1 respectively, the channel numbers are aligned, then are added, the result is spliced with the feature graphs output by the CNN branch and the feature graphs output by the Transformer branch through SCSEBlock (Spatial Squeeze and Channel Excitation Block), and finally, the spliced result is input into a Residual block to obtain the final output of the fusion module.

The SCSEBlock structure is shown in fig. 5.

SCSLOCK comprises two branches, namely an SSE branch and a CSE branch; for SSE branches, firstly, a convolution layer with the convolution kernel size of 1 and the convolution kernel number of 1 is passed through after the input of the feature map, so as to obtain a two-dimensional feature map, and multiplying the two-dimensional feature map with the input feature map according to the space dimension to obtain the output of the SSE branches; for CSE branches, a one-dimensional vector is obtained through an average pooling layer after feature graphs are input, then the dimension is reduced and then restored through two convolution layers with the convolution kernel size of 1, and finally a one-dimensional vector with the same dimension as the channel number is obtained, and the one-dimensional vector is multiplied with the input feature graphs according to the channel dimension to obtain the output of the CSE branches; and adding the output of the SSE branch and the output of the CSE branch to obtain the final output of the SCSEBlock.

It should be noted that, in order to further promote the capturing of global dependencies by the transform branch and the capturing of local details by the CNN branch, the present invention further enhances the features captured by the transform branch and the CNN branch by using an additional attention module in the fusion module, wherein SCSE Block is used to enhance the result obtained by fusing the output of the transform branch with the output of the CNN branch, and finally, the enhanced feature map is spliced and finally, the output of the fusion module is obtained through the Residual Block and is input to the final decoder.

In the decoding process of the decoder, the characteristic diagram output by the ith fusion module is recorded as f _i The feature map of the i+1th layer output of the decoder is

And->

Where Conv is the convolutional layer, up is the upsampling, AG is the attention module.

In an embodiment, an attention (AG module) module is adopted in the decoding process of the present invention to further enhance the final segmentation result, the single pixel of the deep feature map has a larger receptive field, and can better pay attention to the more global information.

Record the feature map of the i-th decoder output as

The feature map output by the (i+1) th fusion module is f _i+1 The output of AG module is +.>

And->

Conv is a convolution layer, the convolution kernel size is 1, the step length is 1, and Up is bilinear interpolation up-sampling.

Specifically, the decoder includes two atlantiongate modules, two convolution layers and a partition head, where the structure of the atlantiongate module is shown in fig. 4, the shallow layer feature map is first convolved by the convolution layer with the convolution kernel size of 1 to align the channel number with the deep layer feature map, then the resolution is also aligned with the deep layer feature map by downsampling, then the obtained result is added with the deep layer feature map, then the result passes through the Relu layer, and the number of channels is restored to the original number by the convolution layer with the convolution kernel size of 1, then the result passes through the sigmoid layer, and then the result is upsampled back to the resolution of the shallow layer feature map, and the obtained result is the attention feature map, and the attention feature map is multiplied with the original shallow layer feature map to obtain the output of the atlantiongate module.

The decoder outputs the fusion module

And f ₁ Input AttenationGate module, output is obtained via a convolution layer>

Then will->

And f ₂ Inputting another AttenationGate module, and obtaining output via a convolution layer

Finally will->

And obtaining a final segmentation result through the segmentation head.

The dividing head comprises a convolution layer and bilinear interpolation, and the output of the convolution layer is restored to the original resolution through bilinear interpolation to obtain a final dividing result.

In one embodiment, the deep neural network is trained by minimizing a loss function.

It should be noted that the minimization loss function is an energy function derived from the conventional registration method:

wherein L is _total The final segmentation precision is improved through deep supervision, alpha, beta and gamma are variable super parameters, G is a segmentation label, head is a pre-measurement head, input is converted into a segmentation result, and t _i Results after i upsampling for the output of the transform branch;

L＝L _IoU +L _bce ；

wherein,,

wherein y is a split tag,

is the output of the deep neural network.

In this embodiment, 200 epochs are set, and Adam optimizer is used to drive network optimization, L _total The alpha, beta and gamma are respectively 0.5, 0.3 and 0.2, and the final model is obtained after the iteration times are completed.

In one embodiment, the deep neural network performs iterative training.

In one embodiment, for a trained deep neural network, IOU (Intersection Over Union, cross-correlation) score and Dice score are used as good and bad indicators of defect segmentation performance.

It should be noted that, the training deep neural network is utilized to carry out reasoning test; specifically, during testing, one image is sequentially selected in a test set to serve as input, and meanwhile, the segmentation labels corresponding to the images are input.

In the present embodiment, there are 31 split labels of the two-dimensional image. The test network outputs a segmentation result and a segmentation evaluation index, and the evaluation index expression is as follows: IOU score, expressed as:

the Dice score, expressed as:

wherein y is a split tag, ">

Is output by the network.

Note that, the IoU coefficient and the Dice coefficient (Dice coefficient is a set similarity measure function, which is generally used to calculate the similarity between two samples) are set similarity measure indexes, which are used to calculate the similarity between two samples, the range of values is [0,1], and the better the segmentation effect, the closer the IoU value and the Dice value are to 1.

The invention discloses an industrial part defect detection method based on deep learning, which comprises the steps of firstly, acquiring an industrial part surface picture by using a camera, and carrying out data enhancement operations such as rotation, translation, scaling and the like on the industrial part surface picture to obtain a preprocessed image; then respectively inputting the preprocessed images into a transducer network and a convolutional neural network to perform feature extraction and feature fusion; then, the feature map is converted back to the original size through a feature up-sampling technology, and a prediction layer is input to obtain a final semantic segmentation result; the training process is a supervision training process, iteration training and parameter optimization are carried out by utilizing the IOU cost loss function and the Dice cost loss function until the model parameters are converged, and a model parameter file is saved; in the test process, the input image is an industrial part defect image with the size of 224 multiplied by 3, and a trained deep neural network is used for testing the industrial part defect image, so that a segmentation result with the size of 224 multiplied by 1 is finally obtained; according to the invention, by combining the advantages of the transducer and the CNN, more accurate defect segmentation can be realized on the industrial part data set, and the precision of a final segmentation result is improved.

Only convolutional neural networks are typically employed in prior art schemes. On the basis of adopting the convolutional neural network, the invention also introduces a transducer, and adds the attention module in the fusion module and the decoder, thereby being capable of improving the weight of useful information, inhibiting the influence of noise and realizing the improvement of segmentation precision.

The invention relates to an industrial part defect detection method based on deep learning, which is an implementation method based on a convolutional neural network and a transducer. The invention combines the advantages of the convolutional neural network and the transducer, improves the segmentation precision, adopts the design of parallel branches, and ensures the convergence speed during training and the time requirement during reasoning test.

As shown in fig. 8, in an embodiment, the method for detecting defects of industrial parts based on deep learning according to the present invention comprises the following steps:

and step H1, acquiring a data set.

It should be noted that the data set includes a target surface defect image of the industrial part.

In one embodiment, the acquiring the data set comprises the steps of:

and (11) acquiring an original surface defect image of the industrial part.

And (12) preprocessing the original surface defect image to obtain a target surface defect image.

In one embodiment, the preprocessing in step (12) includes image enhancement processing; the image enhancement processing at least comprises, but is not limited to, any one of the following processing modes: rotation, translation, scaling.

And H2, constructing a deep neural network for detecting the surface defects of the target industrial part.

In this embodiment, the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network.

In an embodiment, the fusion module is further configured to enhance a fourth result generated after the first result and the second result are fused, generate a fifth result, and splice the fifth result, the first result, and the second result.

In one embodiment, the number of fusion modules is three; the decoder includes: the process of decoding the third result by the decoder comprises the following steps:

step (21), outputting a third result by the fusion module

And a third result f output by another said fusion module ₁ Input to the first attention module.

Step (22), inputting a sixth result output by the first attention module to the first convolution layer to obtain a seventh result

Step (23) of combining the seventh result

And a third result f output by the fusion module ₂ Input to the second attention module.

Step (24), inputting the eighth result output by the second attention module to the second convolution layer to obtain a ninth result

Step (25) of obtaining the ninth result

Input to the dividing head; the output of the split head is taken as the output of the decoder.

In one embodiment, the dividing head comprises: a third convolution layer and a bilinear difference layer; in the process of inputting the ninth result to the dividing head, the ninth result is input to the third convolution layer, and then the output of the third convolution layer is restored to the original resolution through the bilinear difference layer.

And step H3, training the deep neural network by using the data set, and obtaining a trained target neural network so as to detect surface defects of the target industrial part based on the target neural network.

In one embodiment, the training the deep neural network using the data set includes the steps of:

and (31) selecting a training image from the data set.

And (32) inputting the training image into the deep neural network to train the deep neural network.

In one embodiment, the deep neural network is trained by minimizing a loss function during training of the deep neural network.

In one embodiment, the deep neural network is trained by iterative training during training of the deep neural network.

In an embodiment, the training the deep neural network using the data set further comprises the steps of: the deep neural network is evaluated using cross-ratio scores and/or Dice scores.

In one embodiment, the surface defect detection of the target industrial part based on the target neural network comprises: inputting a target surface defect image of the target industrial part to the target neural network to realize detection of the target industrial part surface defect by the target neural network; the output of the target neural network is the result of detecting the surface defects of the target industrial parts.

It should be noted that, the working principle of the method for detecting defects of industrial parts based on deep learning provided in this embodiment may refer to the description of the method for detecting defects of industrial parts based on deep learning in the above specific embodiment, and will not be described in detail herein.

It should be noted that, the protection scope of the method for detecting defects of industrial parts based on deep learning according to the present invention is not limited to the execution sequence of the steps listed in the embodiment, and all the solutions implemented by increasing or decreasing the steps and replacing the steps according to the prior art according to the principles of the present invention are included in the protection scope of the present invention.

The storage medium of the present invention stores a computer program which, when executed by a processor, implements the above-described deep learning-based industrial part defect detection method. The storage medium includes: read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disks, U-discs, memory cards, or optical discs, and the like, which can store program codes.

Any combination of one or more storage media may be employed. The storage medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The present invention is described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks (article of manufacture).

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The electronic device of the invention comprises a processor and a memory.

The memory is used for storing a computer program; preferably, the memory includes: various media capable of storing program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

The processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the electronic equipment to execute the industrial part defect detection method based on deep learning.

Preferably, the processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field programmable gate arrays (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

As shown in fig. 9, in one embodiment, the deep learning-based industrial part defect detection system of the present invention comprises:

an acquisition module 91 for acquiring a data set; the dataset includes a target surface defect image of an industrial part.

A construction module 92 for constructing a deep neural network for detecting surface defects of the target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network.

The training module 93 is configured to train the deep neural network by using the data set, and obtain a trained target neural network, so as to perform surface defect detection on the target industrial part based on the target neural network.

It should be noted that the structures and principles of the obtaining module 91, the constructing module 92, and the training module 93 are in one-to-one correspondence with the steps (step H1 to step H3) in the above-mentioned deep learning-based industrial part defect detection method, and thus will not be described herein.

It should be noted that, it should be understood that the division of the modules of the above system is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the x module may be a processing element that is set up separately, may be implemented in a chip of the system, or may be stored in a memory of the system in the form of program code, and the function of the x module may be called and executed by a processing element of the system. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.

For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more digital signal processors (Digital Signal Processor, abbreviated as DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a System-On-a-Chip (SOC).

It should be noted that, the deep learning-based industrial part defect detection system of the present invention may implement the deep learning-based industrial part defect detection method of the present invention, but the implementation device of the deep learning-based industrial part defect detection method of the present invention includes, but is not limited to, the structure of the deep learning-based industrial part defect detection system listed in this embodiment, and all structural modifications and substitutions made in the prior art according to the principles of the present invention are included in the protection scope of the present invention.

In summary, compared with the prior art, the invention aims to solve the problem that remote dependence information is difficult to capture under the condition that only a convolutional neural network is used, and provides the industrial part defect detection method based on deep learning, which introduces a transducer and a attention module and improves segmentation accuracy; the invention provides an industrial part defect detection method based on deep learning, which realizes the detection of the surface defects of industrial parts by taking a convolutional neural network and a transducer as theoretical bases, combines the advantages of the convolutional neural network and the transducer, improves the segmentation precision, adopts the design of parallel branches, and ensures the convergence speed during the training of the deep neural network and the time requirement during the reasoning test; therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.

The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims

1. The industrial part defect detection method based on deep learning is characterized by comprising the following steps of:

acquiring a data set; the dataset comprising a target surface defect image of an industrial part;

constructing a deep neural network for detecting surface defects of a target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network;

and training the deep neural network by using the data set, and acquiring a trained target neural network so as to detect surface defects of the target industrial part based on the target neural network.

2. The deep learning based industrial part defect detection method of claim 1, wherein the acquiring the data set comprises the steps of:

acquiring an original surface defect image of an industrial part;

and preprocessing the original surface defect image to obtain a target surface defect image.

3. The deep learning based industrial part defect detection method of claim 1, wherein the fusion module is further configured to enhance a fourth result generated after the first result and the second result are fused, generate a fifth result, and splice the fifth result, the first result, and the second result.

4. The deep learning-based industrial part defect detection method of claim 1, wherein the number of fusion modules is three; the decoder includes: the process of decoding the third result by the decoder comprises the following steps:

inputting a third result output by one fusion module and a third result output by the other fusion module into the first attention module;

inputting a sixth result output by the first attention module into the first convolution layer to obtain a seventh result;

inputting the seventh result and a third result output by the fusion module to the second attention module;

inputting an eighth result output by the second attention module to the second convolution layer to obtain a ninth result;

inputting the ninth result to the segmentation head; the output of the split head is taken as the output of the decoder.

5. The deep learning based industrial part defect detection method of claim 4, wherein the segmentation head comprises: a third convolution layer and a bilinear difference layer; in the process of inputting the ninth result to the dividing head, the ninth result is input to the third convolution layer, and then the output of the third convolution layer is restored to the original resolution through the bilinear difference layer.

6. The deep learning based industrial part defect detection method of claim 1, wherein the training the deep neural network with the data set comprises the steps of:

selecting a training image from the dataset;

inputting the training image to the deep neural network to train the deep neural network;

in the training process of the deep neural network, the deep neural network is trained by minimizing a loss function.

7. The deep learning based industrial part defect detection method of claim 1, wherein the deep neural network is trained by iterative training in training the deep neural network;

the training of the deep neural network using the data set further comprises the steps of: the deep neural network is evaluated using cross-ratio scores and/or Dice scores.

8. An industrial part defect detection system based on deep learning, comprising:

the acquisition module is used for acquiring the data set; the dataset comprising a target surface defect image of an industrial part;

the construction module is used for constructing a deep neural network for detecting the surface defects of the target industrial part; the deep neural network includes: a fusion module, a transducer branch, a CNN branch and a decoder; the transducer branch, the CNN branch and the decoder are all connected with the fusion module; the fusion module is used for fusing a first result output by the converter branch and a second result output by the CNN branch, the decoder is used for decoding a third result output by the fusion module, and the output of the decoder is used as the output of the deep neural network;

and the training module is used for training the deep neural network by utilizing the data set, and acquiring a trained target neural network so as to detect the surface defects of the target industrial part based on the target neural network.

9. A storage medium having stored thereon a computer program, which when executed by a processor, implements the deep learning-based industrial part defect detection method of any one of claims 1 to 7.

10. An electronic device, comprising: a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory, so that the electronic device performs the deep learning-based industrial part defect detection method according to any one of claims 1 to 7.