CN112967271B - Casting surface defect identification method based on improved DeepLabv3+ network model - Google Patents
Casting surface defect identification method based on improved DeepLabv3+ network model Download PDFInfo
- Publication number
- CN112967271B CN112967271B CN202110317123.8A CN202110317123A CN112967271B CN 112967271 B CN112967271 B CN 112967271B CN 202110317123 A CN202110317123 A CN 202110317123A CN 112967271 B CN112967271 B CN 112967271B
- Authority
- CN
- China
- Prior art keywords
- layer
- module
- network
- convolution
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Abstract
The invention provides a casting surface defect identification method based on an improved deep Labv3+ network model, which comprises the following steps: s1, collecting a casting image data set to obtain a training set and a test set; step S2, constructing a network model, and performing data training and correction on the network model through a training set and a testing set to generate a defect detection network; step S3, designing a loss function of the defect detection network; and step S4, the defect detection network identifies and outputs the defect detection result of the casting and displays the detection duration. According to the invention, the surface defects of the castings are identified by adopting a deep learning method, so that the accuracy and speed of defect identification are improved, and a new idea is provided for quality detection of industrial castings.
Description
Technical Field
The invention relates to the technical field of defect detection of industrial castings, in particular to a casting surface defect identification method based on an improved deep Labv3+ network model.
Background
In industrial casting production, various types of castings are widely used. However, during the production and manufacturing process, the cast still has inevitable defects such as cracks, shrinkage cavities and the like. These defects can seriously affect the quality of the finished casting and thus the quality of the entire manufactured product. The high-efficiency defect detection technology is one of important links for guaranteeing the quality of finished casting products. Therefore, in the period of high-speed development and production of the modern casting industry, the accurate and efficient casting surface defect detection technology is an important development direction for industrial product detection.
At present, most domestic casting production still adopts a manual sampling detection mode to detect the defects of the castings, and detection is finished mainly by means of visual observation of detection personnel and subjective judgment. The method depends on the priori knowledge of detection personnel, has strong subjectivity, lacks accuracy and normalization, and cannot guarantee efficiency. Other industrial casting surface defect detection methods include eddy current coil detection, ultrasonic detection and the like. However, these conventional methods have different disadvantages, the detection work consumes a lot of manpower and material resources, and the final detection result also needs to be manually processed to make a judgment. Meanwhile, when ultrasonic detection and eddy current coil detection are carried out, the surface defects of the casting are likely to be contacted, physical and chemical changes are likely to occur, the defect area is further enlarged, and the detection precision is not favorably improved.
Disclosure of Invention
Aiming at the problems, in order to improve the current casting defect detection dilemma, the invention provides a casting surface defect identification method based on an improved DeepLabv3+ network model, which is based on a detection method of a neural network, and casting defect detection is carried out by improving the DeepLabv3+ network model, so that the automatic real-time online and intelligent detection of casting surface defects can be realized.
The acquired workpiece surface defect image is made into a defect data set and sent into an improved neural network, and then a trained network is finally obtained through self-defined network training and learning based on semantic segmentation and is used for workpiece surface defect image detection and defect area marking, and then the detection time is output by combining defect detection software, and meanwhile, different model detection comparison can be carried out by replacing a network model.
In order to achieve the aim, the invention provides a casting surface defect identification method based on an improved deep Labv3+ network model, which comprises the following steps:
step S1, collecting a casting image data set to obtain a training set and a testing set: the method comprises the following steps that an industrial camera collects casting images, Labelme marks casting defects to generate an image data set, and the image data set is divided into a training set and a testing set;
step S2, constructing a network model, and performing data training and correction on the network model through a training set and a testing set to generate a defect detection network;
step S3, designing a loss function of the defect detection network: the loss function is a function comprising cross entropy loss of a prediction result and cross entropy loss of a real value;
and step S4, the defect detection network identifies and outputs the defect detection result of the casting and displays the detection duration.
Preferably, step S2 specifically includes the following steps:
step S21, constructing an improved DeepLabV3+ network model: improving a coding and decoding module in an existing DeepLabV3+ network model, wherein the coding and decoding module adopts a depth separable convolutional network and reduces the number of channels;
step S22, carrying out data training and correction to generate a defect detection network: the improved DeepLabv3+ network model is used as a network model for identifying the surface defects of the castings to extract the surface defect characteristics of the castings of the originally collected images of the castings, the improved DeepLabv3+ network model is trained by utilizing training data in a training set, and then the test set is input into the improved DeepLabv3+ network model to be corrected until the prediction accuracy of the generated defect detection network meets a prediction accuracy threshold.
Preferably, the outputting of the casting detection result in the step S4 is implemented by visual software facing defect detection of industrial castings.
Preferably, in step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, the encoding module includes a deep convolutional network layer, an encoder, and a cavity space pyramid pooling module, the deep convolutional network layer is a trunk model, low-level semantic features including defect shape information obtained by convolving an input picture with the deep convolutional network layer are directly transmitted into the decoding module, high-level semantic features including defect position categories obtained by convolving an input picture with the encoder and the deep convolutional network layer are transmitted into the cavity space pyramid pooling module, the cavity space pyramid pooling module includes a cavity convolutional module, four convolutional layers and a pooling layer, the four convolutional layers are connected in series with the pooling layer, the cavity convolutional modules respectively adopt cavity convolutional blocks with different expansion rates, and controlling an encoder to extract the resolution of the features, respectively performing cavity convolution to obtain four feature maps of the four convolutional layers, and performing pooling by the cavity convolution module to obtain one feature map of the one pooling layer.
Preferably, for the low-level semantic feature information obtained by the backbone model, the encoding module operates the low-level semantic feature through a convolution to reduce the number of channels, and obtains the final semantic information of the low-level semantic feature after the operation, and transmits the final semantic information of the low-level semantic feature to a decoder of the decoding module; and for the high-level semantic features obtained by the cavity space pyramid pooling module, the decoding module obtains the final semantic information of the high-level semantic features through one-time up-sampling, and the final semantic information is transmitted to a decoder as an output result.
Preferably, the encoding module further includes a stack layer, a channel adjustment convolutional layer, an intermediate feature layer stack layer, a post-processing convolutional layer, and a post-processing module, where the stack layer is formed by stacking feature maps obtained by the four convolutional layers and the one pooling layer, the stack layer is convolved to adjust a channel to form the channel adjustment convolutional layer, the channel adjustment convolutional layer is subjected to length and width adjustment to form the intermediate feature layer, the intermediate feature layer stack layer is formed after the intermediate feature layer is stacked, the intermediate feature layer stack layer is convolved to form the post-processing convolutional layer, and the post-processing convolutional layer is subjected to post-processing by the post-processing module.
Preferably, the post-processing module includes a pixel point classification module and a pixel point classification probability calculation module, the pixel point classification module classifies each pixel point of the image of the post-processing convolution layer, and the pixel point classification probability calculation module solves the probability of each pixel point classification by softmax.
Preferably, the decoding module performs upsampling on the high-level semantic features output by the encoder by using bilinear upsampling with a sampling factor of 4, the input of the decoding module consists of two parts, one part of the high-level semantic features is directly output from the deep convolutional network layer of the trunk model and is connected with corresponding low-level semantic features with the same spatial resolution output from the trunk model, the other part of the low-level semantic features is led into the hollow space pyramid pooling module through the deep convolutional network layer of the trunk model to find a low-level semantic feature map with the same semantic feature resolution output by the encoder, the number of channels is reduced by 1 × 1 convolution to make the channel proportion of the low-level semantic features and the low-level semantic feature map which are respectively obtained by the input of the two parts of the decoding module equal to the channel proportion of the semantic features output by the encoder, and the low-level semantic feature maps are connected together, after the connection processing, thinning is performed through a 3 × 3 thinning convolution, and then a final prediction result is obtained through bilinear upsampling with a sampling factor of 4.
Preferably, the upsampling used by the output of the encoding module and the upsampling used by the output of the decoding module are bilinear interpolation methods, and the formula is defined as follows:
srcx＝desx*srcω/desω
srcy＝desy*srch/desh
in the formula, srcxRepresenting the x-coordinate, src, of a pixel in the original imageyRepresenting the y-coordinate, des, of a pixel in the original imagexRepresenting the x-coordinate, des, of a pixel in a target imageyRepresenting y coordinates, src, of pixels in the target imageωRepresenting the original image width, srchRepresenting the height of the original image, desωRepresenting the width of the target image, deshRepresenting the target image height.
Preferably, the loss function in step S3 uses a cross-entropy loss function, and the formula is as follows:
J＝-[y·log(p)+(1-y)·log(1-p)]
wherein y represents label of the sample, the positive class is 1, and the negative class is 0; p represents the probability that a sample is predicted to be positive.
The invention can obtain the following beneficial effects: the improved DeepLabv3+ network-based model can be used for defect identification of industrial castings, the identification network can be directly applied to the field of defect detection of industrial castings, digital defect data of defect identification are generated after identification, result storage and query can correspond to casting information one by one, certain data feedback is provided, the feedback can be used for optimization of casting process, guidance is provided for subsequent processes, and particularly, the method mainly takes the grinding defect position of the industrial castings as a target to detect and identify the defects of the castings.
The network model based on the improved DeepLabV3+ has the advantages of high identification speed and high identification accuracy, and the defect detection and identification method based on the deep learning algorithm and the deep neural convolution network is used for effectively solving the defects of the traditional process.
The invention designs a visual software for detecting the defects of the castings, the software can select the defect pictures to be detected by setting a network model for detecting the defects, and the detection result and the detection time are output after the detection is finished, the visual result is clear, the software is simple to operate and easy to operate, and the current situation that the defects of the current casting defect detection are artificially involved is greatly improved.
Drawings
FIG. 1 is a flow chart of a casting surface defect identification method based on a modified DeepLabv3+ network model in the invention;
FIG. 2 is a schematic diagram of the overall network structure of a casting surface defect identification method based on the improved DeepLabv3+ network model according to a preferred embodiment of the present invention;
FIG. 3 is a schematic diagram of the encoding modules of the overall network architecture shown in FIG. 2;
FIG. 4 is a schematic diagram of a decoding module of the overall network architecture shown in FIG. 3;
FIG. 5 is a schematic diagram of a network structure of a backbone model according to a preferred embodiment of the method for identifying surface defects of castings based on the modified deep Labv3+ network model;
FIG. 6 is a schematic diagram of the identification result of a casting surface defect identification method based on the improved deep Labv3+ network model according to a preferred embodiment of the present invention;
FIG. 7 is a diagram of the Qt design software interface of the casting surface defect identification method based on the improved deep Labv3+ network model according to the preferred embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
Aiming at the existing problems, the invention provides a casting surface defect identification method based on an improved deep Labv3+ (a semantic segmentation model) network model. The invention relates to a detection method based on a neural network, which can realize automatic real-time online and intelligent detection of the surface defects of castings by improving a DeepLabv3+ network model for detecting the defects of the castings. The acquired workpiece surface defect image is made into a defect data set and sent into an improved neural network, and then a trained network is finally obtained through self-defined network training and learning based on semantic segmentation and is used for workpiece surface defect image detection and defect area marking, and then the detection time is output by combining defect detection software, and meanwhile, different model detection comparison can be carried out by replacing a network model.
As shown in FIG. 1, the method for identifying the surface defects of the casting based on the improved deep Labv3+ network model comprises the following steps:
step S1, collecting a casting image data set to obtain a training set and a testing set: an industrial camera collects casting images, Labelme (a deep learning labeling tool) labels casting defects, an image data set is generated, and the image data set is divided into a training set and a testing set;
defining a standard of casting defects, and labeling a defect area of each picture by using Labelme, wherein the labeled picture comprises a defect position and a defect area, forming a data set of common industrial casting surface defects, the data set comprises 244 pictures in total, and the collected image has a pixel size of 1408 × 1580.
Each defect picture in the casting defect data set only contains one type of defect, and the size of the picture is uniformly adjusted to 1408 x 1580 pixels. Further, the acquired casting data set is divided into a training set and a testing set, wherein the training set is 211 pictures with marked defects, and the testing set is 33 pictures with unmarked casting defects.
Step S2, constructing a network model, and performing data training and correction on the network model through a training set and a testing set to generate a defect detection network;
step S3, designing a loss function of the defect detection network: the final loss function is a function comprising the cross entropy loss of the prediction result and the cross entropy loss of the true value;
and step S4, the defect detection network identifies and outputs the defect detection result of the casting and displays the detection duration.
Step S2 specifically includes the following steps:
step S21, constructing an improved DeepLabV3+ network model: improving a coding and decoding module in an existing DeepLabV3+ network model, wherein the coding and decoding module adopts a depth separable convolutional network and reduces the number of channels;
the improved DeepLabv3+ network model is adopted to construct a convolutional neural network structure for training, the improved DeepLabv3+ network model is a trunk model and a coding and decoding module in the network model, the trunk model and the coding and decoding module acquire multi-scale convolution characteristics by applying hole convolution and image level characteristics with different ratios, but the number of parameters is too large, and the calculation time is too long.
The method comprises the steps of using an Xception (network convolution structure) network structure for the backbone network, and replacing the Xception network with a MobileNet V2 (light-weight convolution neural network) network in order to increase the network computing speed and reduce the detection time of casting defect pictures.
Step S22, carrying out data training and correction to generate a defect detection network: the improved DeepLabv3+ network model is used as a network model for identifying the surface defects of the castings to extract the surface defect characteristics of the castings of the originally collected images of the castings, the improved DeepLabv3+ network model is trained by utilizing training data in a training set, and then the test set is input into the improved DeepLabv3+ network model to be corrected until the prediction accuracy of the generated defect detection network meets a prediction accuracy threshold. During training, the aim of obtaining a good network prediction model is achieved by adjusting the training parameters.
The relevant parameters set by the training network are as follows: learning rate was set to 0.01, number of threads to load data (batch) was set to 4, training step number was set to 40, batch-size was set to 4, Intel I7 CPU, GPU: nvidia _1080 TI.
As shown in fig. 2, 3, and 4, in step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, the encoding module includes a deep convolutional network layer, an encoder, and a void space pyramid pooling module, the deep convolutional network layer is a backbone model, low-level semantic features obtained by convolution are transmitted to the decoding module, and high-level semantic features obtained by the deep convolutional network layer are transmitted to the void space pyramid pooling module.
In step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, the encoding module includes a trunk model and a cavity space pyramid pooling module, the trunk model is a deep convolutional network layer MobileNet V2, low-level semantic features including shape information of defects obtained by convolving an input picture with the trunk model are directly transmitted into the decoding module, and high-level semantic features including defect position categories obtained by the input picture with an encoder are transmitted into the cavity space pyramid pooling module. Fig. 5 is a schematic diagram of a backbone model network structure.
The pictures with defects are transmitted into a coding module, a deep convolutional network layer is used, a deep neural network model comprises cavity convolution with a serial structure, the obtained low-level semantic features are transmitted into a decoding module, and the high-level semantic features are transmitted into a cavity space pyramid pooling module.
The method comprises the steps that low-level semantic features including defect shape information and obtained after an input picture is convolved through a deep convolution network layer are directly transmitted into a decoding module, the input picture is transmitted into a cavity space pyramid pooling module through an encoder and high-level semantic features including defect position categories and obtained through the deep convolution network layer, the cavity space pyramid pooling module comprises a cavity convolution module, four layers of convolution layers and one layer of pooling layer, the four layers of convolution layers are connected with the one layer of pooling layer in series, the cavity convolution module respectively adopts cavity convolution blocks with different expansion rates to control the encoder to extract resolution of features, four feature maps of the four layers of convolution layers are obtained after cavity convolution is carried out respectively, and the cavity convolution module carries out pooling to obtain one feature map of the one layer of pooling layer.
For the low-level semantic feature information obtained by the backbone model, the coding module operates the low-level semantic feature through convolution to reduce the number of channels, and the final semantic information of the low-level semantic feature is obtained after operation and is transmitted to a decoder of the decoding module; and for the high-level semantic features obtained by the cavity space pyramid pooling module, the decoding module obtains the final semantic information of the high-level semantic features through one-time up-sampling, and the final semantic information is transmitted to a decoder as an output result.
The output of the bottom layer is subjected to hollow convolution processing, the number of channels is adjusted through 1 × 1 convolution to obtain b0 layers, a hollow convolution block with the expansion rate of 6 is obtained as b1 layers, a hollow convolution block with the expansion rate of 12 is obtained as b2 layers, a hollow convolution block with the expansion rate of 18 is obtained as b3 layers, finally four layers of convolution layers and one layer of pool are stacked, the channels are adjusted through 1 × 1 convolution, then the length and the width of the middle feature layer are formed, the processed middle feature layer is stacked, the final processing is performed through 3 × 3 convolution, then the classification of each pixel point is performed, and finally the probability of each pixel point category is solved through softmax.
Based on an improved DeepLabv3+ network model, a coding-decoding structure model using the hole convolution is constructed, in the coding-decoding structure, the resolution of the extracted features of a controllable coder is introduced, and the precision and the time consumption are balanced by the adopted hole convolution. The adopted hole convolution can enlarge the receptive field by cross-pixel extraction during feature extraction under the condition of not losing information, each convolution output contains information with a larger range, and the hole convolution is used for feature extraction in the coding module.
When the coding module with the output step size of 16 outputs, the balance of speed and precision is achieved. And when the coding module with the output step length of 8 outputs, the precision is higher, but the computational complexity is increased. After balancing the advantages of all aspects, the method determines to adopt the coding module with the output step size of 16.
And the high-level semantic features enter the cavity pyramid pooling module for coding and decoding, are respectively convolved and pooled with the four convolution layers of the four cavity convolution layers and the one pooling layer of the one pooling layer to obtain five feature maps, and are further connected into a five-layer cavity space pyramid pooling module.
However, because the corresponding low-level semantic features contain more channel information and may exceed the output coding features to cause training difficulty, before the connection operation, the coding module operates the low-level semantic features through a 1 × 1 convolution to reduce the number of channels, the final semantic information is obtained after the operation, and the output result obtained by the decoding module through one-time up-sampling is transmitted to a decoder.
The coding module further comprises a stacking layer, a channel adjustment convolutional layer, an intermediate characteristic layer stacking layer, a post-processing convolutional layer and a post-processing module, wherein the stacking layer is formed by stacking the four convolutional layers and the characteristic graph obtained by the one pooling layer, the stacking layer is convoluted to adjust the channel to form the channel adjustment convolutional layer, the channel adjustment convolutional layer is subjected to length and width adjustment to form the intermediate characteristic layer, the intermediate characteristic layer stacking layer is formed after stacking the intermediate characteristic layer, the post-processing convolutional layer is formed after convolution is carried out on the intermediate characteristic layer stacking layer, and the post-processing convolutional layer is subjected to post-processing through the post-processing module.
The post-processing module comprises a pixel point classification module and a pixel point classification probability calculation module, the pixel point classification module classifies each pixel point of the image of the post-processing convolution layer, and the pixel point classification probability calculation module adopts softmax (an excitation function of an output layer) to solve the probability of each pixel point classification.
The four convolution layers and the one pooling layer are connected in series, then carry out 1 x 1 convolution operation and transmit the high-level semantic feature information to the decoding module, and transmit the high-level semantic feature information to the decoding module together with the deep neural network layer of the trunk model in the encoding module after one-time fourfold upsampling.
The coding module is used for connecting a result obtained after the high-level semantic features transmitted to the decoding module are subjected to 1 x 1 convolution operation and low-level semantic feature information transmitted by the backbone model to form a parallel structure, then performing 3 x 3 convolution operation once, further forming the decoding module through one-time fourfold up-sampling, and finally transmitting a prediction result.
Specifically, the decoding module performs upsampling on the high-level semantic features output by the encoder by using bilinear upsampling with a sampling factor of 4, the input of the decoding module consists of two parts, one part of the high-level semantic features is directly output from the deep convolutional network layer of the trunk model and is connected with corresponding low-level semantic features with the same spatial resolution output from the trunk model, the other part of the low-level semantic features is led into the hollow space pyramid pooling module through the deep convolutional network layer of the trunk model to find a low-level semantic feature map with the same semantic feature resolution output by the encoder, the number of channels is reduced by 1 × 1 convolution to make the channel proportion of the low-level semantic features and the low-level semantic feature map which are respectively obtained by the input of the two parts of the decoding module equal to the channel proportion of the semantic features output by the encoder, and the low-level semantic feature maps are connected together, after the connection processing, thinning is performed through a 3 × 3 thinning convolution, and then a final prediction result is obtained through bilinear upsampling with a sampling factor of 4.
The upsampling adopted by the output of the encoding module and the upsampling adopted by the output of the decoding module are both bilinear interpolation methods, and a formula is defined as follows:
srcx＝desx*srcω/desω
srcy＝desy*srch/desh
in the formula, srcxRepresenting the x-coordinate, src, of a pixel in the original imageyRepresenting the y-coordinate, des, of a pixel in the original imagexRepresenting the x-coordinate, des, of a pixel in a target imageyRepresenting y coordinates, src, of pixels in the target imageωRepresenting the original image width, srchRepresenting the height of the original image, desωRepresenting the width of the target image, deshRepresenting the target image height.
The loss function in step S3 uses a cross-entropy loss function, and the formula is as follows:
J＝-[y·log(p)+(1-y)·log(1-p)]
wherein y represents label of the sample, the positive class is 1, and the negative class is 0; p represents the probability that a sample is predicted to be positive.
In the step S4, outputting the casting detection result specifically is to output the detection result through a visualization software facing the defect detection of the industrial casting.
As shown in fig. 6 and 7, the software inputs the industrial casting image with defects, different recognition models can be added, and the defect detection result image can be finally output. The visualization software is software facing defect detection of the industrial casting, is Qt-based visualization industrial software, and has the characteristics of simplicity in operation, visualization and clarity, and capability of displaying detection time and optional detection network of defect detection.
The invention can obtain the following beneficial effects: the improved DeepLabv3+ network-based model can be used for defect identification of industrial castings, the identification network can be directly applied to the field of defect detection of industrial castings, digital defect data of defect identification are generated after identification, result storage and query can correspond to casting information one by one, certain data feedback is provided, the feedback can be used for optimization of casting process, guidance is provided for subsequent processes, and particularly, the method mainly takes the grinding defect position of the industrial castings as a target to detect and identify the defects of the castings.
The network model based on the improved DeepLabV3+ has the advantages of high identification speed and high identification accuracy, and the defect detection and identification method based on the deep learning algorithm and the deep neural convolution network is used for effectively solving the defects of the traditional process.
The invention designs a visual software for detecting the defects of the castings, the software can select the defect pictures to be detected by setting a network model for detecting the defects, and the detection result and the detection time are output after the detection is finished, the visual result is clear, the software is simple to operate and easy to operate, and the current situation that the defects of the current casting defect detection are artificially involved is greatly improved.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (5)
1. A casting surface defect identification method based on an improved DeepLabv3+ network model is characterized by comprising the following steps:
step S1, collecting a casting image data set to obtain a training set and a testing set: the method comprises the following steps that an industrial camera collects casting images, Labelme marks casting defects to generate an image data set, and the image data set is divided into a training set and a testing set;
step S2, constructing a network model, and performing data training and correction on the network model through a training set and a testing set to generate a defect detection network;
step S3, designing a loss function of the defect detection network: the loss function is a function comprising cross entropy loss of a prediction result and cross entropy loss of a real value;
step S4, the defect detection network identifies and outputs the defect detection result of the casting, and displays the detection duration;
step S2 specifically includes the following steps:
step S21, constructing an improved DeepLabV3+ network model: improving a coding and decoding module in an existing DeepLabV3+ network model, wherein the coding and decoding module adopts a depth separable convolutional network and reduces the number of channels;
step S22, carrying out data training and correction to generate a defect detection network: the method comprises the steps of taking an improved DeepLabv3+ network model as a network model for identifying the surface defects of the casting to extract the surface defect characteristics of the casting of an originally collected casting picture, training the improved DeepLabv3+ network model by utilizing training data in a training set, and inputting a test set into the improved DeepLabv3+ network model to correct the improved DeepLabv3+ network model until the prediction accuracy of a generated defect detection network meets a prediction accuracy threshold;
in step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, the encoding module includes a deep convolutional network layer, an encoder, and a cavity space pyramid pooling module, the deep convolutional network layer is a trunk model, low-level semantic features including defect shape information obtained by convolving an input picture with the deep convolutional network layer are directly transmitted to the decoding module, high-level semantic features including defect position categories obtained by convolving an input picture with the encoder and the deep convolutional network layer are transmitted to the cavity space pyramid pooling module, the cavity space pyramid pooling module includes a cavity convolution module, four convolution layers, and a pooling layer, the four convolution layers are connected in series with the pooling layer, the cavity convolution modules respectively adopt cavity convolution blocks with different expansion rates, controlling an encoder to extract the resolution of features, respectively performing cavity convolution to obtain four feature maps of the four convolutional layers, and performing pooling by the cavity convolution module to obtain one feature map of the one pooling layer;
for the low-level semantic feature information obtained by the backbone model, the coding module operates the low-level semantic feature through convolution to reduce the number of channels, and the final semantic information of the low-level semantic feature is obtained after operation and is transmitted to a decoder of the decoding module; for the high-level semantic features obtained by the cavity space pyramid pooling module, the decoding module obtains the final semantic information of the high-level semantic features through one-time up-sampling, and the final semantic information is used as an output result and is transmitted to a decoder;
the decoding module performs up-sampling on the high-level semantic features output by the encoder by bilinear up-sampling with a sampling factor of 4, the input of the decoding module consists of two parts, one part of the high-level semantic features is directly output from the deep convolution network layer of the trunk model and is connected with the corresponding low-level semantic features with the same spatial resolution output by the trunk model, the other part of the high-level semantic features is led into the hollow space pyramid pooling module through the deep convolution network layer of the trunk model to find a low-level semantic feature map with the same semantic feature resolution output by the encoder, the number of channels is reduced by 1-1 convolution to be the same as the channel proportion occupied by the semantic features output by the encoder, the low-level semantic features and the low-level semantic feature map which are respectively obtained by the two parts of the decoding module are connected together, and after connection processing, then thinning through a 3-by-3 thinning convolution, and then obtaining a final prediction result through bilinear upsampling with a sampling factor of 4;
the upsampling adopted by the output of the encoding module and the upsampling adopted by the output of the decoding module are both bilinear interpolation methods, and a formula is defined as follows:
srcx＝desx*srcω/desω
srcy＝desy*srch/desh
in the formula, srcxRepresenting the x-coordinate, src, of a pixel in the original imageyRepresenting the y-coordinate, des, of a pixel in the original imagexRepresenting the x-coordinate, des, of a pixel in a target imageyRepresenting y coordinates, src, of pixels in the target imageωRepresenting the original image width, srchRepresenting the height of the original image, desωRepresenting the width of the target image, deshRepresenting the target image height.
2. The method for identifying the surface defects of the castings based on the DeepLabv3+ network model as claimed in claim 1, wherein the outputting the detection results of the castings in step S4 is implemented by visualization software facing industrial casting defect detection.
3. The casting surface defect identification method based on the improved deep Labv3+ network model according to claim 1, characterized in that the coding module further comprises a stacking layer, a channel adjustment convolutional layer, an intermediate characteristic layer stacking layer, a post-processing convolutional layer and a post-processing module, the stacked layers are formed by stacking the characteristic diagrams obtained by the four convolutional layers and the one pooling layer, convolving the stack layer to adjust a channel to form the channel-adjusted convolutional layer, adjusting the length and width of the channel-adjusted convolutional layer to form the intermediate feature layer, the intermediate characteristic layer stacking layer is formed after stacking, the post-processing convolutional layer is formed after convolution of the intermediate characteristic layer stacking layer, and post-processing is carried out on the post-processing convolutional layer through the post-processing module.
4. The method for identifying the surface defects of the castings based on the DeepLabv3+ network model as claimed in claim 3, wherein the post-processing module comprises a pixel point classification module and a pixel point classification probability calculation module, the pixel point classification module classifies each pixel point of the images of the post-processing convolution layer, and the pixel point classification probability calculation module adopts softmax to solve the probability of each pixel point classification.
5. The method for identifying the surface defects of the castings based on the DeepLabv3+ network model as claimed in claim 1, wherein the loss function in the step S3 uses a cross entropy loss function, and the formula is as follows:
J＝-[y·log(p)+(1-y)·log(1-p)]
wherein y represents label of the sample, the positive class is 1, and the negative class is 0; p represents the probability that a sample is predicted to be positive.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110317123.8A CN112967271B (en) | 2021-03-25 | 2021-03-25 | Casting surface defect identification method based on improved DeepLabv3+ network model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110317123.8A CN112967271B (en) | 2021-03-25 | 2021-03-25 | Casting surface defect identification method based on improved DeepLabv3+ network model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112967271A CN112967271A (en) | 2021-06-15 |
| CN112967271B true CN112967271B (en) | 2022-04-19 |
Family
ID=76278400
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110317123.8A Active CN112967271B (en) | 2021-03-25 | 2021-03-25 | Casting surface defect identification method based on improved DeepLabv3+ network model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112967271B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114441552A (en) * | 2022-02-28 | 2022-05-06 | 中国船舶重工集团公司第七0七研究所 | High-precision optical fiber loop glue filling defect detection and correction method |
| CN116703834B (en) * | 2023-05-22 | 2024-01-23 | 浙江大学 | Method and device for judging and grading excessive sintering ignition intensity based on machine vision |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986091A (en) * | 2018-07-12 | 2018-12-11 | 太原科技大学 | Casting defect image detecting method based on depth Hash network |
| CN109389599A (en) * | 2018-10-25 | 2019-02-26 | 北京阿丘机器人科技有限公司 | A kind of defect inspection method and device based on deep learning |
| CN110222681A (en) * | 2019-05-31 | 2019-09-10 | 华中科技大学 | A kind of casting defect recognition methods based on convolutional neural networks |
| CN111223088A (en) * | 2020-01-16 | 2020-06-02 | 东南大学 | Casting surface defect identification method based on deep convolutional neural network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8204291B2 (en) * | 2007-10-15 | 2012-06-19 | General Electric Company | Method and system for identifying defects in a radiographic image of a scanned object |
-
2021
- 2021-03-25 CN CN202110317123.8A patent/CN112967271B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986091A (en) * | 2018-07-12 | 2018-12-11 | 太原科技大学 | Casting defect image detecting method based on depth Hash network |
| CN109389599A (en) * | 2018-10-25 | 2019-02-26 | 北京阿丘机器人科技有限公司 | A kind of defect inspection method and device based on deep learning |
| CN110222681A (en) * | 2019-05-31 | 2019-09-10 | 华中科技大学 | A kind of casting defect recognition methods based on convolutional neural networks |
| CN111223088A (en) * | 2020-01-16 | 2020-06-02 | 东南大学 | Casting surface defect identification method based on deep convolutional neural network |
Non-Patent Citations (2)
| Title |
|---|
| Research on Automatic Recognition of Casting Defects Based on Deep Learning;limin duan;《IEEE：Research on Automatic Recognition of Casting Defects Based on Deep Learning》;IEEE;20210122;第9卷;全文 * |
| 基于Mask R-CNN的铸件X射线DR图像缺陷检测研究;蔡彪等;《仪器仪表学报》;20200315(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112967271A (en) | 2021-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110570396B (en) | Industrial product defect detection method based on deep learning | |
| CN109829891B (en) | Magnetic shoe surface defect detection method based on dense generation of antagonistic neural network | |
| CN110992317A (en) | PCB defect detection method based on semantic segmentation | |
| CN110543878A (en) | pointer instrument reading identification method based on neural network | |
| CN112967271B (en) | Casting surface defect identification method based on improved DeepLabv3+ network model | |
| WO2022236876A1 (en) | Cellophane defect recognition method, system and apparatus, and storage medium | |
| CN112037219A (en) | Metal surface defect detection method based on two-stage convolution neural network | |
| CN112766136B (en) | Space parking space detection method based on deep learning | |
| CN113643268B (en) | Industrial product defect quality inspection method and device based on deep learning and storage medium | |
| CN110598698B (en) | Natural scene text detection method and system based on adaptive regional suggestion network | |
| CN112862811A (en) | Material microscopic image defect identification method, equipment and device based on deep learning | |
| CN114581782A (en) | Fine defect detection method based on coarse-to-fine detection strategy | |
| CN112365478A (en) | Motor commutator surface defect detection model based on semantic segmentation | |
| CN114331869A (en) | Dam face crack semantic segmentation method | |
| CN115908793A (en) | Coding and decoding structure semantic segmentation model based on position attention mechanism | |
| CN114842308A (en) | Method for establishing target prejudgment optimization model based on full-feature fusion | |
| CN114037684A (en) | Defect detection method based on yolov5 and attention mechanism model | |
| CN109815957A (en) | A kind of character recognition method based on color image under complex background | |
| CN112614113A (en) | Strip steel defect detection method based on deep learning | |
| CN115601357B (en) | Stamping part surface defect detection method based on small sample | |
| CN110136098B (en) | Cable sequence detection method based on deep learning | |
| CN116862883A (en) | Concrete slump detection method based on image semantic segmentation | |
| CN114937153B (en) | Visual characteristic processing system and method based on neural network in weak texture environment | |
| CN114926420A (en) | Identification and counting method of target naan based on cross-level feature enhancement | |
| CN112926694A (en) | Method for automatically identifying pigs in image based on improved neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |