CN117237325A - Industrial defect detection method based on depth network - Google Patents
Industrial defect detection method based on depth network Download PDFInfo
- Publication number
- CN117237325A CN117237325A CN202311330420.1A CN202311330420A CN117237325A CN 117237325 A CN117237325 A CN 117237325A CN 202311330420 A CN202311330420 A CN 202311330420A CN 117237325 A CN117237325 A CN 117237325A
- Authority
- CN
- China
- Prior art keywords
- industrial
- depth network
- surface defect
- image
- industrial product
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 90
- 238000001514 detection method Methods 0.000 title claims abstract description 55
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 238000000034 method Methods 0.000 claims description 13
- 230000007246 mechanism Effects 0.000 claims description 7
- 238000005096 rolling process Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 claims description 2
- 238000009826 distribution Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 4
- 229910000831 Steel Inorganic materials 0.000 description 3
- 230000004913 activation Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 239000010959 steel Substances 0.000 description 3
- 239000000463 material Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
The invention belongs to the field of image processing, and particularly relates to an industrial defect detection method based on a depth network, which comprises the following steps: acquiring a surface defect image of an industrial product; carrying out data preprocessing on the surface defect image of the industrial product; inputting the industrial product surface defect image into a trained depth network, and outputting an industrial defect prediction result. The invention aims to solve the problem that the detection effect of the industrial defect detection method on small-size and micro-size defects is poor, and meets the requirement of industrial scenes on rapid and accurate detection of surface defect detection.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to an industrial defect detection method based on a depth network.
Background
The detection of the surface defects of industrial products plays an important role in ensuring the quality of the products and the production safety. In the manufacturing process of industrial products, various defects occur on the surfaces of the products due to the material quality, manufacturing process, production equipment and the like. These defects can affect the appearance of the product, shorten the service life of the product, and cause great economic loss and potential safety risks to enterprises. Therefore, the rapid and accurate detection of the surface defects of the industrial products is of great importance under the conditions.
The defect detection method for the industrial product surface mainly comprises infrared detection, eddy current detection, manual visual inspection, image processing detection method based on traditional feature recognition and the like. The manual visual inspection method has strong subjectivity, does not have real-time performance, and is easy to cause the condition of missed detection and false detection; the eddy current detection and infrared detection methods require higher cost and technology and cannot be popularized; the traditional image processing detection method needs a manual design algorithm to manually extract the characteristics, and the method has poor generalization performance and is inconvenient. At present, a target detection algorithm based on deep learning is developed rapidly, and the method utilizes the strong image feature extraction capability of a convolutional neural network, has the advantages of high instantaneity, strong generalization, low cost and the like, and gradually replaces the traditional detection method.
However, since industrial defects tend to have more complex features, such as multiple scales, multiple small-size defects, complex background interference, etc., this presents a significant challenge for deep learning-based object detection algorithms.
Disclosure of Invention
Aiming at the problems, the invention provides an industrial product defect detection method based on a depth network, which aims to solve the problem that the detection effect of the industrial defect detection method on small-size defects is poor and meet the requirement of industrial scenes on rapid and accurate detection on surface defect detection.
An industrial defect detection method based on a depth network, comprising:
acquiring a surface defect image of an industrial product;
carrying out data preprocessing on the surface defect image of the industrial product;
inputting the industrial product surface defect image into a trained depth network, and outputting an industrial defect prediction result.
Further, the industrial product surface defect image includes an industrial product image with any one or more defects of rolling chips, plaque, cracks, rat bites, or shorts.
Further, the data preprocessing of the industrial product surface defect image comprises the following steps: and randomly grouping all the industrial product surface defect images according to two to four, randomly cutting, turning over and shrinking the defect images in each group, merging the transformed defect images in each group into one picture, and adjusting corresponding label information according to the positions of sub-images in the merged images.
Further, the method further comprises the steps of denoising and contrast enhancement treatment on the obtained industrial product surface defect image before the data preprocessing is carried out on the industrial product surface defect image, and processing the industrial product surface defect image by adopting different color channels to obtain the industrial product surface defect image with the highest background contrast.
Further, the depth network comprises a baseline model YOLOv8, and a cooperative attention mechanism is added into a Backbone network Backbone of the baseline model YOLOv 8; adding a small target detection layer on a network layer Neck of the baseline model YOLOv 8; adding spatial attention in the small target detection path and the PAN path; the complete cross-ratio CIOU loss function of the baseline model YOLOv8 was replaced with a normalized gaussian loss function.
Further, the training process for the depth network comprises the following steps:
s1: cleaning and normalizing the data in the data set, and dividing the normalized data to obtain a training data set;
s2: setting parameters required by deep network training, including learning rate, iteration times, input image size and an optimizer, and initializing the constructed deep network;
s3: inputting data in the training set into a depth network for forward propagation to obtain a model output result and model parameters;
s4: calculating a loss function of the model according to the model output result and the model parameters;
s5: inputting the data in training into a depth network for back propagation, and optimizing model parameters by adopting a gradient descent algorithm;
s6: and repeating the steps S3 to S5, and completing training of the model when the loss function converges.
The invention has the beneficial effects that:
1. the invention adopts a deep learning method, realizes end-to-end target detection, and solves the problems of the traditional method such as no real-time performance, poor robustness and the like.
2. According to the invention, CA attention is added into the top layer path of the backstone, so that the feature extraction capability of the backstone is enhanced, the model pays attention to favorable features, and unimportant features are inhibited.
3. According to the invention, a small target detection layer is added into the Neck network, so that the detection precision of the model on the small target defects is improved.
4. The invention embeds the spatial attention in the Neck network aiming at the extracted position of the small target feature so as to improve the sensitivity of the Neck network to the position information of the small target.
5. The invention adopts normalized Gaussian Wasserstein Distance Loss to improve the detection precision of the model on the micro-size defects.
Drawings
FIG. 1 is a flow chart of a depth network based industrial defect detection method of the present invention;
FIG. 2 is a block diagram of a constructed deep network model of the present invention;
FIG. 3 is a block diagram of an employed CA attention mechanism of the present invention;
fig. 4 is a block diagram of an employed spatial attention mechanism of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
An industrial defect detection method based on a depth network, as shown in fig. 1, the method comprises the following steps:
101. acquiring a surface defect image of an industrial product; in the embodiment of the invention, an industrial product surface defect image is required to be acquired, and an industrial product surface defect data set is constructed, wherein the industrial product surface defect image can be an unlabeled image to be detected or a labeled image, if the unlabeled image is the unlabeled image, the unlabeled image can be used for training and optimizing a depth network, and if the unlabeled image is the unlabeled image, the defect detection can be carried out through the depth network.
Specifically, surface defect images of steel materials and PCB boards, such as defect images of rolling scraps, plaques, cracks, rat bites, short circuits and the like, are obtained. The acquisition mode is to establish a data set for the disclosed data set or a method for acquiring industrial surface defects through shooting. And labeling the image by using a labelimg image labeling tool, generating an xml file after labeling, and converting the corresponding xml file into a txt file.
102. Carrying out data preprocessing on the surface defect image of the industrial product;
in the embodiment of the invention, all the defect images can be randomly grouped according to two to four, operations such as random cutting, overturning, shrinking and the like are carried out on the defect images in each group, the transformed defect images in each group are combined in one picture, and then the corresponding label information is adjusted according to the positions of the sub-images in the combined images.
In some embodiments of the present invention, the present invention may further perform denoising and contrast enhancement processing on the obtained industrial product surface defect image, and process the industrial product surface defect image with different color channels to obtain an industrial product surface defect image with the highest background contrast.
Interference information in the image can be reduced through noise reduction and contrast enhancement, industrial defects such as rolling scraps, plaques, cracks, rat bites or short circuits are highlighted, and subsequent defect detection is facilitated.
103. Inputting the industrial product surface defect image into a trained depth network, and outputting an industrial defect prediction result.
In an embodiment of the present invention, it is also necessary to construct a depth network based on modified YOLOv8 (you only look once v), as exemplified by the following:
(1) Configuration environment: and the YOLOv8 is an official source code on the gitoub, and a corresponding package is installed according to a source code demand file.
(2) Model parameter setting: the learning rate is 1e-2, the optimizer is AdamW, the momentum is 0.9, the iteration number is 200, and the batch size is 32.
(3) Data input: and inputting the constructed steel surface defect data set and the PCB defect data set into a depth network, wherein the data comprise defect images, defect labels and model configuration files.
(4) Model improvement: the baseline model used in the invention is YOLOv8, and the model is specifically improved in the following four ways, a depth network structure diagram based on the improved YOLOv8 is shown in fig. 2, and a cooperative attention mechanism is added into a Backbone network Backbone of the baseline model YOLOv 8; adding a small target detection layer on a network layer Neck of the baseline model YOLOv 8; adding spatial attention in the small target detection path and the PAN path; the complete cross-ratio CIOU loss function of the baseline model YOLOv8 was replaced with a normalized gaussian loss function. Specific:
1) Attention mechanisms (Coordinate Attention, CA) are added to the YOLOv8 Backbone network Backbone:
the CA attention module is added to the top layer of the Backbone to improve the extraction of important information in the strong semantic information and suppress unimportant information. The CA attention module can focus on both channel information and direction-dependent location information.
In the embodiment of the present invention, the overall structure of the CA attention module is shown in FIG. 3, and H, W and C are the height, width and channel number of the image, respectively. The input feature map is X, and the CA attention module will encode the input feature map in the horizontal and vertical directions:
wherein:the output of the c-th channel is shown with height h and width w.
Further, the two results are spliced, and the characteristic diagram after coding is obtained through convolution operation:
f＝δ(F 1 ([z h ,z w ]))
wherein: z h 、z w The outputs of the channels having a height h and a width w, respectively; f (F) 1 Representing a convolution operation; delta represents nonlinear activation.
Further, the intermediate feature map f is split to obtain f respectively h And f w Then, the convolution is used for ascending dimension, and the activation function sigmoid is used for nonlinear activation to obtain the final attention weight:
g h ＝σ(F h (f h ))
g w ＝σ(F w (f w ))
wherein: g h 、g w Horizontal and vertical attention weights representing an input feature map; f (F) h 、F w Representing the horizontal and vertical directionsConvolution of the directions.
Finally, through the calculation, the output y of the input feature map after passing through the CA attention module c (i,j)：
Wherein subscript c represents a channel.
2) Adding a small target detection layer
The adding of the small target detection layer in YOLOv8 specifically comprises the following steps: and adding the output of the second layer of the backup, and splicing the feature map of the output with the deepest layer feature of the FPN. The stitched features are then added to the path of the PAN to output a four-scale feature map to the head structure. The fourth scale of feature map added is 160 x 160, which has a smaller receptive field suitable for detection of small targets.
3) Adding spatial attention in small target detection paths and PAN paths
The adding of the spatial attention in the small target detection path and the PAN path is specifically as follows: after adding the small object detection layer, a spatial attention mechanism, i.e. cbs_spa, is embedded behind the CBS module in the bottom-up path of the PAN to extract the position information of the small object, and the structure diagram of the spatial attention used is shown in fig. 4.
4) The loss function of YOLOv8 was replaced with normalized gaussian Wasserstein Distance Loss (WDLoss), and the CIOU loss function of YOLOv8 was replaced with WDLoss to improve detection of micro-size defects. WDLoss is defined as:
wherein NWD (N) a ,N b ) Represents N a And N b Between them are returned toNormalized Gaussian loss, N a 、N b For bounding box a (cx) a ,cy a ,w a ,h a ) And bounding box b (cx) b ,cy b ,w b ,h b ) A modeled gaussian distribution; cx (cx) a Representing the central abscissa, cy, of the bounding box a a Representing the central ordinate, w, of the bounding box a a Represents the width, h of the bounding box a a Indicating the height of the bounding box a, cx b Representing the central abscissa, cy, of the bounding box b b Representing the central ordinate, w, of the bounding box b b Represents the width, h, of the bounding box b b Representing the height, W, of the bounding box b 2 2 (N a ,N b ) Is N a And N b A distance measure between; c is a constant. Further, the training set of the constructed steel and PCB surface defect data set is input into an improved depth network for training, image data is input into a model to obtain a feature map, the feature map is input into a detection decoupling branch, and the decoupling branch is divided into a classification branch and a regression branch. Regression branches are used to make regression predictions for the bounding boxes. And classifying the defects by using classification branches to obtain the type probability of the defects in the boundary box, and sequencing the classification scores of the boundary box by using non-maximum suppression. And calculating a loss value between the predicted value and the group trunk according to the loss function, and carrying out back propagation through an optimization algorithm to reduce the loss value until the iteration times are completed, wherein the model training is finished.
Further, the trained model weight file is used for verifying the verification set, and the performance of the model can be comprehensively evaluated through average precision, recall rate, average precision mean value and FPS.
The above metrics are defined as follows:
wherein the TP representation model determines a positive sample as a positive sample; FP denotes determining a negative sample as a positive sample; FN denotes determining a positive sample as a negative sample; p (r) represents the precision of the recall; n represents the number of categories; AP (Access Point) i Indicating the accuracy of detection of the category.
And finally, testing the test set by the trained model weight file to obtain an actual defect prediction result.
While the foregoing is directed to embodiments, aspects and advantages of the present invention, other and further details of the invention may be had by the foregoing description, it will be understood that the foregoing embodiments are merely exemplary of the invention, and that any changes, substitutions, alterations, etc. which may be made herein without departing from the spirit and principles of the invention.
Claims (7)
1. An industrial defect detection method based on a depth network, comprising:
acquiring a surface defect image of an industrial product;
carrying out data preprocessing on the surface defect image of the industrial product;
inputting the industrial product surface defect image into a trained depth network, and outputting an industrial defect prediction result.
2. The depth network-based industrial defect detection method of claim 1, wherein the industrial product surface defect image comprises an industrial product image with any one or more defects of rolling, plaque, crack, rat bite, or short circuit.
3. The depth network-based industrial defect detection method of claim 1, wherein the data preprocessing of the industrial product surface defect image comprises: and randomly grouping all the industrial product surface defect images according to two to four, randomly cutting, turning over and shrinking the defect images in each group, merging the transformed defect images in each group into one image, and adjusting corresponding label information according to the positions of sub-images in the merged images.
4. The method for detecting industrial defects based on a depth network according to claim 1, wherein the step of performing data preprocessing on the industrial product surface defect image further comprises the steps of performing denoising and contrast enhancement processing on the obtained industrial product surface defect image, and processing the industrial product surface defect image by adopting different color channels to obtain the industrial product surface defect image with the highest background contrast.
5. The method for detecting industrial defects based on a depth network according to claim 1, wherein the depth network comprises a baseline model YOLOv8, and a cooperative attention mechanism is added into a Backbone network Backbone of the baseline model YOLOv 8; adding a small target detection layer on a network layer Neck of the baseline model YOLOv 8; adding spatial attention in the small target detection path and the PAN path; the complete cross-ratio CIOU loss function of the baseline model YOLOv8 was replaced with a normalized gaussian loss function.
6. The depth network-based industrial defect detection method of claim 5, wherein the normalized gaussian loss function is expressed as:
wherein NWD (N) a ,N b ) Represents N a And N b Normalized Gaussian loss between N a 、N b For bounding box a (cx) a ,cy a ,w a ,h a ) And bounding box b (cx) b ,cy b ,w b ,h b ) A modeled gaussian distribution; cx (cx) a Representing the central abscissa, cy, of the bounding box a a Representing the central ordinate, w, of the bounding box a a Represents the width, h of the bounding box a a Indicating the height of the bounding box a, cx b Representing the central abscissa, cy, of the bounding box b b Representing the central ordinate, w, of the bounding box b b Represents the width, h, of the bounding box b b The height of the bounding box b is indicated,is N a And N b A distance measure between; c is a constant.
7. The method for detecting industrial defects based on a depth network according to claim 1, wherein the training of the depth network comprises:
s1: cleaning and normalizing the data in the data set, and dividing the normalized data to obtain a training data set;
s2: setting parameters required by deep network training, including learning rate, iteration times, input image size and an optimizer, and initializing the constructed deep network;
s3: inputting data in the training set into a depth network for forward propagation to obtain a model output result and model parameters;
s4: calculating a loss function of the model according to the model output result and the model parameters;
s5: inputting the data in training into a depth network for back propagation, and optimizing model parameters by adopting a gradient descent algorithm;
s6: and repeating the steps S3 to S5, and completing training of the model when the loss function converges.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311330420.1A CN117237325A (en) | 2023-10-13 | 2023-10-13 | Industrial defect detection method based on depth network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311330420.1A CN117237325A (en) | 2023-10-13 | 2023-10-13 | Industrial defect detection method based on depth network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117237325A true CN117237325A (en) | 2023-12-15 |
Family
ID=89096695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311330420.1A Pending CN117237325A (en) | 2023-10-13 | 2023-10-13 | Industrial defect detection method based on depth network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237325A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117876375A (en) * | 2024-03-13 | 2024-04-12 | 齐鲁工业大学(山东省科学院) | Water heater liner defect detection system and method based on improved YOLOv8 |
-
2023
- 2023-10-13 CN CN202311330420.1A patent/CN117237325A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117876375A (en) * | 2024-03-13 | 2024-04-12 | 齐鲁工业大学(山东省科学院) | Water heater liner defect detection system and method based on improved YOLOv8 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110543878B (en) | Pointer instrument reading identification method based on neural network | |
| CN111862064B (en) | Silver wire surface flaw identification method based on deep learning | |
| CN110348376B (en) | Pedestrian real-time detection method based on neural network | |
| CN109035233B (en) | Visual attention network system and workpiece surface defect detection method | |
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| CN109215009B (en) | Continuous casting billet surface image defect detection method based on deep convolution neural network | |
| CN113822880A (en) | Crack identification method based on deep learning | |
| Wan et al. | Ceramic tile surface defect detection based on deep learning | |
| CN106610969A (en) | Multimodal information-based video content auditing system and method | |
| CN117237325A (en) | Industrial defect detection method based on depth network | |
| CN106355579A (en) | Defect detecting method of cigarette carton surface wrinkles | |
| Vivekananthan et al. | Concrete bridge crack detection by image processing technique by using the improved OTSU method | |
| CN115170529A (en) | Multi-scale tiny flaw detection method based on attention mechanism | |
| CN114972759A (en) | Remote sensing image semantic segmentation method based on hierarchical contour cost function | |
| CN111915595A (en) | Image quality evaluation method, and training method and device of image quality evaluation model | |
| CN114049305A (en) | Distribution line pin defect detection method based on improved ALI and fast-RCNN | |
| CN109859199B (en) | Method for detecting quality of freshwater seedless pearls through SD-OCT image | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN114881998A (en) | Workpiece surface defect detection method and system based on deep learning | |
| Mi et al. | Research on steel rail surface defects detection based on improved YOLOv4 network | |
| KR20080079798A (en) | Method of face detection and recognition | |
| CN117197146A (en) | Automatic identification method for internal defects of castings | |
| CN116777873A (en) | Strip steel surface defect detection method based on improved YOLOv7 algorithm | |
| CN116630989A (en) | Visual fault detection method and system for intelligent ammeter, electronic equipment and storage medium | |
| CN115953387A (en) | Radiographic image weld defect detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |