CN113222901A - Method for detecting surface defects of steel ball based on single stage - Google Patents
Method for detecting surface defects of steel ball based on single stage Download PDFInfo
- Publication number
- CN113222901A CN113222901A CN202110418127.5A CN202110418127A CN113222901A CN 113222901 A CN113222901 A CN 113222901A CN 202110418127 A CN202110418127 A CN 202110418127A CN 113222901 A CN113222901 A CN 113222901A
- Authority
- CN
- China
- Prior art keywords
- steel ball
- model
- data set
- network
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 48
- 239000010959 steel Substances 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000007547 defect Effects 0.000 title claims abstract description 38
- 238000001514 detection method Methods 0.000 claims abstract description 43
- 238000012549 training Methods 0.000 claims abstract description 31
- 238000012360 testing method Methods 0.000 claims abstract description 10
- 230000003416 augmentation Effects 0.000 claims abstract description 3
- 238000012795 verification Methods 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 238000013434 data augmentation Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 2
- 238000007710 freezing Methods 0.000 claims description 2
- 230000008014 freezing Effects 0.000 claims description 2
- 238000013526 transfer learning Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 claims 3
- 239000003350 kerosene Substances 0.000 claims 1
- 239000007788 liquid Substances 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 7
- 238000011156 evaluation Methods 0.000 abstract 1
- 230000008569 process Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000004886 process control Methods 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for detecting surface defects of a steel ball based on a single stage, which has higher detection precision, detection efficiency and robustness compared with the traditional manual detection. The single-stage detection method is based on a YOLOv4 network structure pre-training model. The method mainly comprises the following steps: acquiring steel ball surface image data, image augmentation, data set image annotation, data set division, constructing a pre-training model, training a model and verifying the model. The method model can automatically extract the characteristics of the steel ball surface defects and accurately and quickly detect the positions of various defects on the steel ball surface. The method uses Python programming language, is realized by taking a Keras framework as a front end, takes Tensorflow as data processing of a rear end, and uses GPU (NVIDIA, GTX1080Ti) to train, verify and test the model so as to obtain corresponding evaluation indexes and test results. According to the experimental result, the method can quickly and accurately detect the surface defects of the steel balls.
Description
Technical Field
The invention relates to the technical field of digital image processing, in particular to a method for detecting surface defects of a steel ball based on a single stage.
Background
Due to the influence of technical factors such as different steel ball raw material batches, the precision of a processing machine tool, the process control in the processing process and the like, local fine surface defects can be generated in the processing process of the steel balls. The traditional manual detection has the problems of low detection precision, low detection efficiency and poor anti-interference performance. Therefore, a target detection method based on deep learning is developed and widely applied to industrial detection technologies, such as Fast RCNN, SSD and other target detection models, but the detection method has problems of poor model accuracy, slow detection speed, and the like when defects are of unbalanced types and small target defects are detected.
Disclosure of Invention
The invention aims to solve the problems of poor accuracy, low detection speed and the like of a steel ball surface defect detection method, and provides a method based on single-stage steel ball surface defect detection. According to the deep learning, the characteristics can be automatically extracted from the original data, and the extracted characteristics are correspondingly processed. Because the characteristics do not need to be extracted manually, the algorithm can enable the output result to be more objective. The single-stage target detection method considers that all areas of the image are potential targets, target candidate areas do not need to be generated, and target classification and accurate positioning are directly carried out on each position. Therefore, single-stage target detection has faster detection efficiency. The invention mainly solves the problems of low stability, low detection precision, low efficiency and the like of the prior art for detecting the surface defects of the steel balls, thereby effectively improving the detection efficiency and the detection precision of products and finally improving the quality of the products. In order to verify the accuracy and reliability of the algorithm, the algorithm is subjected to experimental analysis, the average progress (mAP) of defect identification reaches 91.37%, and the intersection ratio (IoU) reaches 93.25%.
In order to achieve the purpose, the invention adopts the technical scheme that: a method for detecting surface defects of steel balls based on a single stage comprises the following specific steps:
firstly, acquiring steel ball surface image data and establishing a corresponding data set;
step two, properly expanding the steel ball surface image data acquired in the step one by adopting a data augmentation method;
step three, marking the steel balls with surface defects one by the data set obtained in the step two;
step four, dividing the data set obtained in the step three into a training set, a verification set and a test set, wherein 80% of the data set is used as the training data set, 10% of the data set is used as the verification data set, and 10% of the data set is used as the test data set;
constructing a pre-training model of a YOLOv4 network structure, wherein the main network part is a CSPDarknet53 network, the feature fusion part is SPP and PAN, and the prediction part is YOLOv 3;
step six, training a model: adopting a transfer learning mode, training a finished YOLOv4 pre-training model by using an MS COCO data set, initializing weights in an original model, finely adjusting parameters in the method, freezing a part of convolution layers (generally the first three layers) in a trunk network, setting parameters such as dynamic attenuation learning rate, and the like, wherein a loss function from the training model to a verification set does not fall, finally realizing the accurate positioning of the surface defects of the various types of steel balls, and simultaneously saving the best weights of the trained models to obtain a qualified YOLOv4 model;
step seven, verifying the model: and (4) carrying out defect detection on the steel ball surface image by using a qualified YOLOv4 model to obtain a detection result.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention can accurately and quickly identify the defects and position the defects in actual production by the detection method, the average progress (mAP) of defect identification reaches 91.37%, and the intersection ratio (IoU) reaches 93.25%.
2. The method can be applied to the surface quality detection of other industrial products, and provides a new method for processing by digital image technology.
3. The method can replace manual detection, unify detection standards, reduce labor force, reduce production cost and improve detection efficiency and accuracy.
Drawings
FIG. 1 is a flow chart of a single stage detection method of the present invention;
FIG. 2 is a network architecture diagram of YOLOv4 of the present invention;
FIG. 3 is a schematic view of the detecting device of the present invention;
FIG. 4 is a schematic view of the position of the camera according to the present invention;
FIG. 5 is an image captured by a camera of the present invention;
FIG. 6 is a standard image generated from a camera capturing a picture according to the present invention;
FIG. 7 is a data set image annotation of the present invention;
FIG. 8 is a diagram illustrating the detection results of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are merely illustrative of the invention and are not intended to be limiting.
As shown in fig. 1, image data is acquired through a steel ball surface image acquisition system based on machine vision, the acquired image is preprocessed, namely, an image of a data set is enlarged, then the data set is labeled by using a tool LabelImg, the labeled data set is divided to obtain a training set, a verification set and a test set, wherein 80% of the training set, 10% of the verification set and 10% of the test set are used as training data sets, a pre-training model is constructed and trained and adjusted to obtain a qualified Yolov4 model, and finally, a steel ball surface image to be detected is input into an optimal model for defect detection to obtain a detection result.
The method for detecting the surface defects of the steel ball based on the single-stage comprises the following processing steps:
acquiring steel ball surface image data, establishing a steel ball surface defect image acquisition system based on a machine vision method, acquiring the steel ball surface image data by using a double black and white CCD camera, and establishing a corresponding data set;
step two, properly expanding the steel ball surface image data acquired in the step one by adopting a data augmentation method, mainly aiming at preventing the overfitting of the model and enhancing the robustness and the bloom of the model, wherein the specific image augmentation method comprises the following steps: methods that do not change the data properties, such as horizontal flipping, vertical flipping, etc.;
thirdly, marking the steel balls with the surface defects one by using a tool LabelImg on the data set obtained in the second step, and obtaining an xml format file corresponding to each image in the data set after marking, wherein the xml file comprises marking information of the corresponding image;
step four, dividing the data set obtained in the step three into a training set, a verification set and a test set, wherein 80% of the data set is used as the training data set, 10% of the data set is used as the verification data set, and 10% of the data set is used as the test data set;
constructing a pre-training model of a YOLOv4 network structure, wherein a main network part is a CSPDarknet53 network, a feature fusion part is SPP and PAN, a prediction part is YOLOv3, and a CSPDarknet53 network is formed by adding a CSP module into a Darknet53 network, so that the feature extraction capability of the network can be effectively improved, the accuracy can be kept while the weight is reduced, the calculation bottleneck is reduced, and the memory cost is reduced; the SPP network is called Spatial Pyramid Pooling, and the purpose of the method is mainly to increase the receptive field of the YOLOv4 network, complete the maximum Pooling operation on the learned features in the network, and finally splice all the obtained feature maps together according to dimensions to form a new feature map; the PAN is mainly used for further carrying out down-sampling on the features after the features in the network complete the up-sampling, more specifically embodying the sampled features, and finally splicing the sampled features with other features for target detection;
step six, training a model: the method adopts the idea of deep migration learning to carry out model training on the method because the number of collected samples in a data set is limited and cannot support the original training of the method, and in the training process of the method, firstly, a model which is trained by an MS COCO data set, namely a pre-training model, is used for initializing the weight in the original model; then, fine adjustment is carried out on parameters in the method, partial parameters (generally the first three layers) in the main network are frozen firstly, the parameters of the layers are ensured not to participate in updating, the updating efficiency of the model parameters is improved conveniently, parameters such as dynamic attenuation learning rate and the like are set, the curve from the training model to the loss function is stable and does not fall down, accurate positioning of various defects on the surface of the steel ball is finally realized, meanwhile, the weight of the model is saved, and a qualified Yolov4 model is obtained;
step seven, verifying the model: and (4) carrying out defect detection on the steel ball surface image by using a qualified YOLOv4 model to obtain a detection result.
The invention can automatically position the surface defect characteristics of the steel ball and improve the detection precision and the detection efficiency of the surface defect detection of the steel ball.
Claims (6)
1. A method for detecting surface defects of steel balls based on a single stage is characterized by comprising the following specific steps:
firstly, acquiring steel ball surface image data and establishing a corresponding data set;
step two, properly expanding the steel ball surface image data acquired in the step one by adopting a data augmentation method;
step three, marking the steel balls with surface defects one by the data set obtained in the step two;
step four, dividing the data set obtained in the step three into a training set, a verification set and a test set, wherein 80% of the data set is used as the training data set, 10% of the data set is used as the verification data set, and 10% of the data set is used as the test data set;
constructing a pre-training model of a YOLOv4 network structure, wherein the main network part is a CSPDarknet53 network, the feature fusion part is SPP and PAN, and the prediction part is YOLOv 3;
step six, training a model: adopting a transfer learning mode, training a finished YOLOv4 pre-training model by using an MS COCO data set, initializing weights in an original model, finely adjusting parameters in the method, freezing a part of convolution layers (generally the first three layers) in a trunk network, setting parameters such as dynamic attenuation learning rate, and the like, wherein a loss function from the training model to a verification set does not fall, finally realizing the accurate positioning of the surface defects of the various types of steel balls, and simultaneously saving the best weights of the trained models to obtain a qualified YOLOv4 model;
step seven, verifying the model: and (4) carrying out defect detection on the steel ball surface image by using a qualified YOLOv4 model to obtain a detection result.
2. The method for detecting the surface defects of the steel ball based on the single stage as claimed in claim 1, wherein the steel ball surface image acquisition system based on the machine vision is selected to be divided into a feeding mechanism, an illuminating mechanism, a steel ball surface unfolding mechanism and an image acquisition device; in order to reduce the reflection of the surface of the steel ball, the unfolding system is soaked in a detection liquid with a medium of aviation kerosene; the unfolding disc is provided with 90 detection cavities, two CCD cameras are arranged at the same time, each CCD camera can image seven steel balls, the two CCD cameras can image 14 steel balls at the same time, and seven steel balls are arranged between the two cameras at intervals.
3. The method for detecting the surface defects of the steel balls based on the single-stage as claimed in claim 1, wherein the image augmentation method is selected from the methods of horizontal inversion, vertical inversion and the like without changing the data properties.
4. The method for detecting the steel ball surface defects based on the single stage as claimed in claim 1, wherein the image data labels are of the type of dots, clusters of dots, scratches, pits, etc., which are common steel ball surface defects.
5. The method as claimed in claim 1, wherein the labeling of the image of the data set is performed by a LabelImg tool, and an xml-format file corresponding to each image in the data set is obtained after labeling, wherein the xml file includes the labeling information of the corresponding image.
6. The method for detecting the surface defects of the steel balls based on the single stage as claimed in claim 1, wherein the YOLOv4 model takes CSPDarkne53 as a backbone network, and the CSPDarknet53 network is formed by adding a CSP module into a Darknet53 network, so that the feature extraction capability of the network can be effectively improved; the method comprises the steps of taking SPP (spatial pyramid pooling) and PAN (personal area network) as feature fusion layers of the network, improving the feature fusion capability of the method to all levels, mainly aiming at increasing the receptive field of the YOLOv4 network, completing maximum pooling operation on learned features in the network, finally splicing all obtained feature graphs together according to dimensions to form a new feature graph, mainly aiming at the PAN, after completing up-sampling work on the features in the network, further performing down-sampling work on the features, further embodying the sampled features, finally splicing the sampled features together with other features for target detection, and taking YOLOv3 as a prediction part, wherein loss functions during training and nms screened by a prediction frame are mainly improved to be DIOU _ nms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110418127.5A CN113222901A (en) | 2021-04-19 | 2021-04-19 | Method for detecting surface defects of steel ball based on single stage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110418127.5A CN113222901A (en) | 2021-04-19 | 2021-04-19 | Method for detecting surface defects of steel ball based on single stage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113222901A true CN113222901A (en) | 2021-08-06 |
Family
ID=77087734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110418127.5A Pending CN113222901A (en) | 2021-04-19 | 2021-04-19 | Method for detecting surface defects of steel ball based on single stage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113222901A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114495003A (en) * | 2022-01-24 | 2022-05-13 | 上海申视信科技有限公司 | People number identification and statistics method and system based on improved YOLOv3 network |
| CN116229197A (en) * | 2022-11-24 | 2023-06-06 | 浙江大学 | Pre-labeling model construction method, pre-labeling method and device and electronic equipment |
| WO2023244500A1 (en) * | 2022-06-16 | 2023-12-21 | Schaeffler Technologies AG & Co. KG | Method for defect detection for rolling elements |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111344553A (en) * | 2018-09-11 | 2020-06-26 | 合刃科技（深圳）有限公司 | Defect detection method and system for curved surface object |
| CN111951232A (en) * | 2020-07-24 | 2020-11-17 | 上海微亿智造科技有限公司 | Metal powder injection molding appearance defect detection method and system |
| CN112418155A (en) * | 2020-12-07 | 2021-02-26 | 成都川哈工机器人及智能装备产业技术研究院有限公司 | Method for detecting position and type of workpiece on subway car side inspection image |
-
2021
- 2021-04-19 CN CN202110418127.5A patent/CN113222901A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111344553A (en) * | 2018-09-11 | 2020-06-26 | 合刃科技（深圳）有限公司 | Defect detection method and system for curved surface object |
| CN111951232A (en) * | 2020-07-24 | 2020-11-17 | 上海微亿智造科技有限公司 | Metal powder injection molding appearance defect detection method and system |
| CN112418155A (en) * | 2020-12-07 | 2021-02-26 | 成都川哈工机器人及智能装备产业技术研究院有限公司 | Method for detecting position and type of workpiece on subway car side inspection image |
Non-Patent Citations (3)
| Title |
|---|
| ALEXEY BOCHKOVSKIY等: "YOLOv4: Optimal Speed and Accuracy of Object Detection", ARXIV:2024.10934V1, 23 April 2020 (2020-04-23) * |
| HAILI ZHAO等: "Detection of Metal Surface Defects Based on YOLOv4 Algorithm", JOURNAL OF PHYSICS: CONFERENCE SERIES, vol. 1907, 11 April 2021 (2021-04-11) * |
| 陈涛等: "基于BP神经网络的钢球表面缺陷识别", 机械工程师, no. 7, 10 July 2010 (2010-07-10), pages 56 - 57 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114495003A (en) * | 2022-01-24 | 2022-05-13 | 上海申视信科技有限公司 | People number identification and statistics method and system based on improved YOLOv3 network |
| WO2023244500A1 (en) * | 2022-06-16 | 2023-12-21 | Schaeffler Technologies AG & Co. KG | Method for defect detection for rolling elements |
| CN116229197A (en) * | 2022-11-24 | 2023-06-06 | 浙江大学 | Pre-labeling model construction method, pre-labeling method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111507990B (en) | Tunnel surface defect segmentation method based on deep learning | |
| CN111223088B (en) | Casting surface defect identification method based on deep convolutional neural network | |
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| CN113222901A (en) | Method for detecting surface defects of steel ball based on single stage | |
| CN113239930B (en) | Glass paper defect identification method, system, device and storage medium | |
| CN107123111B (en) | Deep residual error network construction method for mobile phone screen defect detection | |
| CN107966454A (en) | A kind of end plug defect detecting device and detection method based on FPGA | |
| CN109447979B (en) | Target detection method based on deep learning and image processing algorithm | |
| CN113643268B (en) | Industrial product defect quality inspection method and device based on deep learning and storage medium | |
| CN112037219A (en) | Metal surface defect detection method based on two-stage convolution neural network | |
| CN110473179A (en) | A kind of film surface defects detection method, system and equipment based on deep learning | |
| CN113177924A (en) | Industrial production line product flaw detection method | |
| CN114120317B (en) | Optical element surface damage identification method based on deep learning and image processing | |
| CN112270659A (en) | Rapid detection method and system for surface defects of pole piece of power battery | |
| CN112131982A (en) | Tea tree tender shoot identification method based on convolutional neural network | |
| CN116612106A (en) | Method for detecting surface defects of optical element based on YOLOX algorithm | |
| CN116071315A (en) | Product visual defect detection method and system based on machine vision | |
| NL2030315A (en) | Computer-implemented ai method for detecting surface defects of electroluminescent semiconductor panel | |
| CN117152735A (en) | Tomato maturity grading method based on improved yolov5s | |
| CN114549489A (en) | Carved lipstick quality inspection-oriented instance segmentation defect detection method | |
| CN115661681B (en) | Landslide hazard automatic identification method and system based on deep learning | |
| CN117611571A (en) | Strip steel surface defect detection method based on improved YOLO model | |
| CN117911399A (en) | Light-weight multi-scale aluminum profile surface defect detection method | |
| CN117830253A (en) | YOLOv 7-tiny-based rubber seal ring defect detection method | |
| CN115112509B (en) | Material surface indentation measurement method based on Mask R-CNN network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |