CN117408970A - Semantic segmentation-based method for polishing surface defects of medium plate by robot - Google Patents
Semantic segmentation-based method for polishing surface defects of medium plate by robot Download PDFInfo
- Publication number
- CN117408970A CN117408970A CN202311402492.2A CN202311402492A CN117408970A CN 117408970 A CN117408970 A CN 117408970A CN 202311402492 A CN202311402492 A CN 202311402492A CN 117408970 A CN117408970 A CN 117408970A
- Authority
- CN
- China
- Prior art keywords
- polishing
- medium plate
- defects
- defect
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 144
- 238000005498 polishing Methods 0.000 title claims abstract description 108
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000011218 segmentation Effects 0.000 title claims abstract description 49
- 238000004519 manufacturing process Methods 0.000 claims abstract description 35
- 238000001514 detection method Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 12
- 238000007517 polishing process Methods 0.000 claims abstract description 11
- 230000002708 enhancing effect Effects 0.000 claims abstract description 5
- 238000000227 grinding Methods 0.000 claims description 47
- 229910000831 Steel Inorganic materials 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 14
- 239000010959 steel Substances 0.000 claims description 14
- 244000137852 Petrea volubilis Species 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 8
- 238000005336 cracking Methods 0.000 claims description 7
- 206010039509 Scab Diseases 0.000 claims description 6
- 244000290333 Vanilla fragrans Species 0.000 claims description 6
- 235000009499 Vanilla fragrans Nutrition 0.000 claims description 6
- 235000012036 Vanilla tahitensis Nutrition 0.000 claims description 6
- 230000010485 coping Effects 0.000 claims description 5
- 238000007689 inspection Methods 0.000 claims description 5
- 239000000945 filler Substances 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 239000000463 material Substances 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 231100000241 scar Toxicity 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 description 8
- 230000004927 fusion Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 239000000428 dust Substances 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 238000005096 rolling process Methods 0.000 description 3
- 238000004378 air conditioning Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000001816 cooling Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000009413 insulation Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 239000003082 abrasive agent Substances 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005266 casting Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 235000019580 granularity Nutrition 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007494 plate polishing Methods 0.000 description 1
- 239000000843 powder Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000010926 purge Methods 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000037390 scarring Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000004381 surface treatment Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B24—GRINDING; POLISHING
- B24B—MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING; DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- B24B27/00—Other grinding machines or devices
- B24B27/033—Other grinding machines or devices for grinding a surface for cleaning purposes, e.g. for descaling or for grinding off flaws in the surface
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B24—GRINDING; POLISHING
- B24B—MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING; DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- B24B49/00—Measuring or gauging equipment for controlling the feed movement of the grinding tool or work; Arrangements of indicating or measuring equipment, e.g. for indicating the start of the grinding operation
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B24—GRINDING; POLISHING
- B24B—MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING; DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- B24B51/00—Arrangements for automatic control of a series of individual steps in grinding a workpiece
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Mechanical Engineering (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Finish Polishing, Edge Sharpening, And Grinding By Specific Grinding Devices (AREA)
Abstract
The invention provides a method for polishing surface defects of a medium plate by a robot based on semantic segmentation, belonging to the technical field of polishing processes, wherein the method comprises the following steps: constructing a medium plate surface defect segmentation data set and enhancing image data; improving and optimizing a YOLOv8 network model and training; deploying the improved YOLOv8 model on a production line; determining whether polishing treatment is carried out on the medium plate according to the segmentation result; the ABB robot is called by the upper computer to polish the defects; and setting the ABB robot procedure according to different defect types, and optimizing the polishing path. According to the invention, the YOLOv8 segmentation model is adopted to realize online real-time non-contact detection on the medium plate production line, the polishing process is used to improve the surface quality of the medium plate, the real-time requirement of the production line is met, and the automatic production efficiency is improved.
Description
Technical Field
The invention relates to the field of robot polishing, in particular to a method for polishing surface defects of a medium plate by a robot based on semantic segmentation.
Background
In the rolling process of the medium-thickness plate, due to the influence of factors such as raw material casting blanks, rolling equipment, rolling technology and the like, the surface of the plate is subjected to defects such as roll marks, skin tilting, transverse cracks, longitudinal cracks, scratches and the like in the production process, and the surface quality of the medium-thickness plate is greatly influenced. In order to reduce the huge loss caused by the quality problem of the plate as much as possible, the surface defects of the steel plate need to be detected, positioned and polished in time. The traditional manual polishing process needs personnel to be continuously carried out, long-time heavy physical labor can cause fatigue of the body of an operator, so that the working efficiency is reduced, consistency of polishing effect is difficult to ensure, and the operator can contact with a grinding tool, dust and the like in the polishing process, so that the body health of the operator is endangered.
With the development of the current industrial robot technology, the robot can carry out polishing work in a high-efficiency automatic mode, and the surface defects of the medium-thick plate can be continuously and accurately treated. The robot can achieve a more stable and consistent operation than manual sanding and is not affected by fatigue and distraction during long-term operation. Through preset polishing parameters and algorithms, the robot can keep the same polishing force and speed in each cycle, so that a consistent surface treatment effect is achieved.
Aiming at the problem of detecting and polishing the surface defects of the medium plate, a YOLOv8 network segmentation model with high detection precision and good real-time performance is adopted and is in contact with a robot polishing technology, the cooled medium plate on the medium plate production line is subjected to surface defect detection and polishing, different polishing technologies are set according to defect types, an optimal processing mode is selected, quality detection of the surface of the medium plate is accelerated, and automatic production efficiency is improved.
Disclosure of Invention
The invention provides a method for polishing the surface defects of a medium plate by a robot based on semantic segmentation, which aims to solve the problems of low defect detection precision and different polishing quality of the current medium plate production line. The invention comprises the following steps:
the invention provides a method for polishing surface defects of a medium plate by a robot based on semantic segmentation, which comprises the following steps:
step 1: constructing a medium plate surface defect segmentation data set and enhancing image data;
step 2: improving and optimizing a YOLOv8 network model and training;
step 3: deploying the improved YOLOv8 model on a production line;
step 4: determining whether polishing treatment is carried out on the medium plate according to the segmentation result;
step 5: the ABB robot is called by the upper computer to polish the defects;
step 6: and setting the ABB robot procedure according to different defect types, and optimizing the polishing path.
Further, constructing a medium plate surface defect segmentation data set and enhancing image data, comprising:
acquiring image data of the surface defects of the medium plate by using a Dalsa series CameraLink camera, arranging an image acquisition cabinet on a production line, acquiring the image data of the surface of the medium plate and completing splicing to obtain an original data set;
preprocessing the acquired pictures, linearly interpolating different samples by using a MixUp method to generate new samples, and completing the first-step image data enhancement on the data set;
generating a new image with a plurality of repeated targets by using a CopyPaste method, and completing second-step image data enhancement on the data set;
splicing a plurality of random images by using a mosaics method to generate Mosaic images, and completing the third step of image data enhancement on the data set;
labeling the processed data set by using LabelImg labeling software to finish the manufacturing of the cut data set of the surface defects of the medium plate.
Further, improving and optimizing the YOLOv8 network model and training comprises the following steps:
aiming at the surface defect characteristics of the medium plate, improving a backbone (basic backbone network) by referring to FasterViT, inceptionNext and Conv2Former backbone networks; the EMA attention mechanism, the RFAConv attention mechanism and the SeaFormer attention module are introduced and compared, so that the detection precision of the YOLOv8 is improved, the delay is reduced, and the detection efficiency is improved; the model is improved in light weight by replacing a Bottleneck module in a C2f framework with a Vanilla block module and introducing a DCN module to reduce the number of parameters.
Further, deploying the improved YOLOv8 model onto a production line, comprising:
the improved YOLOv8 network model is deployed on a medium plate production line, and the cooled medium plate is subjected to real-time high-precision surface defect segmentation and positioning by combining a linear array CCD and a depth camera.
Further, determining whether to polish the medium plate according to the segmentation result includes:
the defect types on the medium plate picture acquired by the linear array CCD are mainly divided into: pitting, scabbing, cracking, roll marks, scratches, pinholes, scales and holes, and when detecting that the defect picture contains pitting, scabbing, cracking and roll marks, the upper computer is subjected to polishing treatment;
when scratches, pinholes, scales and holes are detected, the proportion of defects on the defect picture is determined according to the segmentation result, and signals are given out to polish the medium plate when the proportion exceeds a certain proportion.
Further, invoking the ABB robot to polish the defects through the upper computer, wherein the method comprises the following steps:
the polishing of the defects of the medium plate is carried out by adopting an advanced side inspection and polishing method and a flexible polishing method. The polishing method of the edge inspection and polishing is adopted to detect the defects after primary polishing in real time, if the defects do not meet the quality requirements, the ABB robot performs polishing again, the polishing quality is ensured, the defects are corrected in time, and the production efficiency is improved; the flexible coping adopts different grinding tools and different grinding paths to grind different defect types, and defects are effectively eliminated through accurate classification and targeted coping;
aiming at the defects of different types of medium plates, different polishing equipment and polishing processes are adopted, so that more efficient polishing of the defects of the steel plates is realized; for harder steels, a greater grinding force and less feed are required for grinding; for softer steels, the grinding force can be properly reduced and the feed rate can be increased to avoid excessive wear; for larger defects, larger polishing force and smaller feeding amount are needed to grind, so that the defect removal speed is increased; while for smaller defects, the sharpening force is suitably reduced and the feed is increased to avoid excessive wear.
Further, setting ABB robot procedures for different defect types, optimizing a polishing path, including:
the scar defect, the ABB robot sets a large feeding amount, and a hard grinding wheel is used for grinding to rapidly remove the high point of the material;
shallow cracks, pitting surfaces and roll marks, the ABB robot sets small feeding amount, uses a thicker grinding wheel or sand paper to grind the surface of a defect part flat, and then uses a thinner grinding wheel or sand paper to polish;
cracks, pitting surfaces and roll marks with larger depth are filled with filler, and then are polished to be smooth by a grinding wheel or sand paper;
shallow scratches, pinholes, scales and holes can be used for edge repair, and then the defect surface is ground and polished by using small feeding amount;
after the first polishing of the defect is finished, the ABB robot automatically adjusts the pose, photographs the polishing part, carries out detection of the defect polishing effect by a YOLOv8 segmentation model carried by an upper computer, and polishes the defect which does not meet the requirement again.
Compared with the prior art, the invention has the following advantages and beneficial effects:
according to the method for polishing the surface defects of the medium plate by the robot based on semantic segmentation, the YOLOv8 network segmentation model is combined with the industrial robot polishing technology, and compared with the existing detection polishing method, the detection accuracy is higher, the instantaneity is better, and the polishing quality is higher;
according to the invention, a Vanilla block module and a DCN module are used for completing the light-weight design of the segmentation model, so that the problem of limitation of the fixation of the receptive field of the traditional convolution network is solved;
according to the method, the ABB robot working procedures are set according to different defect types, the polishing path is optimized, different grinding tools are adopted according to different defects, and different polishing working procedures are adopted according to different characteristics of similar defects;
the method adopts the advanced method of checking and grinding at the same time and flexibly coping to grind the defects of the medium plate, and adds the YOLOv8 segmentation model image recognition technology in the grinding process to test the grinding quality in real time, thereby improving the automatic production efficiency.
Drawings
In order to more clearly describe the technical solution and the optimization features of the present invention, the following description will further illustrate the present invention with reference to the accompanying drawings and specific examples, so that those skilled in the art can better understand the present invention and implement it, but the examples are not limited thereto.
FIG. 1 is a schematic diagram of an overall logic flow of a method for polishing surface defects of a medium plate by a robot based on semantic segmentation according to an embodiment of the present invention;
FIG. 2 is a diagram of a basic architecture of a YOLOv8 network segmentation model used in an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of Conv module, detect module, SPPF module, and C2f module according to an embodiment of the present invention;
FIG. 4 is a view showing the overall arrangement of a medium plate production line in the embodiment of the invention;
FIG. 5 is a schematic diagram of a cross section of image acquisition of a surface defect of a medium plate in an image acquisition cabinet according to an embodiment of the present invention;
in the figure: 10. long Menzhu; 20. ABB robot; 30. ABB robot control cabinet; 40. a depth camera; 50. a medium plate; 60. an image acquisition cabinet; 70. a ground rail; 80. grinding tool and grinding tool frame; 90. a roller way; 100. an air cooling machine; 110. a server cabinet; 120. an electric control cabinet; 130. an LED light source; 140. an air conditioning system; 150. a line camera; 160. a heat insulating glass; 170. a purging mechanism.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention provides a method for polishing surface defects of a medium plate by a robot based on semantic segmentation, which is shown in a figure 1, and is an overall logic flow diagram of the method for polishing surface defects of the medium plate by the robot based on semantic segmentation, and comprises the following steps:
step 1: constructing a medium plate surface defect segmentation data set and enhancing image data;
step 2: improving and optimizing a YOLOv8 network model and training;
step 3: deploying the improved YOLOv8 model on a production line;
step 4: determining whether polishing treatment is carried out on the medium plate according to the segmentation result;
step 5: the ABB robot is called by the upper computer to polish the defects;
step 6: and setting the ABB robot procedure according to different defect types, and optimizing the polishing path.
Based on the above embodiment, step 1 includes the steps of:
step 1.1: acquiring image data of the surface defects of the medium plate by using a Dalsa series CameraLink camera, arranging an image acquisition cabinet on a production line, acquiring the image data of the surface of the medium plate and completing splicing to obtain an original data set;
step 1.2: preprocessing the acquired pictures, linearly interpolating different samples by using a MixUp method to generate new samples, and completing the first-step image data enhancement on the data set;
step 1.3: generating a new image with a plurality of repeated targets by using a CopyPaste method, and completing second-step image data enhancement on the data set;
step 1.4: splicing a plurality of random images by using a mosaics method to generate Mosaic images, and completing the third step of image data enhancement on the data set;
step 1.5: labeling the processed data set by using LabelImg labeling software to finish the manufacturing of the cut data set of the surface defects of the medium plate.
The cameras commonly used on the steel plate production line comprise an area-array camera and a linear-array camera, and compared with the two cameras, the linear-array camera adopts a continuous scanning mode, can obtain higher image resolution and definition in a scene of high-speed movement, and is more suitable for defect detection and size measurement on the medium and heavy plate production line. As shown in fig. 5, the line camera 150 adopts a Dalsa series camera link camera, the number of line cameras is determined according to the width of the production line and the optimal field of view of the line camera, the LED light sources 130 are arranged in the image acquisition cabinet 60 side by side, the air conditioning system 140 and the heat insulation glass 160 reduce the influence of the external temperature of the production line, and the blowing mechanism 170 prevents the heat insulation glass 160 from generating mist and cleaning dust and sundries on the surface, so as to ensure the quality of the acquired image on the surface of the medium plate. The defect categories of the collected initial pictures are mainly divided into: pitting, scarring, cracking, roll marks, scratches, pinholes, scale, and holes.
In the training process, two different samples are subjected to linear interpolation by using a MixUp method, linear interpolation is performed between an input sample and a corresponding label, new input and labels are generated, and the first-step image data enhancement is completed. The linear interpolation process calculates their weighted sum by randomly selecting two samples. The MixUp method can reduce the overfitting to a certain extent and enhance the generalization capability of the model.
The second step of image data enhancement is accomplished by selecting one or more target areas from one image using the CopyPaste method and copying and pasting them to other locations on the same image, creating a new image with multiple repeated targets. The CopyPaste method can enhance the number and diversity of targets and improve the robustness of the model to the target positions and the background.
And randomly selecting a plurality of images by using a mosaics method, splicing the images into a new Mosaic image, and inputting the Mosaic image into a model as a training sample to complete the third-step image data enhancement. Through the Mosaic operation, the model can contact target areas of a plurality of different images, so that the diversity and complexity of data are increased, and the robustness of the model to problems such as target boundaries, background interference and scale change is improved.
Based on the above embodiment, step 2 includes the steps of:
step 2.1: selecting a particular segmentation network model of a YOLOv8 series algorithm according to the parameter size of the processed segmentation data set, and determining a training strategy;
step 2.2: setting an optimizer as SGD, a basic learning rate base learning rate as 0.01, a basic weight attenuation base weight decay as 0.005, an optimizer momentum optimizer momentum as 0.937, a batch size as 128, a learning rate schedule learning rate schedule as linear, a training iteration number of training epochs as 500, a preheating iteration number warmup iterations as max (1000,3 ×iters_per_epochs), an input size as 640×640, an index moving average attenuation EMA decay as 0.9999 and loading initial segmentation weights to train a data set;
step 2.3: aiming at the surface defect characteristics of the medium plate, improving a backbone (basic backbone network) by referring to FasterViT, inceptionNext and Conv2Former backbone networks; the EMA attention mechanism, the RFAConv attention mechanism and the SeaFormer attention module are introduced and compared, so that the detection precision of the YOLOv8 is improved, the delay is reduced, and the detection efficiency is improved; the model is improved in light weight by replacing a Bottleneck module in a C2f framework with a Vanilla block module and introducing a DCN module to reduce the number of parameters.
The network model of the Yolov8 series algorithm is divided into YOLOv8-n, YOLOv8-s, YOLOv8-m, YOLOv8-l and YOLOv8-x according to the parameter quantity from small to large, initial weights are sequentially arranged according to the tasks of detection and segmentation, the segmentation parameters are detected according to the multi-category surface defects of the medium plate, and the training speed is accelerated by adopting the YOLOv8-l model and loading the corresponding initial weights of YOLOv8 l-seg.
As shown in fig. 2, the YOLOv8 basic model mainly includes: input, basic Backbone network (Backbone), feature fusion layer (Neck), decoupling header (Head);
input: RGB three-channel pictures with data set size 640 x 640;
basic backbone network: the feature extraction network refers to an original Darknet-53 network and is improved to a certain extent, and is used for extracting picture information of an input data set, and a Conv convolution module and a Residual Block Residual module are overlapped four times in series, and a convolution module and a Residual module are combined to be called a stage. As shown in fig. 3, the Conv convolution module consists of three parts: the residual error module uses a C2f module, obtains more gradient flow information while ensuring light weight, and finally adds an SPPF module, wherein the SPPF module comprises a Conv convolution module, three Maxpooling modules and is formed by performing concat splicing on feature maps which are not subjected to Maxpooling and feature maps obtained after carrying out Maxpooling once more;
feature fusion layer: and carrying out feature fusion on the extracted picture information, designing a PAFPN feature extraction framework by referring to a feature pyramid FPN, and transmitting high-level strong semantic features from top to bottom by FPN. PAN is to add a bottom-up pyramid behind the FPN, supplement the FPN, and transfer the low-level strong localization features. The up-sampling is firstly carried out, then the down-sampling is carried out, and two cross-layer fusion connections are arranged between the two branches of the up-sampling and the down-sampling;
decoupling head: as shown in fig. 3, the classification is separated from the detection Head using the current mainstream decoupling Head structure (coupled-Head).
The FasterViT backbone network is a target detection network combining Faster R-CNN and Vision Transformer (ViT), and ViT is used as a backbone network to extract features from an input image using a self-attention mechanism and perform target detection by a detection head of Faster R-CNN. The InceptionNext backbone network is a convolutional neural network structure which is expanded and improved on the basis of the Inception network, multi-scale characteristic branches are introduced, characteristics are extracted under different scales through parallel convolutional branches, and richer characteristic representations are obtained through residual connection and layer-by-layer aggregation. Conv2Former backbone network is a convolutional neural network structure that replaces convolutional layers with a transducer module, converting convolutional operations into a self-attention mechanism of the transducer for extracting and encoding image features. The method is characterized in that the backspace of the YOLOv8 model is improved by referring to the network aiming at the surface defect characteristics of the medium plate, a transducer module is introduced, the lightweight design of the model is completed, richer characteristic representations are obtained, and the response speed of the system is accelerated.
EMA attention mechanism (Exponential Moving Average Attention) is a variant that introduces an Exponential Moving Average (EMA) in the self-attention mechanism. In EMA attention, in addition to using the similarity between query-key-values (QKV), the moving average of the attention distribution of previous time steps is introduced as an additional attention score, which can make the model better memorize and use the history information, thereby improving the modeling ability of long-term dependencies. RFAConv introduces query-key-values (QKV) and position coding in the self-attention mechanism, and also contains a convolution operation for weighting the aggregate features according to the attention profile, which can adaptively adjust the feature representation while preserving spatial information. SeaFormer is an attention module integrating self-attention and position attention, and utilizes position feature coding and position attention distribution to perform weighted fusion on local features and global information so as to enhance the expression capability of the features. The modules are inserted into a model architecture of the YOLOv8 network and replaced with partial Conv modules in the model architecture, so that the space characteristics of the receptive field are more concentrated, the operation amount is reduced, and the detection efficiency is higher.
The Vanilla Block module is a very simple module consisting of a series of convolution layers (typically 3x3 size convolution kernels) and activation functions, which is easy to implement and train without additional complexity. The DCN module introduces deformable convolution operation, and has stronger perceptibility by performing position translation and deformation on convolution kernel weights in the convolution process. The Vanilla block module and the DCN module are used for completing the lightweight design of the model, and the problem of limitation of the fixation of the receptive field of the traditional convolution network is solved.
Based on the above embodiment, step 3 includes the steps of:
as shown in fig. 4 and fig. 5, an improved YOLOv8 network model is deployed on a medium plate production line, the medium plate 50 is placed on a roller way 90, the roller way is controlled to run and stop by a motor, ground rails 70 and Long Menzhu are installed on two sides of the roller way 90, a gantry column 10 can move back and forth along the roller way direction by utilizing the ground rails 70, an ABB robot 20 is installed on the gantry column 10, the medium plate 50 is polished, grinding tools and grinding tool frames 80 are arranged on two sides of the medium plate 50, the grinding tools are automatically replaced according to a preset program by the ABB robot 20 of a defect type, a technician can change the ABB robot program through the ABB robot control cabinet 30 to meet the requirement of the production line, the influence of the surrounding environment of the production line on detection is reduced by the air cooling machine 100, the server cabinet 110 is installed to meet the requirement of real-time detection on the calculation force of the server station, the electric control cabinet 120 realizes unified switch on the production line, and emergency shutdown can be realized to ensure the safety of personnel and equipment. The defect positioning of the medium plate 50 is realized through the image acquisition cabinet 60, and then the real-time high-precision dividing positioning and polishing of the surface defect of the medium plate are realized through the polishing treatment of the two-wheel ABB robot 20.
Based on the above embodiment, step 4 includes the steps of:
step 4.1: the defect categories on the medium plate picture collected by the line camera 150 are mainly divided into: pitting, scabbing, cracking, roll marks, scratches, pinholes, scales and holes, and when detecting that the defect picture contains pitting, scabbing, cracking and roll marks, the upper computer is subjected to polishing treatment;
step 4.2: when scratches, pinholes, scales and holes are detected, the proportion of defects on the defect picture is determined according to the segmentation result, and signals are given out to polish the medium plate when the proportion exceeds a certain proportion.
Based on the above embodiment, step 5 includes the steps of:
step 5.1: and polishing the defects of the medium plate by adopting an advanced method of checking and polishing and flexible polishing. The polishing method of polishing while checking is adopted to detect the defect after primary polishing in real time, if the defect does not meet the quality requirement, the ABB robot 20 performs polishing again, thereby ensuring the polishing quality, correcting the defect in time and improving the production efficiency; the defects are flexibly polished by classifying different defects, polishing the defects by adopting different cutters and different polishing paths, and effectively eliminating the defects by accurately classifying and pointedly polishing;
step 5.2: different polishing equipment and polishing processes are adopted for defects of different types of medium plates, so that more effective and more efficient polishing of the defects of the steel plates is realized. For harder steels, a greater grinding force and less feed are required for grinding; for softer steels, the grinding force can be properly reduced and the feed rate can be increased to avoid excessive wear; for larger defects, larger polishing force and smaller feeding amount are needed to grind, so that the defect removal speed is increased; while for smaller defects, the sharpening force is suitably reduced and the feed is increased to avoid excessive wear.
The steel plate polishing equipment adopts an ABB robot 20, the model IRB 6700-200/2.60, the tail end of the ABB robot 20 is connected with a flexible force position compensator for maintaining the polishing force constant, and the force position compensator is connected with electric polishing equipment. The ABB robot 20 is provided with a gantry column 10 and a sliding table device, the gantry column 10 can move along the roller way direction to drive the ABB robot 20 to move back and forth, and the beam sliding table can meet the requirement of the ABB robot 20 on moving left and right. When the medium plate 50 is transported to the photoelectric switch position, the roller way 90 stops moving, a workpiece coordinate system is established by using one corner point of the contact position of the medium plate 50 and the photoelectric switch, and the conversion relation with the ABB robot coordinate system is determined. The apparatus is equipped with a dust suction device for collecting the powder produced during grinding. ABB robot 20 is equipped with an eye-on-hand depth camera 40 for specific localization of defects and defect grinding quality inspection after grinding to determine if regrinding is required. The calibration of the ABB robot 20 and the depth camera 40 adopts a Zhang Zhengyou calibration method, so that the calibration accuracy is ensured to be within 0.01 mm. The ABB robot 20 is equipped with an automated control system, and a more accurate and stable polishing process is realized by intelligently adjusting parameters of the ABB robot 20. The automatic control system researches influences of different defects, different polishing force parameters and different abrasives on polishing effects in advance through polishing control, and designs an optimal polishing procedure aiming at different defects; dynamic force position polishing is achieved through researches on a closed-loop control algorithm, a compound control algorithm, an accurate control algorithm and an information fusion algorithm, and polishing quality is ensured. The automatic control system can automatically adjust parameters such as polishing force, feeding amount, grinding wheel speed and the like according to different defect types and sizes so as to improve polishing effect to the greatest extent and reduce errors and labor intensity of manual operation.
Based on the above embodiment, step 6 includes the steps of:
the scar defect, ABB robot 20 sets a large feed, uses a hard grinding wheel to grind to quickly remove the high points of material;
shallow cracks, pitting surfaces and roll marks, the ABB robot 20 sets a small feed amount, uses a thicker grinding wheel or sand paper to flatten the surface of the defect, and then uses a thinner grinding wheel or sand paper to polish;
cracks, pitting surfaces and roll marks with larger depth are filled with filler, and then are polished to be smooth by a grinding wheel or sand paper;
shallow scratches, pinholes, scales and holes can be used for edge repair, and then the defect surface is ground and polished by using small feeding amount;
after the first polishing of the defect is completed, the ABB robot 20 automatically adjusts the pose, photographs the polished part, detects the polishing effect of the defect by using the YOLOv8 segmentation model carried by the upper computer, and polishes the defect which does not meet the requirement again.
According to the surface defect condition of the steel plate, polishing parameters including polishing force, speed, track and other factors are reasonably set. The grinding effect is detected and verified in real time by additionally arranging depth cameras 40 on the two ABB robots 20 as shown in figure 4, and the grinding accuracy and consistency are ensured. And sand paper or grinding heads with different granularities are selected according to the size and shape of the defects, meanwhile, sequential procedures of rough grinding and fine grinding are set, and the grinding quality is ensured through detection verification of the two-wheel ABB robot 20 on a production line. And setting polishing sequence according to the size and the position of the defect. Typically, the polishing is gradually performed from larger defects to smaller defects, ensuring that the defects are effectively removed and that the surface finish of the medium plate is uniform.
In summary, after the scheme is adopted, the invention provides a new method for polishing the surface defects of the medium plate by the robot, combines the industrial robot polishing technology and the YOLOv8 network segmentation model, introduces an attention mechanism module, improves the backbone network backup to a certain extent, realizes the light improvement of the model, accelerates the system response speed, improves the detection precision, implements different polishing processes for different defect types, utilizes an automatic control system to adjust polishing parameters, effectively promotes the development of the robot polishing steel plate surface defect technology, and has practical application value.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (7)
1. The method for polishing the surface defect of the medium plate by the robot based on semantic segmentation is characterized by comprising the following steps of:
step 1: constructing a medium plate surface defect segmentation data set and enhancing image data;
step 2: improving and optimizing the YOLOv8 network and training;
step 3: deploying the improved YOLOv8 model on a production line;
step 4: determining whether polishing treatment is carried out on the medium plate according to the segmentation result;
step 5: the ABB robot is called by the upper computer to polish the defects;
step 6: and setting the ABB robot procedure according to different defect types, and optimizing the polishing path.
2. The method for polishing surface defects of a medium plate by a robot based on semantic segmentation according to claim 1, wherein constructing a medium plate surface defect segmentation dataset and performing image data enhancement comprises:
acquiring image data of the surface defects of the medium plate by using a Dalsa series CameraLink camera, arranging an image acquisition cabinet on a production line, acquiring the image data of the surface of the medium plate and completing splicing to obtain an original data set;
preprocessing the acquired pictures, linearly interpolating different samples by using a MixUp method to generate new samples, and completing the first-step image data enhancement on the data set;
generating a new image with a plurality of repeated targets by using a CopyPaste method, and completing second-step image data enhancement on the data set;
splicing a plurality of random images by using a mosaics method to generate Mosaic images, and completing the third step of image data enhancement on the data set;
labeling the processed data set by using LabelImg labeling software to finish the manufacturing of the cut data set of the surface defects of the medium plate.
3. The method for polishing surface defects of a medium plate by a semantic segmentation-based robot according to claim 1, wherein the method for improving and optimizing the YOLOv8 network model and training comprises the following steps:
aiming at the surface defect characteristics of the medium plate, improving a backbone (basic backbone network) by referring to FasterViT, inceptionNext and Conv2Former backbone networks; the EMA attention mechanism, the RFAConv attention mechanism and the SeaFormer attention module are introduced and compared, so that the detection precision of the YOLOv8 is improved, the delay is reduced, and the detection efficiency is improved; the model is improved in light weight by replacing a Bottleneck module in a C2f framework with a Vanilla block module and introducing a DCN module to reduce the number of parameters.
4. The method for polishing surface defects of a medium plate by a semantic segmentation-based robot according to claim 1, wherein the deploying of the improved YOLOv8 model on a production line comprises:
the improved YOLOv8 network model is deployed on a medium plate production line, and the cooled medium plate is subjected to real-time high-precision surface defect segmentation and positioning by combining a linear array CCD and a depth camera.
5. The method for polishing surface defects of a medium plate by a robot based on semantic segmentation according to claim 1, wherein determining whether to polish the medium plate according to the segmentation result comprises:
the defect types on the medium plate picture acquired by the linear array CCD are mainly divided into: pitting, scabbing, cracking, roll marks, scratches, pinholes, scales and holes, and when detecting that the defect picture contains pitting, scabbing, cracking and roll marks, the upper computer is subjected to polishing treatment;
when scratches, pinholes, scales and holes are detected, the proportion of defects on the defect picture is determined according to the segmentation result, and signals are given out to polish the medium plate when the proportion exceeds a certain proportion.
6. The method for polishing the surface defects of the medium plate by the robot based on semantic segmentation according to claim 1, wherein the polishing process of the defects by calling the ABB robot through the upper computer comprises the following steps:
the polishing of the defects of the medium plate is carried out by adopting an advanced side inspection and polishing method and a flexible polishing method. The polishing method of the edge inspection and polishing is adopted to detect the defects after primary polishing in real time, if the defects do not meet the quality requirements, the ABB robot performs polishing again, the polishing quality is ensured, the defects are corrected in time, and the production efficiency is improved; the flexible coping adopts different grinding tools and different grinding paths to grind different defect types, and defects are effectively eliminated through accurate classification and targeted coping;
different polishing equipment and polishing processes are adopted for defects of different types of medium plates, so that more efficient polishing of the defects of the steel plates is realized. For harder steels, a greater grinding force and less feed are required for grinding; for softer steels, the grinding force can be properly reduced and the feed rate can be increased to avoid excessive wear; for larger defects, larger polishing force and smaller feeding amount are needed to grind, so that the defect removal speed is increased; while for smaller defects, the sharpening force is suitably reduced and the feed is increased to avoid excessive wear.
7. The method for polishing surface defects of a medium plate by a robot based on semantic segmentation according to claim 1, wherein the ABB robot procedure is set for different defect types, and the polishing path is optimized, comprising:
the scar defect, the ABB robot sets a large feeding amount, and a hard grinding wheel is used for grinding to rapidly remove the high point of the material;
shallow cracks, pitting surfaces and roll marks, the ABB robot sets small feeding amount, uses a thicker grinding wheel or sand paper to grind the surface of a defect part flat, and then uses a thinner grinding wheel or sand paper to polish;
cracks, pitting surfaces and roll marks with larger depth are filled with filler, and then are polished to be smooth by a grinding wheel or sand paper; shallow scratches, pinholes, scales and holes can be used for edge repair, and then the defect surface is ground and polished by using small feeding amount;
after the first polishing of the defect is completed, the ABB robot automatically adjusts the pose, photographs the polishing part, carries an improved YOLOv8 segmentation model by an upper computer to detect the polishing effect of the defect, and polishes the defect which does not meet the requirement again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311402492.2A CN117408970A (en) | 2023-10-27 | 2023-10-27 | Semantic segmentation-based method for polishing surface defects of medium plate by robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311402492.2A CN117408970A (en) | 2023-10-27 | 2023-10-27 | Semantic segmentation-based method for polishing surface defects of medium plate by robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117408970A true CN117408970A (en) | 2024-01-16 |
Family
ID=89498601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311402492.2A Pending CN117408970A (en) | 2023-10-27 | 2023-10-27 | Semantic segmentation-based method for polishing surface defects of medium plate by robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117408970A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117953350A (en) * | 2024-03-26 | 2024-04-30 | 杭州豪悦护理用品股份有限公司 | Diaper image detection method based on improved YOLOv network model |
-
2023
- 2023-10-27 CN CN202311402492.2A patent/CN117408970A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117953350A (en) * | 2024-03-26 | 2024-04-30 | 杭州豪悦护理用品股份有限公司 | Diaper image detection method based on improved YOLOv network model |
| CN117953350B (en) * | 2024-03-26 | 2024-06-11 | 杭州豪悦护理用品股份有限公司 | Diaper image detection method based on improved YOLOv network model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210073975A1 (en) | Method for enhancing optical feature of workpiece, method for enhancing optical feature of workpiece through deep learning, and non transitory computer readable recording medium | |
| CN111007073A (en) | Method and system for online detection of part defects in additive manufacturing process | |
| CN107578464A (en) | A kind of conveyor belt workpieces measuring three-dimensional profile method based on line laser structured light | |
| CN117408970A (en) | Semantic segmentation-based method for polishing surface defects of medium plate by robot | |
| CN100486476C (en) | Method and system for automatic generating shoe sole photopolymer coating track based on linear structure optical sensor | |
| CN110153567A (en) | A kind of laser cutting system based on image recognition | |
| CN107966454A (en) | A kind of end plug defect detecting device and detection method based on FPGA | |
| EP0429616A1 (en) | Optical robotic canopy polishing system | |
| CN107230203A (en) | Casting defect recognition methods based on human eye vision attention mechanism | |
| CN111126174A (en) | Visual detection method for robot to grab parts | |
| CN108489986A (en) | A kind of increasing material manufacturing on-line checking and restorative procedure | |
| CN112529858A (en) | Welding seam image processing method based on machine vision | |
| CN111266304A (en) | Coal briquette identification, detection and sorting system | |
| CN113012149B (en) | Intelligent cleaning robot path planning method and system | |
| CN110633738B (en) | Rapid classification method for industrial part images | |
| CN113146172A (en) | Multi-vision-based detection and assembly system and method | |
| Anvar et al. | Automatic metallic surface defect detection using shuffledefectnet | |
| CN116309275A (en) | Method and device for detecting edges of sub-pixels of battery piece image and storage medium | |
| CN112530267B (en) | Intelligent mechanical arm teaching method based on computer vision and application | |
| CN113763462A (en) | Method and system for automatically controlling feeding | |
| CN117173151B (en) | Visual identification device, method and system for defects of outer surface of long steel | |
| CN116797569A (en) | Wafer defect detection method and system | |
| CN117169247A (en) | Metal surface defect multi-dimensional detection method and system based on machine vision | |
| CN114926531A (en) | Binocular vision based method and system for autonomously positioning welding line of workpiece under large visual field | |
| CN111028231A (en) | Workpiece position acquisition system based on ARM and FPGA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |