CN114677334A - Method, system and device for controlling surface quality of special-shaped blank - Google Patents
Method, system and device for controlling surface quality of special-shaped blank Download PDFInfo
- Publication number
- CN114677334A CN114677334A CN202210214912.3A CN202210214912A CN114677334A CN 114677334 A CN114677334 A CN 114677334A CN 202210214912 A CN202210214912 A CN 202210214912A CN 114677334 A CN114677334 A CN 114677334A
- Authority
- CN
- China
- Prior art keywords
- image
- defect
- convolution
- blank
- special
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000007547 defect Effects 0.000 claims abstract description 214
- 238000001514 detection method Methods 0.000 claims abstract description 54
- 238000005259 measurement Methods 0.000 claims abstract description 19
- 238000005507 spraying Methods 0.000 claims description 41
- 238000001914 filtration Methods 0.000 claims description 24
- 238000003384 imaging method Methods 0.000 claims description 19
- 238000000227 grinding Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 9
- 230000001629 suppression Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000005498 polishing Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 230000005764 inhibitory process Effects 0.000 description 3
- 238000003908 quality control method Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000010485 coping Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002893 slag Substances 0.000 description 2
- 238000003466 welding Methods 0.000 description 2
- 206010039509 Scab Diseases 0.000 description 1
- 238000012300 Sequence Analysis Methods 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000053 physical method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- -1 welding marks Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a method, a system and a device for controlling the surface quality of a beam blank, wherein the method comprises the steps of acquiring a first image for measuring the surface size of the beam blank and a second image for detecting the surface defects of the beam blank; extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as input of a first packet convolution network, setting convolution layers according to the defect category number, and extracting defect feature graph groups of corresponding categories; finally, inputting the first image into a second grouped convolution network, and classifying the output defect feature map group by adopting the grouped convolution network to obtain defect type information contained in the second image; and finally obtaining the surface quality detection result of the special-shaped blank. Based on the method, the system and the device for controlling the surface quality of the beam blank are also provided. The invention realizes the fusion of line structure light and depth learning algorithm based on the dual-channel image, and simultaneously realizes the physical dimension measurement and surface defect detection of the special-shaped blank.
Description
Technical Field
The invention belongs to the technical field of quality detection of a beam blank, and particularly relates to a beam blank surface quality control method, a beam blank surface quality control system and a beam blank surface quality control device.
Background
With rapid development of economy and deep advance of supply-side reform, demand for high-quality steel is increasing in various fields such as automobiles and real estate. The special-shaped blank is an economic section efficient section with optimized section area distribution and reasonable strength-weight ratio, and is widely applied to the fields of high-rise buildings, ocean platforms, power equipment and the like. Various defects are inevitably generated in the production process of the special-shaped blank, the physical dimension measurement of the special-shaped blank is indispensable, the requirements on the special-shaped blank demand and the quality are continuously improved along with the remarkable improvement of the living standard of people, and the physical dimension measurement and the surface quality detection and control of the special-shaped blank are increasingly important.
Because the types of the special-shaped blanks are numerous, the size measurement parameters and the surface defect conditions are complex, and in the aspect of size measurement, the manual measurement is mainly performed by a measurer through a physical measurement tool such as a three-coordinate system at present; in the aspect of surface defect detection, a quality inspector only observes the surface of the special-shaped blank by naked eyes to search defects, and the method has high subjectivity. No matter the measurement of physical dimension, or the detection of surface defect, manual operation all has and detects slowly, and the precision is low, causes the lack of defect to examine easily and leaks to examine, and operating personnel danger coefficient is high simultaneously. With the great improvement of the computing power of a computer and the deep research of computer vision theory and method, the machine vision technology can be used for landing roots, blooming and fruiting in various industry fields at present, and the effect is obvious. The prior art provides various defect detection methods for the surface of the beam blank and provides a detection device for practical application, can realize online and offline continuous detection, classification and recording of surface defects, and has important practical significance for improving production efficiency and product quality, reducing personnel and improving efficiency of enterprises and improving core competitiveness of products. On the other hand, however, the technical means disclosed in the prior art are limited to the detection of surface defects of the parisons, and lack of corresponding control measures for the detected defects, as well as lack of physical dimensional measurements of the parisons.
Disclosure of Invention
In order to solve the technical problem, the invention provides a method, a system and a device for controlling the surface quality of a beam blank. Based on the fusion of the line structured light and the depth learning algorithm of the dual-channel image, the physical dimension measurement and the surface defect detection of the special-shaped blank are realized at the same time.
In order to realize the purpose, the invention adopts the following technical scheme:
a method for controlling the surface quality of a beam blank comprises the following steps:
acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank;
extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as the input of a first packet convolution network, setting convolution layers according to the number of defect types, and extracting a defect feature graph set of the corresponding type; finally, inputting the first image into a second grouped convolution network, and classifying the output defect feature map group by adopting the grouped convolution network to obtain defect type information contained in the second image;
and calculating the boundary box information of the defects according to the defect types, and then filtering the boundary box information output by the boundary box regression network to obtain the final detection result of the surface quality of the beam blank.
Further, the method further comprises the steps of carrying out code spraying and marking on the special-shaped blank according to the final special-shaped blank surface quality detection result, and carrying out corresponding coping on the special-shaped blank surface at the position where the code spraying and marking are carried out on the special-shaped blank.
Further, the method for acquiring the first image for measuring the surface dimension of the beam blank and acquiring the second image for detecting the surface defect of the beam blank comprises the following steps:
acquiring a first image for measuring the surface size of the special-shaped blank through an infrared channel and a near-infrared channel in an imaging module;
and acquiring a second image for detecting the surface defects of the special-shaped blank through a blue light channel in the imaging module.
Further, the process of taking the image including the common features as an input of the first packet convolution network, setting convolution layers according to the number of defect categories, and extracting the defect feature map set of the corresponding category includes:
setting the convolution layers according to the defect type number, wherein the defect type number is equal to the number of the convolution layers, so that each convolution layer can detect one type of defect;
and extracting each type of defect by adopting a preset number of convolution kernels, wherein each group of convolution layers are respectively independent and carry out convolution calculation with the image comprising the common features, and extracting a defect feature map group of a corresponding category.
Further, when the defect feature classification is performed, the same classifier respectively calculates the defect feature map groups of the corresponding classes to judge whether the defect type represented by the current feature map group appears in the second image.
Further, the step of calculating the information of the boundary frame of the defect according to the defect type, and then filtering the information of the boundary frame output by the regression network of the boundary frame to obtain the final detection result of the surface quality of the beam blank comprises: and selecting a corresponding characteristic diagram group according to the defect type, inputting the characteristic diagram group into a boundary frame regression network to calculate the boundary frame information of the defect, and filtering the output result of the boundary frame regression network by a non-maximum inhibition method to obtain the final surface quality detection result of the beam blank.
The invention also provides a system for controlling the surface quality of the beam blank, which comprises an acquisition module, an extraction module and a filtering module;
the acquisition module is used for acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank;
the extraction module extracts common features in the second image through a shared convolution network, then takes the image comprising the common features as the input of a first packet convolution network, sets convolution layers according to the defect type number, and extracts defect feature graph groups of corresponding types; finally, inputting the first image into a second packet convolution network, and classifying the output defect feature map group by adopting the packet convolution network to obtain defect type information contained in the second image;
And the filtering module is used for selecting corresponding characteristic graph groups to be input into the boundary box regression network to calculate the boundary box information of the defects, and then filtering the output result of the boundary box regression network by a non-maximum suppression method to obtain the final detection result of the surface quality of the beam blank.
The invention also provides a device for controlling the surface quality of the beam blank, which comprises an imaging module and an upper computer;
the imaging module acquires a first image for measuring the surface size of the special-shaped blank through an infrared channel and a near-infrared channel, and acquires a second image for detecting the surface defects of the special-shaped blank through a blue light channel;
the upper computer is used for extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as the input of the first packet convolution network, setting convolution layers according to the defect type number, and extracting defect features of corresponding types; finally, inputting the first image into a second grouped convolution network, and classifying the output defect characteristics by adopting the grouped convolution network to obtain defect type information contained in the second image; and selecting a corresponding characteristic graph group, inputting the characteristic graph group into the boundary box regression network to calculate the boundary box information of the defects, and filtering the output result of the boundary box regression network by a non-maximum suppression method to obtain the final detection result of the surface quality of the beam blank.
Furthermore, the device also comprises a code spraying and marking machine;
and the code spraying standard machine is used for receiving an instruction of an upper computer after the encoder determines the physical position of the special-shaped blank and spraying and marking the special-shaped blank according to the final special-shaped blank surface quality detection result.
Further, the device also comprises a sharpening machine;
and the sharpening machine is used for carrying out corresponding sharpening on the surface of the special-shaped blank according to the code-spraying and marking result and the code-spraying and marking position of the special-shaped blank by the physical position.
The effects provided in the summary of the invention are only the effects of the embodiments, not all of the effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
the invention provides a method, a system and a device for controlling the surface quality of a beam blank, wherein the method comprises the steps of acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank; extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as the input of a first packet convolution network, setting convolution layers according to the number of defect types, and extracting a defect feature graph set of the corresponding type; finally, inputting the first image into a second grouped convolution network, and classifying the output defect feature map group by adopting the grouped convolution network to obtain defect type information contained in the second image; and calculating the boundary box information of the defects according to the defect types, and then filtering the boundary box information output by the boundary box regression network to obtain the final detection result of the surface quality of the beam blank. The method further comprises the steps of carrying out code spraying and marking on the special-shaped blank according to the final special-shaped blank surface quality detection result, and carrying out corresponding polishing on the special-shaped blank surface at the position of the code spraying and marking of the special-shaped blank. The invention realizes the fusion of line structured light and depth learning algorithm based on a dual-channel image, and simultaneously realizes the physical dimension measurement and surface defect detection of the special-shaped blank.
The invention adopts the imaging module to realize multi-spectral and multi-directional acquisition of the special-shaped blank image and provide complete data support for the physical dimension measurement, surface defect detection and control of the special-shaped blank.
The invention also carries out code spraying and marking on the special-shaped blank according to the final special-shaped blank surface quality detection result, and carries out corresponding coping on the special-shaped blank surface at the position of the code spraying and marking of the special-shaped blank, thereby realizing defect control and meeting the product production standard.
Drawings
Fig. 1 is a flowchart of a method for controlling the surface quality of a beam blank according to embodiment 1 of the present invention;
fig. 2 is a schematic diagram of a neural network in the method for controlling the surface quality of a beam blank according to embodiment 1 of the present invention;
FIG. 3 is a schematic view of a system for controlling the surface quality of a beam blank according to embodiment 2 of the present invention;
fig. 4 is a schematic connection diagram of a device for controlling the surface quality of a beam blank according to embodiment 3 of the present invention.
Detailed Description
In order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
Example 1
The embodiment 1 of the invention provides a method for controlling the surface quality of a special-shaped blank, which is based on the fusion of a line structure light and a depth learning algorithm of a dual-channel image and realizes the physical dimension measurement and the surface defect detection of the special-shaped blank at the same time.
Fig. 1 is a flowchart of a method for controlling the surface quality of a beam blank according to embodiment 1 of the present invention;
acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank; according to the method, a first image for measuring the surface size of the special-shaped blank is acquired through an infrared channel and a near-infrared channel in an imaging module; and acquiring a second image for detecting the surface defects of the special-shaped blank through a blue light channel in the imaging module.
Extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as input of a first packet convolution network, setting convolution layers according to the defect category number, and extracting defect feature graph groups of corresponding categories; finally, inputting the first image into a second packet convolution network, and classifying the output defect feature map group by adopting the packet convolution network to obtain defect type information contained in the second image; and calculating the boundary frame information of the defects according to the defect types, and then filtering the boundary frame information output by the boundary frame regression network to obtain the final detection result of the surface quality of the beam blank.
Fig. 2 is a schematic diagram of a neural network in the method for controlling the surface quality of a beam blank according to embodiment 1 of the present invention; building a defect detection network and initializing parameters: a grouping convolution classification network based on deep learning and used for detecting common defects of a special-shaped blank is built, the grouping convolution classification network not only comprises a shared convolution layer to extract common bottom layer characteristics in an image, but also introduces mutually independent convolution groups respectively used for extracting abstract characteristics of different types of defects.
The first part extracts common signs from the built model, the first part has the same structure as the traditional convolutional neural network, the shared convolutional layer is adopted to extract common characteristics in the image, the CNN network convolutional layer is adopted as a pre-training model to improve the generalization capability of the network, and the specific structure is shown in the following table. Meanwhile, in consideration of the difference of abstract features of different types of defects, the Conv3-3 convolutional layer with a smaller receptive field is adopted in the project.
| Types of | Parameter(s) |
| Input device | |
| Conv1-1 | 3x3x64 |
| Conv1-2 | 3x3x64 |
| Maximum pooling layers 1-3 | |
| Conv3-1 | 3x3x128 |
| Conv3-2 | 3x3x128 |
| Maximum pooling layer 3-3 | |
| Conv3-1 | 3x3x256 |
| Conv3-2 | 3x3x256 |
| Conv3-3 | 1x1x256 |
| Maximum pooling layers 3-4 | |
| Conv4-1 | 3x3x512 |
| Conv4-2 | 3x3x512 |
| Conv4-3 | 1x1x512 |
| Maximum pooling layer 4-4 |
The image including the common features is used as the input of a first packet convolution network, convolution layers are set according to the defect type number, and the process of extracting the defect feature map group of the corresponding type comprises the following steps:
Setting the convolution layers according to the defect type number, wherein the defect type number is equal to the number of the convolution layers, and each convolution layer is enabled to detect one type of defect; extracting each type of defect by adopting a preset number of convolution kernels, performing convolution calculation on each group of convolution layers independently and the image comprising the common features, and extracting defect feature map groups of corresponding categories
And in the second part, each class optimizes a group convolution kernel to extract the characteristics of the class, and the second part is k groups of convolution layers which are independent from each other, wherein k is the defect class number, and each convolution layer group is only responsible for detecting one type of defect. In the experiment, 20 convolution kernels are respectively used by the grouped convolution classification network to extract each type of defect features, namely each group of convolution layers (kernel:3x3x20) are respectively and independently convolved with the feature map output by the first part to extract the defect features of the corresponding type.
And when the defect feature classification is carried out, the same classifier respectively calculates the defect feature map groups of the corresponding classes so as to judge whether the defect type represented by the current feature map group appears in the second image.
The third part distributes the weights of the convolution kernels and the convolution logic models to different classes to enable the classification results to be uniform, and the third part distributes the weights of the convolution kernels and the convolution logic models to different classes to enable the classification results to be uniform. The third part is a final defect classifier, and since the grouping convolution classification network adopts mutually independent feature map groups to represent different types of defects, the final classifier only needs to carry out secondary classification on the k feature map groups output by the second part, so that defect type information contained in the image can be obtained. In order to reduce the difference of the convolution layer group output feature maps which are independent of each other, a shared parameter strategy is adopted by the defect classifier, namely the same classifier respectively calculates the k feature map groups output by the second part so as to judge whether the defect type represented by the feature map group appears in the original image. By adopting a shared parameter strategy in the third part of the network, the consistency of the outputs of different types of convolution layer groups can be effectively improved, and particularly when the number difference of different types of defect samples is large, the response of the feature map group can be more uniform, thereby being beneficial to subsequent defect positioning.
The beam blank surface image is input into a packet convolution classification network to predict the type of defect present in the image. And then selecting a corresponding feature map group according to the classification result, inputting the feature map group into another boundary frame regression network to calculate the boundary frame information of the defect, and finally filtering the output result of the boundary frame regression network by a non-maximum suppression method to obtain a final detection result.
According to the method for controlling the surface quality of the special-shaped blank, provided by the embodiment 1 of the invention, various defects such as cracks, scratches, welding slag, welding marks, water slag marks, depressions and scabs on the surface of the special-shaped blank can be identified by adopting a deep learning network model, and the typical defect identification accuracy rate is 90%. And meanwhile, a time sequence analysis algorithm is adopted, and visual display and effective statistics are given to the distribution situation of the defects.
The method also comprises the steps of spraying and marking the special-shaped blank according to the final special-shaped blank surface quality detection result, and carrying out corresponding grinding on the special-shaped blank surface at the position of spraying and marking the special-shaped blank. And (3) setting up a coder scheme for measuring the physical position of the special-shaped blank, uniformly and coordinately controlling position information by an upper computer, and timely starting the code-spraying marking device to mark the special-shaped blank. The grinding machine starts an automatic grinding device to correspondingly grind the code-spraying and marking positions of the special-shaped blank according to the code-spraying and marking results and position information fed back by the speed measuring system, and grinds surface defects so as to eliminate the defects; the size part is polished to meet the national standard requirement.
The method for controlling the surface quality of the special-shaped blank provided by the embodiment 1 of the invention realizes the fusion of the line structured light and the depth learning algorithm based on the dual-channel image, and simultaneously realizes the physical dimension measurement and the surface defect detection of the special-shaped blank.
The method for controlling the surface quality of the special-shaped blank provided by the embodiment 1 of the invention adopts the imaging module to realize multispectral and multidirectional acquisition of the special-shaped blank image and provide complete data support for the measurement of the physical dimension of the special-shaped blank and the detection and control of surface defects.
The method for controlling the surface quality of the beam blank provided by the embodiment 1 of the invention also carries out code spraying and marking on the beam blank according to the final beam blank surface quality detection result, and carries out corresponding polishing on the beam blank surface at the position of the code spraying and marking of the beam blank, thereby realizing defect control and meeting the production standard of products.
Example 2
The method for controlling the surface quality of the beam blank is provided based on the embodiment 1 of the invention. Embodiment 2 of the present invention further provides a system for controlling the surface quality of a beam blank, and fig. 3 is a schematic diagram of the system for controlling the surface quality of a beam blank according to embodiment 2 of the present invention; the system comprises an acquisition module, an extraction module and a filtering module;
The acquisition module is used for acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank;
the extraction module extracts common features in the second image through a shared convolution network, then takes the image comprising the common features as the input of a first packet convolution network, sets convolution layers according to the defect type number, and extracts a defect feature graph set of a corresponding type; finally, inputting the first image into a second grouped convolution network, and classifying the output defect feature map group by adopting the grouped convolution network to obtain defect type information contained in the second image;
and the filtering module is used for selecting corresponding characteristic graph groups to be input into the boundary frame regression network to calculate the boundary frame information of the defects, and then filtering the output result of the boundary frame regression network by a non-maximum inhibition method to obtain the final detection result of the surface quality of the beam blank.
The system also comprises a marking and polishing module; and the marking and grinding module is used for spraying codes and marking the special-shaped blank according to the final special-shaped blank surface quality detection result and carrying out corresponding grinding on the surface of the special-shaped blank at the position of the code and marking of the special-shaped blank.
The acquisition module acquires a first image for measuring the surface size of the special-shaped blank through an infrared channel and a near-infrared channel in the imaging module; and acquiring a second image for detecting the surface defects of the special-shaped blank through a blue light channel in the imaging module.
The detailed process realized by the extraction module comprises the following steps: building a defect detection network and initializing parameters: a grouping convolution classification network based on deep learning and used for detecting common defects of a special-shaped blank is built, the grouping convolution classification network not only comprises a shared convolution layer to extract common bottom layer characteristics in an image, but also introduces mutually independent convolution groups respectively used for extracting abstract characteristics of different types of defects.
The first part extracts common signs from the built model, the first part has the same structure as the traditional convolutional neural network, the shared convolutional layer is adopted to extract common features in the image, the CNN network convolutional layer is adopted as a pre-training model to improve the generalization capability of the network, and meanwhile, the Conv3-3 convolutional layer with smaller receptive field is adopted in the project in consideration of the difference of different defect abstract features.
Setting the convolution layers according to the defect type number, wherein the defect type number is equal to the number of the convolution layers, and each convolution layer is enabled to detect one type of defect; and extracting each type of defect by adopting a preset number of convolution kernels, wherein each group of convolution layers are respectively independent and carry out convolution calculation with the image comprising the common features, and extracting a defect feature map group of a corresponding category.
And in the second part, each class optimizes a group convolution kernel to extract the characteristics of the class, and the second part is k groups of convolution layers which are independent from each other, wherein k is the defect class number, and each convolution layer group is only responsible for detecting one type of defect. In the experiment, 20 convolution kernels are respectively used by the grouped convolution classification network to extract each type of defect features, namely each group of convolution layers (kernel:3x3x20) are respectively and independently convolved with the feature map output by the first part to extract the defect features of the corresponding type.
And when the defect feature classification is carried out, the same classifier respectively calculates the defect feature map groups of the corresponding classes so as to judge whether the defect type represented by the current feature map group appears in the second image.
The third part distributes the weights of the convolution kernels and the convolution logic models to different classes to enable classification results to be uniform, the third part is a final defect classifier, and the grouping convolution classification network adopts mutually independent feature map groups to represent different types of defects, so that the final classifier only needs to carry out secondary classification on the k feature map groups output by the second part, and defect type information contained in the image can be obtained. In order to reduce the difference of the convolution layer group output feature maps which are independent of each other, a shared parameter strategy is adopted by the defect classifier, namely the same classifier respectively calculates the k feature map groups output by the second part so as to judge whether the defect type represented by the feature map group appears in the original image. By adopting a shared parameter strategy in the third part of the network, the consistency of the outputs of different types of convolution layer groups can be effectively improved, and particularly when the number difference of different types of defect samples is large, the response of the feature map group can be more uniform, thereby being beneficial to subsequent defect positioning.
The process implemented by the filtering module comprises the following steps: the preform surface image is input into a packet convolution classification network to predict the type of defect present in the image. And then selecting a corresponding feature map group according to the classification result, inputting the feature map group into another boundary frame regression network to calculate the boundary frame information of the defect, and finally filtering the output result of the boundary frame regression network by a non-maximum suppression method to obtain a final detection result.
The process of marking and polishing module implementation comprises the following steps: and carrying out code spraying and marking on the special-shaped blank according to the final special-shaped blank surface quality detection result, and carrying out corresponding grinding on the special-shaped blank surface at the position of the code spraying and marking of the special-shaped blank. And (3) setting up a coder scheme for measuring the physical position of the special-shaped blank, uniformly and coordinately controlling position information by an upper computer, and timely starting the code-spraying marking device to mark the special-shaped blank. The grinding machine starts an automatic grinding device to correspondingly grind the code-spraying and marking positions of the special-shaped blank according to the code-spraying and marking results and position information fed back by the speed measuring system, and grinds surface defects so as to eliminate the defects; the size part is polished to meet the national standard requirement.
The system for controlling the surface quality of the beam blank, which is provided by the embodiment 2 of the invention, realizes the fusion of a line structured light and a depth learning algorithm based on a dual-channel image, and simultaneously realizes the physical dimension measurement and the surface defect detection of the beam blank.
The system for controlling the surface quality of the beam blank, which is provided by the embodiment 2 of the invention, adopts the imaging module to realize multi-spectral and multi-directional acquisition of the beam blank image and provide complete data support for beam blank physical dimension measurement, surface defect detection and control.
The system for controlling the surface quality of the beam blank provided by the embodiment 2 of the invention also carries out code spraying and marking on the beam blank according to the final beam blank surface quality detection result, and carries out corresponding polishing on the beam blank surface at the position where the code spraying and marking are carried out on the beam blank, thereby realizing defect control and meeting the product production standard.
Example 3
The method for controlling the surface quality of the beam blank is provided based on the embodiment 1 of the invention. Embodiment 3 of the present invention further provides a device for controlling the surface quality of the beam blank, and as shown in fig. 4, the device for controlling the surface quality of the beam blank in embodiment 3 of the present invention is schematically connected, and the device includes an imaging module and an upper computer;
the imaging module is connected with a first input port of the upper computer, and the encoder is connected with a second input port of the upper computer; the output port of the upper computer is connected with the input port of the code spraying marking machine, and the output port of the code spraying marking machine is connected with the sharpening machine.
The imaging module acquires a first image for measuring the surface size of the special-shaped blank through an infrared channel and a near-infrared channel, and acquires a second image for detecting the surface defects of the special-shaped blank through a blue light channel;
the upper computer is used for extracting common features in the second image through the shared convolution network, then taking the image comprising the common features as the input of the first packet convolution network, setting the convolution layers according to the defect type number and extracting the defect features of corresponding types; finally, inputting the first image into a second packet convolutional network, and classifying the output defect characteristics by adopting the packet convolutional network to obtain defect type information contained in the second image; and selecting a corresponding characteristic diagram group, inputting the characteristic diagram group into the boundary frame regression network to calculate the boundary frame information of the defects, and filtering the output result of the boundary frame regression network by a non-maximum inhibition method to obtain the final surface quality detection result of the beam blank.
The process of the deep learning algorithm in the upper computer comprises the following steps: building a defect detection network and initializing parameters: a grouping convolution classification network based on deep learning and used for detecting common defects of a special-shaped blank is built, the grouping convolution classification network not only comprises a shared convolution layer to extract common bottom layer characteristics in an image, but also introduces mutually independent convolution groups respectively used for extracting abstract characteristics of different types of defects.
The first part extracts common signs from the built model, the first part has the same structure as the traditional convolutional neural network, the shared convolutional layer is adopted to extract common features in the image, the CNN network convolutional layer is adopted as a pre-training model to improve the generalization capability of the network, and meanwhile, the Conv3-3 convolutional layer with smaller receptive field is adopted in the project in consideration of the difference of different defect abstract features.
The method comprises the following steps of taking an image including common features as input of a first packet convolution network, setting convolution layers according to the number of defect types, and extracting a defect feature map group of a corresponding type, wherein the process comprises the following steps:
setting the convolution layers according to the defect type number, wherein the defect type number is equal to the number of the convolution layers, and each convolution layer is enabled to detect one type of defect; extracting each type of defect by adopting a preset number of convolution kernels, performing convolution calculation on each group of convolution layers independently and the image comprising the common features, and extracting defect feature map groups of corresponding categories
And in the second part, each class optimizes a group convolution kernel to extract the characteristics of the class, and the second part is k groups of convolution layers which are independent from each other, wherein k is the defect class number, and each convolution layer group is only responsible for detecting one type of defect. In the experiment, 20 convolution kernels are respectively used by the grouped convolution classification network to extract each type of defect features, namely each group of convolution layers (kernel:3x3x20) are respectively and independently convolved with the feature map output by the first part to extract the defect features of the corresponding type.
And when the defect feature classification is carried out, the same classifier respectively calculates the defect feature map groups of the corresponding classes so as to judge whether the defect type represented by the current feature map group appears in the second image.
The third part is a final defect classifier, and since the grouping convolution classification network adopts mutually independent feature map groups to represent different types of defects, the final classifier only needs to carry out secondary classification on the k feature map groups output by the second part, so that defect type information contained in the image can be obtained. In order to reduce the difference of the convolution layer group output feature maps which are independent of each other, a shared parameter strategy is adopted by the defect classifier, namely the same classifier respectively calculates the k feature map groups output by the second part so as to judge whether the defect type represented by the feature map group appears in the original image. By adopting a shared parameter strategy in the third part of the network, the consistency of the outputs of convolution layer groups of different types can be effectively improved, and particularly when the number difference of defect samples of different types is large, the response of the feature map group can be more uniform, thereby being beneficial to subsequent defect positioning.
The preform surface image is input into a packet convolution classification network to predict the type of defect present in the image. And then selecting a corresponding feature map group according to the classification result, inputting the feature map group into another boundary frame regression network to calculate the boundary frame information of the defect, and finally filtering the output result of the boundary frame regression network by a non-maximum suppression method to obtain a final detection result.
The device also comprises a code spraying and marking machine;
and the code spraying standard reaching machine is used for receiving an instruction of the upper computer after the encoder determines the physical position of the special-shaped blank and spraying and marking the special-shaped blank according to the final special-shaped blank surface quality detection result.
The device also comprises a sharpening machine;
the grinding machine is used for carrying out corresponding grinding on the surface of the special-shaped blank according to the code-spraying marking result and the code-spraying marking position of the special-shaped blank by the physical position.
The device for controlling the surface quality of the special-shaped blank, which is provided by the embodiment 3 of the invention, realizes the fusion of a line structured light and a depth learning algorithm based on a dual-channel image, and simultaneously realizes the physical dimension measurement and the surface defect detection of the special-shaped blank.
The device for controlling the surface quality of the beam blank, which is provided by the embodiment 3 of the invention, adopts the imaging module to realize multi-spectral and multi-directional acquisition of the beam blank image and provide complete data support for beam blank physical dimension measurement and surface defect detection and control.
The device for controlling the surface quality of the beam blank provided by the embodiment 3 of the invention also carries out code spraying and marking on the beam blank according to the final beam blank surface quality detection result, and carries out corresponding polishing on the beam blank surface at the position where the code spraying and marking are carried out on the beam blank, thereby realizing defect control and meeting the product production standard.
It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include the inherent elements. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element. In addition, parts of the technical solutions provided in the embodiments of the present application that are consistent with implementation principles of corresponding technical solutions in the prior art are not described in detail, so as to avoid redundant description.
Although the specific embodiments of the present invention have been described with reference to the accompanying drawings, the scope of the present invention is not limited thereto. Various other modifications and variations to the foregoing description may be apparent to those skilled in the art. This need not be, nor should it be exhaustive of all embodiments. On the basis of the technical scheme of the invention, various modifications or changes which can be made by a person skilled in the art without creative efforts are still within the protection scope of the invention.
Claims (10)
1. A method for controlling the surface quality of a beam blank is characterized by comprising the following steps:
acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank;
extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as the input of a first packet convolution network, setting convolution layers according to the number of defect types, and extracting a defect feature graph set of the corresponding type; finally, inputting the first image into a second grouped convolution network, and classifying the output defect feature map group by adopting the grouped convolution network to obtain defect type information contained in the second image;
and calculating the boundary box information of the defects according to the defect types, and then filtering the boundary box information output by the boundary box regression network to obtain the final detection result of the surface quality of the beam blank.
2. The method for controlling the surface quality of the beam blank according to claim 1, further comprising the steps of carrying out code spraying and marking on the beam blank according to the final beam blank surface quality detection result, and carrying out corresponding grinding on the beam blank surface at the position where the code spraying and marking are carried out on the beam blank.
3. The method of claim 1, wherein the method of obtaining the first image for the measurement of the surface dimension of the beam and obtaining the second image for the detection of the surface defects of the beam is:
acquiring a first image for measuring the surface size of the special-shaped blank through an infrared channel and a near-infrared channel in the imaging module;
and acquiring a second image for detecting the surface defects of the special-shaped blank through a blue light channel in the imaging module.
4. The method of claim 1, wherein the image comprising the common features is used as an input of a first packet convolution network, convolution layers are set according to the defect type number, and the process of extracting the defect feature map set of the corresponding type comprises the following steps:
setting the convolution layers according to the defect type number, wherein the defect type number is equal to the number of the convolution layers, and each convolution layer is enabled to detect one type of defect;
and extracting each type of defect by adopting a preset number of convolution kernels, wherein each group of convolution layers are respectively independent and carry out convolution calculation with the image comprising the common features, and extracting a defect feature map group of a corresponding category.
5. The method as claimed in claim 4, wherein, in the step of classifying the defect features, the same classifier respectively calculates the defect feature map groups of the corresponding classes to determine whether the defect type represented by the current feature map group appears in the second image.
6. The method for controlling the surface quality of the beam blank according to claim 5, wherein the step of calculating the bounding box information of the defect according to the defect type and then filtering the bounding box information output by the bounding box regression network to obtain the final beam blank surface quality detection result comprises the following steps: and selecting a corresponding characteristic graph group according to the defect type, inputting the characteristic graph group into a boundary box regression network to calculate the boundary box information of the defect, and then filtering the output result of the boundary box regression network by a non-maximum suppression method to obtain the final detection result of the surface quality of the beam blank.
7. A system for controlling the surface quality of a beam blank is characterized by comprising an acquisition module, an extraction module and a filtering module;
the acquisition module is used for acquiring a first image for measuring the surface size of the beam blank and acquiring a second image for detecting the surface defects of the beam blank;
The extraction module extracts common features in the second image through a shared convolution network, then takes the image comprising the common features as the input of a first packet convolution network, sets convolution layers according to the defect type number, and extracts defect feature graph groups of corresponding types; finally, inputting the first image into a second grouped convolution network, and classifying the output defect feature map group by adopting the grouped convolution network to obtain defect type information contained in the second image;
and the filtering module is used for selecting corresponding characteristic graph groups to be input into the boundary box regression network to calculate the boundary box information of the defects, and then filtering the output result of the boundary box regression network by a non-maximum suppression method to obtain the final detection result of the surface quality of the beam blank.
8. A device for controlling the surface quality of a beam blank is characterized by comprising an imaging module and an upper computer;
the imaging module acquires a first image for measuring the surface size of the special-shaped blank through an infrared channel and a near-infrared channel, and acquires a second image for detecting the surface defects of the special-shaped blank through a blue light channel;
the upper computer is used for extracting common features in the second image through a shared convolution network, then taking the image comprising the common features as the input of the first packet convolution network, setting convolution layers according to the defect type number, and extracting defect features of corresponding types; finally, inputting the first image into a second grouped convolution network, and classifying the output defect characteristics by adopting the grouped convolution network to obtain defect type information contained in the second image; and selecting a corresponding characteristic graph group, inputting the characteristic graph group into the boundary box regression network to calculate the boundary box information of the defects, and filtering the output result of the boundary box regression network by a non-maximum suppression method to obtain the final detection result of the surface quality of the beam blank.
9. The apparatus of claim 8, further comprising a code-jet marking machine;
the code spraying standard machine is used for receiving an instruction of an upper computer after the encoder determines the physical position of the special-shaped blank and spraying and marking the special-shaped blank according to the final special-shaped blank surface quality detection result.
10. The apparatus of claim 9, further comprising a sharpening machine;
and the grinding machine is used for correspondingly grinding the surface of the special-shaped blank according to the code-spraying marking result and the code-spraying marking position of the special-shaped blank by the physical position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210214912.3A CN114677334A (en) | 2022-03-04 | 2022-03-04 | Method, system and device for controlling surface quality of special-shaped blank |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210214912.3A CN114677334A (en) | 2022-03-04 | 2022-03-04 | Method, system and device for controlling surface quality of special-shaped blank |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114677334A true CN114677334A (en) | 2022-06-28 |
Family
ID=82072709
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210214912.3A Pending CN114677334A (en) | 2022-03-04 | 2022-03-04 | Method, system and device for controlling surface quality of special-shaped blank |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114677334A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474861A (en) * | 2023-10-31 | 2024-01-30 | 东北石油大学 | Surface mounting special-shaped element parameter extraction method and system based on improved RetinaNet and Canny-Franklin moment sub-pixels |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109829893A (en) * | 2019-01-03 | 2019-05-31 | 武汉精测电子集团股份有限公司 | A kind of defect object detection method based on attention mechanism |
| CN109859163A (en) * | 2018-12-19 | 2019-06-07 | 重庆邮电大学 | A kind of LCD defect inspection method based on feature pyramid convolutional neural networks |
| CN110599469A (en) * | 2019-08-30 | 2019-12-20 | 中国铁道科学研究院集团有限公司 | Method and system for detecting defects of key parts of motor train unit and electronic equipment |
| CN111929309A (en) * | 2020-09-14 | 2020-11-13 | 成都卓识维景科技有限公司 | Cast part appearance defect detection method and system based on machine vision |
| CN112207687A (en) * | 2020-09-27 | 2021-01-12 | 太原理工大学 | Polishing robot system for finishing operation of special steel bar |
| CN112257849A (en) * | 2020-10-23 | 2021-01-22 | 西南石油大学 | Intelligent detection method, system and device based on deep learning and application thereof |
| CN113256624A (en) * | 2021-06-29 | 2021-08-13 | 中移(上海)信息通信科技有限公司 | Continuous casting round billet defect detection method and device, electronic equipment and readable storage medium |
-
2022
- 2022-03-04 CN CN202210214912.3A patent/CN114677334A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859163A (en) * | 2018-12-19 | 2019-06-07 | 重庆邮电大学 | A kind of LCD defect inspection method based on feature pyramid convolutional neural networks |
| CN109829893A (en) * | 2019-01-03 | 2019-05-31 | 武汉精测电子集团股份有限公司 | A kind of defect object detection method based on attention mechanism |
| CN110599469A (en) * | 2019-08-30 | 2019-12-20 | 中国铁道科学研究院集团有限公司 | Method and system for detecting defects of key parts of motor train unit and electronic equipment |
| CN111929309A (en) * | 2020-09-14 | 2020-11-13 | 成都卓识维景科技有限公司 | Cast part appearance defect detection method and system based on machine vision |
| CN112207687A (en) * | 2020-09-27 | 2021-01-12 | 太原理工大学 | Polishing robot system for finishing operation of special steel bar |
| CN112257849A (en) * | 2020-10-23 | 2021-01-22 | 西南石油大学 | Intelligent detection method, system and device based on deep learning and application thereof |
| CN113256624A (en) * | 2021-06-29 | 2021-08-13 | 中移(上海)信息通信科技有限公司 | Continuous casting round billet defect detection method and device, electronic equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 邱锡鹏: "《神经网络与深度学习》", 北京：机械工业出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474861A (en) * | 2023-10-31 | 2024-01-30 | 东北石油大学 | Surface mounting special-shaped element parameter extraction method and system based on improved RetinaNet and Canny-Franklin moment sub-pixels |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110806736B (en) | Method for detecting quality information of forge pieces of die forging forming intelligent manufacturing production line | |
| CN106952250B (en) | Metal plate strip surface defect detection method and device based on fast R-CNN network | |
| CN108355987B (en) | A kind of screen printing of battery quality determining method based on piecemeal template matching | |
| WO2022160170A1 (en) | Method and apparatus for detecting metal surface defects | |
| CN108074231B (en) | Magnetic sheet surface defect detection method based on convolutional neural network | |
| CN106934800B (en) | Metal plate strip surface defect detection method and device based on YOLO9000 network | |
| CN107657603A (en) | A kind of industrial appearance detecting method based on intelligent vision | |
| CN105251707B (en) | Bad part eject sorting equipment based on medical large transfusion visible foreign matters detecting system | |
| CN110443791B (en) | Workpiece detection method and device based on deep learning network | |
| CN104914111A (en) | Strip steel surface defect on-line intelligent identification and detection system and detection method | |
| CN113393438B (en) | Resin lens defect detection method based on convolutional neural network | |
| CN113393426B (en) | Steel rolling plate surface defect detection method | |
| CN111127417B (en) | Printing defect detection method based on SIFT feature matching and SSD algorithm improvement | |
| CN113222982A (en) | Wafer surface defect detection method and system based on improved YOLO network | |
| CN112419291B (en) | Training method of bottle embryo defect detection model, storage medium and terminal equipment | |
| CN114235837A (en) | LED packaging surface defect detection method, device, medium and equipment based on machine vision | |
| CN114677334A (en) | Method, system and device for controlling surface quality of special-shaped blank | |
| CN108509947A (en) | A kind of automatic identification polishing process based on artificial neural network | |
| CN117890389B (en) | Online detection method and equipment for surface quality of refined products and storage medium thereof | |
| CN116678826A (en) | Appearance defect detection system and method based on rapid three-dimensional reconstruction | |
| CN114092415A (en) | Large-size light guide plate defect visual detection method | |
| CN112906302A (en) | Injection molding surface micro-defect automatic and rapid detection method based on depth residual error network | |
| CN115511775A (en) | Light-weight ceramic tile surface defect detection method based on semantic segmentation | |
| CN113205136A (en) | Real-time high-precision detection method for appearance defects of power adapter | |
| CN112200762A (en) | Diode glass bulb defect detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220628 |