CN112669313A - Metal surface defect positioning and classifying method - Google Patents
Metal surface defect positioning and classifying method Download PDFInfo
- Publication number
- CN112669313A CN112669313A CN202110052440.1A CN202110052440A CN112669313A CN 112669313 A CN112669313 A CN 112669313A CN 202110052440 A CN202110052440 A CN 202110052440A CN 112669313 A CN112669313 A CN 112669313A
- Authority
- CN
- China
- Prior art keywords
- network
- image data
- defect
- metal
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 89
- 239000002184 metal Substances 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013528 artificial neural network Methods 0.000 claims abstract description 8
- 230000003321 amplification Effects 0.000 claims description 7
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a metal surface defect positioning and classifying method, and relates to the technical field of image detection; acquiring metal defect image data, acquiring a prediction mask of the metal defect image data by using a cascade network, cutting an interested area according to the prediction mask to obtain a minimum rectangular image of a defect part, and acquiring metal surface defect classification by using a classification neural network.
Description
Technical Field
The invention discloses a method, relates to the technical field of image detection, and particularly relates to a metal surface defect positioning and classifying method.
Background
Automatic detection of metal surface defects is very important in industrial product quality control. The existing methods are based on image processing or machine learning technology, but the methods can only detect defects under specific detection conditions, such as obvious defect outline, strong contrast and low noise.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a metal surface defect positioning and classifying method which is used for automatically detecting metal defects, acquiring images from a real industrial environment and accurately positioning and classifying the defects.
The specific scheme provided by the invention is as follows:
a method for positioning and classifying metal surface defects comprises the steps of collecting metal defect image data, utilizing a cascade network to obtain a prediction mask of the metal defect image data, cutting an interested area according to the prediction mask to obtain a minimum rectangular image of a defect part, and utilizing a classification neural network to obtain metal surface defect classification.
Preferably, at least two self-coding networks with the same structure in the metal surface defect positioning and classifying method form a cascade network.
Preferably, the self-coding network in the metal surface defect positioning and classifying method includes an encoder network and a decoder network, the encoder network converts an image into a multi-dimensional feature image to perform feature extraction and representation, and obtains a feature map, and the decoder network finely adjusts the pixel-level label by combining context information of the feature map learned by all intermediate layers.
Preferably, in the method for locating and classifying metal surface defects, data amplification is performed on the acquired metal defect image data.
Preferably, in the metal surface defect positioning and classifying method, collected metal defect image data is converted into gray image data, size normalization processing is performed, and the gray image data is input into the cascade network.
A metal surface defect positioning and classifying system comprises an acquisition module, a cascade network processing module, a cutting module and a classifying module,
the collecting module collects metal defect image data, the cascade network processing module utilizes a cascade network to obtain a prediction mask of the metal defect image data, the cutting module cuts an interested area according to the prediction mask to obtain a minimum rectangular image of a defect part, and the classifying module utilizes a classifying neural network to obtain metal surface defect classification.
Preferably, the cascade network processing module in the metal surface defect positioning and classifying system forms a cascade network by using at least two self-coding networks with the same structure.
Preferably, the self-coding network utilized by the cascade network processing module in the metal surface defect positioning and classifying system comprises an encoder network and a decoder network, wherein the encoder network converts an image into a multi-dimensional feature image for feature extraction and representation, and obtains a feature map, and the decoder network finely adjusts the pixel level label by combining context information of the feature map learned by all intermediate layers.
Preferably, the metal surface defect positioning and classifying system further comprises a data amplification module, and the data amplification module performs data amplification on the acquired metal defect image data.
Preferably, the metal surface defect positioning and classifying system further comprises a preprocessing module, wherein the preprocessing module converts the acquired metal defect image data into gray image data, performs size normalization processing, and inputs the gray image data into the cascade network.
The invention has the advantages that:
the invention provides a metal surface defect positioning and classifying method which is used for automatically detecting metal defects, obtains images from a real industrial environment, is not influenced by the environment and accurately positions and classifies the defects. The cascade network converts the input defect image into a pixel-level prediction mask based on semantic segmentation, and the classification is carried out by utilizing a deep learning network.
Drawings
FIG. 1 is a schematic diagram of a data set creation process in the method of the present invention;
FIG. 2 is a schematic flow diagram of the process of the present invention;
fig. 3 is a schematic diagram of the self-coding network structure in the present invention.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The invention provides a metal surface defect positioning and classifying method, which comprises the steps of collecting metal defect image data, utilizing a cascade network to obtain a prediction mask of the metal defect image data, cutting an interested area according to the prediction mask to obtain a minimum rectangular image of a defect part, and utilizing a classifying neural network to obtain metal surface defect classification.
The method is used for automatically detecting the metal defects, obtains images from a real industrial environment, is not influenced by the environment, and accurately positions and classifies the defects. The method converts the input original image into a prediction mask, utilizes the defect region detection to extract the characteristics and cuts the defect regions as the input of the next network, and finally obtains the classification result of the defect regions through CNN.
In particular applications, in some embodiments of the present invention, metal surface defects have a different characterization from background structures because defects are local anomalies in a uniform structure. The common characteristics of the metal surface defects can be found out by utilizing a self-coding network learning defect data representation method. Therefore, the metal surface defect detection problem is converted into a semantic segmentation problem. The input defect image may be converted into a prediction mask at the pixel level. The specific process is as follows:
the method comprises the steps of collecting metal defect image data, carrying out graying and normalization processing on the data, and obtaining a prediction mask of the metal defect image data by utilizing a cascade network, wherein the cascade network is formed by utilizing cascade self-coding networks, and each self-coding network comprises an encoder network and a decoder network. The encoder network is a conversion unit by which the input image is converted into a multi-dimensional feature image for feature extraction and representation, the obtained feature map having rich semantic information. The decoder network fine-tunes the pixel-level labels by merging the context information of the feature maps learned by all the middle layers. In addition, the decoder network may use an upsampling operation to restore the final output to the same size as the input image. For example, two self-coding networks with the same structure are used for cascading to complete the image segmentation task. The prediction mask of the first network is used as an input to the second network, where the pixel labels are further fine-tuned, which enhances the prediction of the former network.
The upper layer of the self-coding network is an encoder network and comprises a plurality of convolution layers, and each convolution layer comprises f multiplied by f convolution operation, RELU nonlinear operation and s multiplied by s maximal pooling operation. To reduce the loss of semantic information, the number of features after each max pooling layer is doubled. The lower layer is the decoder network, which applies the up-sampling operation to recover the feature mapping. Meanwhile, the conventional convolution operation has a stable receptive field, so that it is difficult to survey the whole situation and detect all defects when generating a prediction mask, and the sizes and shapes of the defects are various in a real industrial detection environment. Aiming at the situation, the method replaces the conventional convolution with the hole convolution so as to increase the receptive field of the network and detect larger defects.
After semantic segmentation results of all possible defects are obtained, based on this, the minimum bounding rectangular region is extracted from the final image according to the defect outline. And the random area is adjusted to the positive direction by affine transformation and is used as the input of a classification neural network to obtain the classification of the defects.
On the basis of the embodiment, because the acquired data of the metal defects is limited, the generation of the countermeasure network is introduced for data amplification, and the purpose of avoiding serious imbalance of the data samples is achieved. The main data amplification modes include random rotation, translation, shearing, elastic transformation and the like.
The invention also provides a metal surface defect positioning and classifying system, which comprises an acquisition module, a cascade network processing module, a cutting module and a classifying module,
the collecting module collects metal defect image data, the cascade network processing module utilizes a cascade network to obtain a prediction mask of the metal defect image data, the cutting module cuts an interested area according to the prediction mask to obtain a minimum rectangular image of a defect part, and the classifying module utilizes a classifying neural network to obtain metal surface defect classification.
The system is also used for automatically detecting the metal defects, obtains images from a real industrial environment, is not influenced by the environment, and accurately positions and classifies the defects. The method converts the input original image into a prediction mask, utilizes the defect region detection to extract the characteristics and cuts the defect regions as the input of the next network, and finally obtains the classification result of the defect regions through CNN.
The information interaction, execution process and other contents between the modules in the system are based on the same concept as the method embodiment of the present invention, and specific contents can be referred to the description in the method embodiment of the present invention, and are not described herein again.
It should be noted that not all steps and modules in the above flows and system structures are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (10)
1. A method for locating and classifying metal surface defects is characterized in that metal defect image data are collected, a prediction mask of the metal defect image data is obtained through a cascade network, a region of interest is cut according to the prediction mask to obtain a minimum rectangular image of a defect part, and classification neural networks are used for obtaining classification of the metal surface defects.
2. The method as claimed in claim 1, wherein at least two self-coding networks with the same structure form a cascade network.
3. The method of claim 2, wherein the self-encoding network comprises an encoder network and a decoder network, the encoder network converts the image into a multi-dimensional feature image for feature extraction and representation and obtaining a feature map, and the decoder network fine-tunes the pixel-level label by merging context information of the feature map learned by all intermediate layers.
4. A method according to any one of claims 1 to 3 wherein the acquired metal defect image data is subjected to data amplification.
5. The method as claimed in claim 4, wherein the collected metal defect image data is converted into gray scale image data, and the gray scale image data is subjected to size normalization and then input into the cascade network.
6. A metal surface defect positioning and classifying system is characterized by comprising an acquisition module, a cascade network processing module, a cutting module and a classifying module,
the collecting module collects metal defect image data, the cascade network processing module utilizes a cascade network to obtain a prediction mask of the metal defect image data, the cutting module cuts an interested area according to the prediction mask to obtain a minimum rectangular image of a defect part, and the classifying module utilizes a classifying neural network to obtain metal surface defect classification.
7. The system of claim 6, wherein the cascade network processing module utilizes at least two self-coding networks having the same structure to form the cascade network.
8. The system of claim 7, wherein the self-encoding network utilized by the cascaded network processing modules comprises an encoder network and a decoder network, the encoder network converting the image into a multi-dimensional feature image for feature extraction and representation and obtaining a feature map, the decoder network fine-tuning the pixel-level labels by merging context information of the feature maps learned by all intermediate layers.
9. The system of any one of claims 6 to 8, further comprising a data expansion module for performing data expansion on the collected metal defect image data.
10. The system of claim 9, further comprising a preprocessing module, wherein the preprocessing module converts the collected metal defect image data into gray scale image data, performs size normalization, and inputs the gray scale image data into the cascade network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110052440.1A CN112669313A (en) | 2021-01-15 | 2021-01-15 | Metal surface defect positioning and classifying method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110052440.1A CN112669313A (en) | 2021-01-15 | 2021-01-15 | Metal surface defect positioning and classifying method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112669313A true CN112669313A (en) | 2021-04-16 |
Family
ID=75415261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110052440.1A Pending CN112669313A (en) | 2021-01-15 | 2021-01-15 | Metal surface defect positioning and classifying method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112669313A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113205513A (en) * | 2021-05-27 | 2021-08-03 | 山东浪潮科学研究院有限公司 | Industrial equipment surface defect fault early warning method based on edge calculation |
| CN115018847A (en) * | 2022-08-09 | 2022-09-06 | 海门市华呈精密标准件有限公司 | Automatic identification and classification method for surface defects of metal plate |
| CN116012375A (en) * | 2023-03-22 | 2023-04-25 | 成都唐源电气股份有限公司 | Method and system for detecting cotter pin defects of overhead contact system soft crossing suspension pulley |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993735A (en) * | 2019-03-29 | 2019-07-09 | 成都信息工程大学 | Image partition method based on concatenated convolutional |
| CN111008973A (en) * | 2018-10-05 | 2020-04-14 | 罗伯特·博世有限公司 | Method, artificial neural network and device for semantic segmentation of image data |
| CN111709910A (en) * | 2020-05-18 | 2020-09-25 | 扬州小纳熊机器人有限公司 | PCB defect detection algorithm based on convolutional neural network |
| CN112037219A (en) * | 2020-09-27 | 2020-12-04 | 上海电力大学 | Metal surface defect detection method based on two-stage convolution neural network |
-
2021
- 2021-01-15 CN CN202110052440.1A patent/CN112669313A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111008973A (en) * | 2018-10-05 | 2020-04-14 | 罗伯特·博世有限公司 | Method, artificial neural network and device for semantic segmentation of image data |
| CN109993735A (en) * | 2019-03-29 | 2019-07-09 | 成都信息工程大学 | Image partition method based on concatenated convolutional |
| CN111709910A (en) * | 2020-05-18 | 2020-09-25 | 扬州小纳熊机器人有限公司 | PCB defect detection algorithm based on convolutional neural network |
| CN112037219A (en) * | 2020-09-27 | 2020-12-04 | 上海电力大学 | Metal surface defect detection method based on two-stage convolution neural network |
Non-Patent Citations (1)
| Title |
|---|
| XIAN TAO ET AL: "Automatic Metallic Surface Defect Detection and Recognition with Convolutional Neural Networks", 《MDPI》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113205513A (en) * | 2021-05-27 | 2021-08-03 | 山东浪潮科学研究院有限公司 | Industrial equipment surface defect fault early warning method based on edge calculation |
| CN115018847A (en) * | 2022-08-09 | 2022-09-06 | 海门市华呈精密标准件有限公司 | Automatic identification and classification method for surface defects of metal plate |
| CN116012375A (en) * | 2023-03-22 | 2023-04-25 | 成都唐源电气股份有限公司 | Method and system for detecting cotter pin defects of overhead contact system soft crossing suspension pulley |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112669313A (en) | Metal surface defect positioning and classifying method | |
| Xiao et al. | Surface defect detection using image pyramid | |
| France et al. | A new approach to automated pollen analysis | |
| CN111862025A (en) | PCB defect automatic detection method based on deep learning | |
| CN111598856A (en) | Chip surface defect automatic detection method and system based on defect-oriented multi-point positioning neural network | |
| CN105931246A (en) | Fabric flaw detection method based on wavelet transformation and genetic algorithm | |
| CN114119480A (en) | Crack defect detection system based on deep learning | |
| CN115439458A (en) | Industrial image defect target detection algorithm based on depth map attention | |
| CN113962951B (en) | Training method and device for detecting segmentation model, and target detection method and device | |
| CN112507876A (en) | Wired table picture analysis method and device based on semantic segmentation | |
| CN112257711B (en) | Method for detecting damage fault of railway wagon floor | |
| CN113393426A (en) | Method for detecting surface defects of rolled steel plate | |
| CN116030050A (en) | On-line detection and segmentation method for surface defects of fan based on unmanned aerial vehicle and deep learning | |
| CN113506239A (en) | Strip steel surface defect detection method based on cross-stage local network | |
| Feng et al. | Improved SOLOv2 detection method for shield tunnel lining water leakages | |
| CN113506281A (en) | Bridge crack detection method based on deep learning framework | |
| CN117975087A (en) | Casting defect identification method based on ECA-ConvNext | |
| CN114596242A (en) | Defect detection method, defect detection device, electronic equipment and computer readable storage medium | |
| CN117522735A (en) | Multi-scale-based dense-flow sensing rain-removing image enhancement method | |
| CN117593244A (en) | Film product defect detection method based on improved attention mechanism | |
| CN117036266A (en) | Industrial image anomaly detection method and system based on knowledge distillation | |
| CN111831570A (en) | Test case generation method oriented to automatic driving image data | |
| CN115587989B (en) | Workpiece CT image defect detection segmentation method and system | |
| CN116342496A (en) | Abnormal object detection method and system for intelligent inspection | |
| Das et al. | Object Detection on Scene Images: A Novel Approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210416 |