CN106952250B - Metal plate strip surface defect detection method and device based on fast R-CNN network - Google Patents
Metal plate strip surface defect detection method and device based on fast R-CNN network Download PDFInfo
- Publication number
- CN106952250B CN106952250B CN201710110525.4A CN201710110525A CN106952250B CN 106952250 B CN106952250 B CN 106952250B CN 201710110525 A CN201710110525 A CN 201710110525A CN 106952250 B CN106952250 B CN 106952250B
- Authority
- CN
- China
- Prior art keywords
- metal plate
- rectangular frame
- plate strip
- image
- defects
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a method and a device for detecting surface defects of a metal plate strip based on an Faster R-CNN network, which can improve the accuracy of the detection of the surface defects of the metal plate strip. The method comprises the following steps: collecting an image of the surface of a metal plate strip for training a Faster R-CNN network; carrying out data enhancement on the acquired image of the surface of the metal plate strip; marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame; training a Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame; and acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained Faster R-CNN network, and classifying and positioning the detected defects. The invention is suitable for the technical field of machine vision.
Description
Technical Field
The invention relates to the technical field of machine vision, in particular to a method and a device for detecting surface defects of a metal plate strip based on an Faster R-CNN network.
Background
The metal plate strip is an indispensable raw material in the industries of automobiles, mechanical manufacturing, chemical engineering, aerospace, shipbuilding and the like. The surface defect of the metal plate strip refers to that the physical or chemical properties of local areas on the surface of the metal plate strip are not uniform due to the process or other various reasons in the production and processing processes of the metal plate strip. Common surface defects of the metal plate strip include roll marks, stains, scratches, holes, missing coatings, depressions, bubbles, foreign matters, peeling and the like. The surface defects are the parts with higher atomic activity and are often the starting points of metal corrosion, and the existence of the surface defects can greatly reduce the fatigue resistance of parts, damage the surface quality of the parts, influence the service performance and the service life of machines and instruments, and influence the performance and the quality of final products. Therefore, the method can detect the surface defects of the metal plate strip in time, evaluate the severity of the defects and has important significance for improving the surface quality and the economic benefit of products.
With the continuous development of the industrialization level in China, the requirements on the surface quality of the metal plate strip are higher and higher, and how to quickly and accurately detect the surface defects becomes a very critical link in the metal plate strip rolling process. The current mainstream surface defect detection technology is a machine vision technology based on image processing, and is basically divided into two steps:
(1) firstly, processing an image obtained by an industrial camera, and then extracting features, wherein the method mainly comprises four types of methods such as a statistical method, a structural method, a spectral method, a model method and the like, wherein the statistical method and the spectral method [2] are widely used;
(2) the features extracted in the step (1) are classified through a trained classifier, and the currently used classifiers include BP neural networks, SVM and the like.
However, these methods have the problems of low contrast between defective and non-defective regions, similarity between noise and fine defects, slow detection speed, low identification precision, and the like, and cannot meet the requirements of accuracy and performance in industrial production.
Disclosure of Invention
The invention aims to solve the technical problem of providing a method and a device for detecting the surface defects of a metal plate strip based on an Faster R-CNN network, so as to solve the problem that the accuracy and real-time requirements of industrial production cannot be met in the prior art.
In order to solve the technical problem, an embodiment of the present invention provides a method for detecting surface defects of a metal plate strip based on a Faster R-CNN network, including:
collecting an image of the surface of a metal plate strip for training a Faster R-CNN network;
performing data enhancement on the acquired image of the surface of the metal plate strip;
marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame;
training a Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame;
and acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained Faster R-CNN network and the acquired surface image of the metal plate strip to be detected in real time, and classifying and positioning the detected defects.
Further, the data enhancement of the acquired image of the surface of the metal plate strip includes:
and performing data enhancement on the acquired image of the surface of the metal plate strip by using a turning and translation strategy.
Further, the step of labeling the acquired image of the surface of the metal plate strip and the defect part in the image obtained by enhancing the data with a rectangular frame, and the step of obtaining the coordinates of the rectangular frame and the types of the defects contained in the rectangular frame includes:
marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the central point of the rectangular frame, the width and the height of the rectangular frame and the type of the defect contained in the rectangular frame;
and generating a data set by the acquired image of the surface of the metal plate strip, the image obtained after data enhancement, the acquired coordinates of the central point of the rectangular frame, the width and the height of the rectangular frame and the types of defects contained in the rectangular frame.
Further, before training the faster r-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained by enhancing the data, the acquired coordinates of the rectangular frame, and the type of the defect included in the rectangular frame, the method further includes:
the total defect classes that can be detected by the Faster R-CNN network are modified according to the defect classes to be detected, and each defect class to be detected is assigned a fixed number.
Further, the training of the Faster R-CNN network according to the image of the surface of the metal plate strip acquired in real time and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of defects included in the rectangular frame includes:
and training the Faster R-CNN network according to the generated data set.
The embodiment of the invention also provides a metal plate strip surface defect detection device based on the Faster R-CNN network, which comprises:
the acquisition module is used for acquiring images of the surface of the metal plate strip for training the Faster R-CNN network;
the enhancement module is used for enhancing the data of the acquired image of the surface of the metal plate strip;
the acquisition module is used for marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame;
the training module is used for training the FasterR-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame;
and the detection module is used for acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained FasterR-CNN network and the acquired surface image of the metal plate strip to be detected in real time, and classifying and positioning the detected defects.
Further, the enhancement module is used for performing data enhancement on the acquired image of the surface of the metal plate strip through a turning and translation strategy.
Further, the obtaining module comprises:
the marking unit is used for marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame;
the acquisition unit is used for acquiring the coordinates of the center point of the rectangular frame, the width and the height of the rectangular frame and the types of defects contained in the rectangular frame;
and the generation unit is used for generating a data set by the acquired image of the surface of the metal plate strip and the image obtained by enhancing the data, the acquired coordinates of the central point of the rectangular frame, the width and the height of the rectangular frame and the types of the defects contained in the rectangular frame.
Further, the apparatus further comprises:
and the numbering module is used for modifying the total defect types which can be detected by the Faster R-CNN network according to the defect types to be detected and assigning a fixed number to each defect type to be detected.
Further, the training module is used for training the Faster R-CNN network according to the generated data set.
The technical scheme of the invention has the following beneficial effects:
in the scheme, an image of the surface of a metal plate strip for training a Faster R-CNN network is collected; performing data enhancement on the acquired image of the surface of the metal plate strip; marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame; training a Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame; the method comprises the steps of collecting a surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to a trained Faster R-CNN network and the collected surface image of the metal plate strip to be detected in real time, classifying and positioning the detected defects, meeting the requirement of industrial production on real-time detection of the surface defects of the metal plate strip, and improving the accuracy of detection of the surface defects of the metal plate strip.
Drawings
FIG. 1 is a schematic flow chart of a method for detecting surface defects of a metal plate strip based on a Faster R-CNN network according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating the working principle of the Faster R-CNN network according to the embodiment of the present invention;
fig. 3 is a surface image of a metal plate strip to be detected, which is acquired in real time according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the effect of the method for detecting the surface defect of the metal plate strip based on the Faster R-CNN network according to the embodiment of the present invention;
fig. 5 is a schematic structural diagram of a metal plate strip surface defect detection device based on the Faster R-CNN network according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
The invention provides a method and a device for detecting surface defects of a metal plate strip based on an Faster R-CNN network, aiming at the problem that the existing method cannot meet the requirements of accuracy and real-time property of industrial production.
Example one
Referring to fig. 1, a method for detecting surface defects of a metal plate strip based on a Faster R-CNN network according to an embodiment of the present invention includes:
s101, collecting an image of the surface of a metal plate strip for training a Faster R-CNN network;
s102, performing data enhancement on the acquired image of the surface of the metal plate strip;
s103, marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame;
s104, training the Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the acquired defect types contained in the rectangular frame;
and S105, acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained Faster R-CNN network and the acquired surface image of the metal plate strip to be detected in real time, and classifying and positioning the detected defects.
The method for detecting the surface defects of the metal plate strip based on the Faster R-CNN network comprises the following steps of collecting images of the surface of the metal plate strip for training the Faster R-CNN network; performing data enhancement on the acquired image of the surface of the metal plate strip; marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame; training a Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame; the method comprises the steps of collecting a surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to a trained Faster R-CNN network and the collected surface image of the metal plate strip to be detected in real time, classifying and positioning the detected defects, meeting the requirement of industrial production on real-time detection of the surface defects of the metal plate strip, and improving the accuracy of detection of the surface defects of the metal plate strip.
In this embodiment, in order to better understand the method for detecting the surface defect of the metal plate strip based on the fast R-CNN network, the working principle of the fast R-CNN network is explained first, and as shown in fig. 2, the fast R-CNN mainly comprises two parts: (1) a Region Network (RPN) for generating a Region (candidate Region), where RPN is a full convolution Network, and is mainly used to perform computation and analysis on convolution layer features of a picture, and then, under different image proportions, generate some rectangular frames (coordinates of a rectangular frame are represented by four numbers, coordinates x and y of a frame center point, a height h, and a width w) of the same picture, where the rectangular frames are thousands of rectangular frames, and the rectangular frames are Region (i.e., regions that may be defective); (2) performing defect detection on the region propofol by using a Fast R-CNN (Fast R-CNN) of a deep convolution neural network, and performing calculation analysis on the region propofol obtained by RPN output by using the Fast R-CNN to screen out redundant or wrong region propofol to obtain an optimal rectangular frame and category score, namely a final detection result; in fig. 2, image represents an image; conv layers represent convolutional layers; feature maps represent feature maps; propofol denotes the candidate region; classifier denotes a classifier; the RoI posing is all called region of interest posing, and chinese is the region of interest pooling layer/RoI pooling layer.
In a specific embodiment of the method for detecting surface defects of a metal strip based on the fast R-CNN network, further, the enhancing the collected image of the surface of the metal strip includes:
and performing data enhancement on the acquired image of the surface of the metal plate strip by using a turning and translation strategy.
In the embodiment, the image of the surface of the metal plate strip for training the Faster R-CNN network can be acquired through the industrial camera, and the acquired image of the surface of the metal plate strip is subjected to data enhancement through strategies such as overturning and translation.
In this embodiment, the data enhancement is to enlarge an existing data set by using a strategy such as flipping and translating when data in the data set is too little or data is difficult to obtain, and the obtained data set includes original data and enhanced data. The same object is observed under different angles and different backgrounds, the obtained images may be completely different, and a computer may not be capable of correctly identifying the images, so that the images are subjected to mirror image overturning and horizontal movement, and the data set can comprise various data of the same picture at different angles and different positions.
In a specific embodiment of the method for detecting defects on a surface of a metal strip based on the fast R-CNN network, further, the labeling a defect portion in the acquired image of the surface of the metal strip and the image obtained by enhancing the data with a rectangular frame, and acquiring coordinates of the rectangular frame and a type of the defect included in the rectangular frame includes:
marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the central point of the rectangular frame, the width and the height of the rectangular frame and the type of the defect contained in the rectangular frame;
and generating a data set by the acquired image of the surface of the metal plate strip, the image obtained after data enhancement, the acquired coordinates of the central point of the rectangular frame, the width and the height of the rectangular frame and the types of defects contained in the rectangular frame.
In this embodiment, the acquired image of the surface of the metal plate strip and the defect part in the image obtained by enhancing the data are labeled by using a rectangular frame, the coordinates of the rectangular frame and the types of the defects contained in the rectangular frame are taken (specifically, five parameters of each rectangular frame are acquired, the five parameters of the rectangular frame include the center point coordinate of the rectangular frame (the center point coordinate is a two-dimensional coordinate), the width and the height of the rectangular frame and the types of the defects contained in the rectangular frame), the acquired image of the surface of the metal plate strip and the image obtained by enhancing the data and the five parameters are generated into a text file, the text file includes five parameters of all the rectangular frames in each picture, so as to obtain a data set, the FasterR-CNN network is trained from the generated data set, wherein the data set includes all images and text files.
In this embodiment, the data set includes three parts, the first part is a training set used to train the FasterR-CNN network; the second part is a verification set used for adjusting the Faster R-CNN network hyper-parameter; the third part is a test set to measure network performance.
In this embodiment, when the confidence is calculated for each bounding box, the confidence of each defect type is calculated. In a specific embodiment of the method for detecting surface defects of a metal strip based on a Faster R-CNN network, before training the Faster R-CNN network according to the acquired image of the surface of the metal strip and the image obtained by enhancing the data, the acquired coordinates of the rectangular frame, and the types of defects included in the rectangular frame, the method further includes:
the total defect classes that can be detected by the Faster R-CNN network are modified according to the defect classes to be detected, and each defect class to be detected is assigned a fixed number.
In the present embodiment, the total defect types that can be detected by the Faster R-CNN network are modified according to the defect types to be detected (e.g., roll mark, stain, scratch, hole, smear, dent, bubble, foreign matter, peel, etc.), and a fixed number is assigned to each defect type to be detected.
In this embodiment, when a trained fast R-CNN network is used to detect a surface defect of a metal plate strip, the surface defect of the metal plate strip needs to be acquired in real time, and then whether the surface of the metal plate strip is defective or not is detected in real time according to the trained fast R-CNN network and the surface image of the metal plate strip to be detected acquired in real time, and the detected defects are classified and positioned The effect diagram after the detection of the metal plate strip surface defect detection method.
In this embodiment, an industrial camera may be erected on the metal plate strip production line to collect the surface image of the metal plate strip to be detected in real time, the industrial camera is connected with a computer, and when the surface image of the metal plate strip to be detected collected by the industrial camera is transmitted to the computer, the trained fast R-CNN network may process the image in real time, and the defects in the image are marked with a rectangular frame and output.
Example two
The invention also provides a concrete implementation mode of the metal plate strip surface defect detection device based on the Faster R-CNN network, because the device for detecting the surface defect of the metal plate strip based on the Faster R-CNN network provided by the invention corresponds to the specific implementation mode of the method for detecting the surface defect of the metal plate strip based on the Faster R-CNN network, the device for detecting the surface defects of the metal plate strip based on the Faster R-CNN network can realize the aim of the invention by executing the flow steps in the specific embodiment of the method, therefore, the explanation in the specific embodiment of the method for detecting surface defects of metal strips based on the Faster R-CNN network is also applicable to the specific embodiment of the device for detecting surface defects of metal strips based on the Faster R-CNN network provided by the present invention, and will not be repeated in the following specific embodiment of the present invention.
Referring to fig. 5, an embodiment of the present invention further provides a device for detecting surface defects of a metal plate strip based on the Faster R-CNN network, including:
the acquisition module 11 is used for acquiring images of the surface of the metal plate strip for training the Faster R-CNN network;
the enhancement module 12 is used for performing data enhancement on the acquired image of the surface of the metal plate strip;
the acquiring module 13 is configured to label a defect portion in the acquired image of the surface of the metal plate strip and the image obtained after enhancing the data with a rectangular frame, and acquire coordinates of the rectangular frame and a type of the defect included in the rectangular frame;
the training module 14 is used for training the Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame;
and the detection module 15 is used for acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained Faster R-CNN network and the acquired surface image of the metal plate strip to be detected in real time, and classifying and positioning the detected defects.
The metal plate and strip surface defect detection device based on the Faster R-CNN network of the embodiment of the invention collects the image of the metal plate and strip surface for training the Faster R-CNN network; performing data enhancement on the acquired image of the surface of the metal plate strip; marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame; training a Faster R-CNN network according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame; the method comprises the steps of collecting a surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to a trained Faster R-CNN network and the collected surface image of the metal plate strip to be detected in real time, classifying and positioning the detected defects, meeting the requirements of industrial production and improving the accuracy of surface defect detection.
In a specific embodiment of the aforementioned metal plate strip surface defect detecting apparatus based on the fast R-CNN network, further, the enhancing module is configured to perform data enhancement on the acquired image of the metal plate strip surface by using a flip and translation strategy.
In a specific embodiment of the aforementioned metal plate strip surface defect detecting apparatus based on the fast R-CNN network, further, the acquiring module includes:
the marking unit is used for marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame;
the acquisition unit is used for acquiring the coordinates of the center point of the rectangular frame, the width and the height of the rectangular frame and the types of defects contained in the rectangular frame;
and the generation unit is used for generating a data set by the acquired image of the surface of the metal plate strip and the image obtained by enhancing the data, the acquired coordinates of the central point of the rectangular frame, the width and the height of the rectangular frame and the types of the defects contained in the rectangular frame.
In the foregoing specific embodiment of the device for detecting surface defects of metal strips based on the fast R-CNN network, further, the device further includes:
and the numbering module is used for modifying the total defect types which can be detected by the Faster R-CNN network according to the defect types to be detected and assigning a fixed number to each defect type to be detected.
In a specific embodiment of the aforementioned device for detecting surface defects of metal strips based on the Faster R-CNN network, the training module is further configured to train the Faster R-CNN network according to the generated data set.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (4)
1. A method for detecting surface defects of metal plates and strips based on a Faster R-CNN network is characterized by comprising the following steps:
collecting an image of the surface of a metal plate strip for training a Faster R-CNN network;
performing data enhancement on the acquired image of the surface of the metal plate strip; performing data enhancement on the acquired image of the surface of the metal plate strip through a turning and translation strategy;
marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame; modifying the total defect types which can be detected by the Faster R-CNN network according to the defect types to be detected, and allocating a fixed number to each defect type to be detected; wherein the defect categories include: roll marks, stains, scratches, holes, missing coating, depressions, bubbles, foreign matters and peeling;
generating a data set according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame, and training a Faster R-CNN network according to the data set;
and acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained Faster R-CNN network and the acquired surface image of the metal plate strip to be detected in real time, and classifying and positioning the detected defects.
2. The method for detecting the surface defects of the metal plate strip based on the Faster R-CNN network as claimed in claim 1, wherein the step of labeling the acquired images of the surface of the metal plate strip and the defects in the images obtained by enhancing the data with a rectangular frame, and the step of obtaining the coordinates of the rectangular frame and the types of the defects contained in the rectangular frame comprises the steps of:
marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the central point of the rectangular frame, the width and the height of the rectangular frame and the type of the defect contained in the rectangular frame;
and generating a data set by the acquired image of the surface of the metal plate strip, the image obtained after data enhancement, the acquired coordinates of the central point of the rectangular frame, the width and the height of the rectangular frame and the types of defects contained in the rectangular frame.
3. A metal plate belt surface defect detection device based on fast R-CNN network is characterized by comprising:
the acquisition module is used for acquiring images of the surface of the metal plate strip for training the Faster R-CNN network;
the enhancement module is used for carrying out data enhancement on the acquired image of the surface of the metal plate strip through a turning and translation strategy;
the acquisition module is used for marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame, and acquiring the coordinate of the rectangular frame and the type of the defect contained in the rectangular frame;
the numbering module is used for modifying the total defect types which can be detected by the Faster R-CNN network according to the defect types to be detected and distributing a fixed number for each defect type to be detected; wherein the defect categories include: roll marks, stains, scratches, holes, missing coating, depressions, bubbles, foreign matters and peeling;
the training module is used for generating a data set according to the acquired image of the surface of the metal plate strip and the image obtained after data enhancement, the acquired coordinates of the rectangular frame and the types of the defects contained in the rectangular frame, and training a Faster R-CNN network according to the data set;
and the detection module is used for acquiring the surface image of the metal plate strip to be detected in real time, detecting whether the surface of the metal plate strip has defects in real time according to the trained Faster R-CNN network and the acquired surface image of the metal plate strip to be detected in real time, and classifying and positioning the detected defects.
4. The device for detecting the surface defects of the metal plate strip based on the Faster R-CNN network as claimed in claim 3, wherein the acquiring module comprises:
the marking unit is used for marking the acquired image of the surface of the metal plate strip and the defect part in the image obtained after data enhancement by using a rectangular frame;
the acquisition unit is used for acquiring the coordinates of the center point of the rectangular frame, the width and the height of the rectangular frame and the types of defects contained in the rectangular frame;
and the generation unit is used for generating a data set by the acquired image of the surface of the metal plate strip and the image obtained by enhancing the data, the acquired coordinates of the central point of the rectangular frame, the width and the height of the rectangular frame and the types of the defects contained in the rectangular frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710110525.4A CN106952250B (en) | 2017-02-28 | 2017-02-28 | Metal plate strip surface defect detection method and device based on fast R-CNN network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710110525.4A CN106952250B (en) | 2017-02-28 | 2017-02-28 | Metal plate strip surface defect detection method and device based on fast R-CNN network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106952250A CN106952250A (en) | 2017-07-14 |
| CN106952250B true CN106952250B (en) | 2021-05-07 |
Family
ID=59468055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710110525.4A Active CN106952250B (en) | 2017-02-28 | 2017-02-28 | Metal plate strip surface defect detection method and device based on fast R-CNN network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106952250B (en) |
Families Citing this family (44)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107424150A (en) * | 2017-07-27 | 2017-12-01 | 济南浪潮高新科技投资发展有限公司 | A kind of road damage testing method and device based on convolutional neural networks |
| CN107657603B (en) * | 2017-08-21 | 2020-07-14 | 北京精密机电控制设备研究所 | Industrial appearance detection method based on intelligent vision |
| CN107561738B (en) * | 2017-08-30 | 2020-06-12 | 湖南理工学院 | Fast TFT-LCD surface defect detection method based on FCN |
| CN107607554A (en) * | 2017-09-26 | 2018-01-19 | 天津工业大学 | A kind of Defect Detection and sorting technique of the zinc-plated stamping parts based on full convolutional neural networks |
| CN108021926A (en) * | 2017-09-28 | 2018-05-11 | 东南大学 | A kind of vehicle scratch detection method and system based on panoramic looking-around system |
| CN108061735A (en) * | 2017-12-01 | 2018-05-22 | 工业互联网创新中心（上海）有限公司 | The recognition methods of component surface defect and device |
| CN107944505A (en) * | 2017-12-19 | 2018-04-20 | 青岛科技大学 | A kind of metal failure type automatization judgement method |
| CN110021005B (en) * | 2018-01-05 | 2022-03-15 | 财团法人工业技术研究院 | Method and device for screening defects of circuit board and computer readable recording medium |
| CN110111293B (en) * | 2018-01-29 | 2021-05-11 | 国科赛思(北京)科技有限公司 | Failure identification method and device for plastic package device |
| CN108564563A (en) * | 2018-03-07 | 2018-09-21 | 浙江大学 | A kind of tire X-ray defect detection method based on Faster R-CNN |
| CN108416294B (en) * | 2018-03-08 | 2022-04-01 | 上海天数智芯半导体有限公司 | Fan blade fault intelligent identification method based on deep learning |
| CN108492291B (en) * | 2018-03-12 | 2022-07-22 | 苏州天准科技股份有限公司 | CNN segmentation-based solar photovoltaic silicon wafer defect detection system and method |
| CN108520274B (en) * | 2018-03-27 | 2022-03-11 | 天津大学 | High-reflectivity surface defect detection method based on image processing and neural network classification |
| CN108562589B (en) * | 2018-03-30 | 2020-12-01 | 慧泉智能科技（苏州）有限公司 | Method for detecting surface defects of magnetic circuit material |
| CN108596883B (en) * | 2018-04-12 | 2021-07-13 | 福州大学 | Aerial image vibration damper slip fault diagnosis method based on deep learning and distance constraint |
| CN108830144B (en) * | 2018-05-03 | 2022-02-22 | 华南农业大学 | Lactating sow posture identification method based on improved Faster-R-CNN |
| CN108846841A (en) * | 2018-07-02 | 2018-11-20 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| CN108961238A (en) | 2018-07-02 | 2018-12-07 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| CN108921840A (en) * | 2018-07-02 | 2018-11-30 | 北京百度网讯科技有限公司 | Display screen peripheral circuit detection method, device, electronic equipment and storage medium |
| CN108986090A (en) * | 2018-07-11 | 2018-12-11 | 天津工业大学 | A kind of depth convolutional neural networks method applied to the detection of cabinet surface scratch |
| CN109115501A (en) * | 2018-07-12 | 2019-01-01 | 哈尔滨工业大学（威海） | A kind of Civil Aviation Engine Gas path fault diagnosis method based on CNN and SVM |
| CN109242830A (en) * | 2018-08-18 | 2019-01-18 | 苏州翔升人工智能科技有限公司 | A kind of machine vision technique detection method based on deep learning |
| CN109300114A (en) * | 2018-08-30 | 2019-02-01 | 西南交通大学 | The minimum target components of high iron catenary support device hold out against missing detection method |
| CN110570316A (en) * | 2018-08-31 | 2019-12-13 | 阿里巴巴集团控股有限公司 | method and device for training damage recognition model |
| CN109291657B (en) * | 2018-09-11 | 2020-10-30 | 东华大学 | Convolutional neural network-based aerospace structure industrial Internet of things identification laser coding system |
| CN109472769A (en) * | 2018-09-26 | 2019-03-15 | 成都数之联科技有限公司 | A kind of bad image defect detection method and system |
| CN109377483A (en) * | 2018-09-30 | 2019-02-22 | 云南电网有限责任公司普洱供电局 | Porcelain insulator crack detecting method and device |
| CN109544533B (en) * | 2018-11-23 | 2021-04-02 | 聚时科技(上海)有限公司 | Metal plate defect detection and measurement method based on deep learning |
| CN109360204B (en) * | 2018-11-28 | 2021-07-16 | 燕山大学 | Inner defect detection method of multilayer metal lattice structure material based on Faster R-CNN |
| CN109724984B (en) * | 2018-12-07 | 2021-11-02 | 上海交通大学 | Defect detection and identification device and method based on deep learning algorithm |
| CN109711474B (en) * | 2018-12-24 | 2023-01-17 | 中山大学 | Aluminum product surface defect detection algorithm based on deep learning |
| CN109934088A (en) * | 2019-01-10 | 2019-06-25 | 海南大学 | Sea ship discrimination method based on deep learning |
| CN109859207B (en) * | 2019-03-06 | 2023-06-23 | 华南理工大学 | Defect detection method of high-density flexible substrate |
| CN110175982B (en) * | 2019-04-16 | 2021-11-02 | 浙江大学城市学院 | Defect detection method based on target detection |
| CN110120035A (en) * | 2019-04-17 | 2019-08-13 | 杭州数据点金科技有限公司 | A kind of tire X-ray defect detection method differentiating defect grade |
| CN110335238A (en) * | 2019-05-08 | 2019-10-15 | 吉林大学 | A kind of Automotive Paint Film Properties defect recognition system constituting method of deep learning |
| CN110322430A (en) * | 2019-05-17 | 2019-10-11 | 杭州数据点金科技有限公司 | A kind of two stage tire X-ray defect detection method |
| CN110276754B (en) * | 2019-06-21 | 2021-08-20 | 厦门大学 | Surface defect detection method, terminal device and storage medium |
| CN110346953B (en) * | 2019-07-02 | 2022-08-09 | 晋城市龙鑫达光电科技有限公司 | Tearing detection system and method for stripping polaroid of liquid crystal display module |
| CN110782183A (en) * | 2019-11-07 | 2020-02-11 | 山东浪潮人工智能研究院有限公司 | Aluminum product defect detection method based on Faster R-CNN |
| CN111079820B (en) * | 2019-12-12 | 2020-08-25 | 哈尔滨市科佳通用机电股份有限公司 | Image recognition-based rail wagon fire-proof plate fault recognition method |
| CN113245238A (en) * | 2021-05-13 | 2021-08-13 | 苏州迪宏人工智能科技有限公司 | Elbow welding flaw detection method, device and system |
| CN113379745B (en) * | 2021-08-13 | 2021-11-30 | 深圳市信润富联数字科技有限公司 | Product defect identification method and device, electronic equipment and storage medium |
| CN116580030B (en) * | 2023-07-13 | 2023-10-20 | 厦门微图软件科技有限公司 | Welding quality anomaly detection method based on anomaly simulation |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424629B (en) * | 2013-08-19 | 2018-01-26 | 深圳先进技术研究院 | A kind of x-ray chest radiograph lung segmentation method and apparatus |
| CN103886610B (en) * | 2014-04-05 | 2016-08-17 | 东北电力大学 | A kind of insulator image defect detection method |
| CN104463104B (en) * | 2014-11-14 | 2018-06-08 | 武汉工程大学 | A kind of stationary vehicle target rapid detection method and device |
| CN104515786B (en) * | 2015-01-08 | 2018-05-11 | 北京科技大学 | The detection and analysis method that metal casting fatigue process internal flaw develops |
| US10043282B2 (en) * | 2015-04-13 | 2018-08-07 | Gerard Dirk Smits | Machine vision for ego-motion, segmenting, and classifying objects |
| CN106127173B (en) * | 2016-06-30 | 2019-05-07 | 北京小白世纪网络科技有限公司 | A kind of human body attribute recognition approach based on deep learning |
| CN106156744B (en) * | 2016-07-11 | 2019-01-29 | 西安电子科技大学 | SAR target detection method based on CFAR detection and deep learning |
-
2017
- 2017-02-28 CN CN201710110525.4A patent/CN106952250B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106952250A (en) | 2017-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106952250B (en) | Metal plate strip surface defect detection method and device based on fast R-CNN network | |
| CN106934800B (en) | Metal plate strip surface defect detection method and device based on YOLO9000 network | |
| CN111080622A (en) | Neural network training method, workpiece surface defect classification and detection method and device | |
| CN103439348B (en) | Remote controller key defect detection method based on difference image method | |
| CN113239930B (en) | Glass paper defect identification method, system, device and storage medium | |
| CN109142383B (en) | Character defect detection method based on morphology | |
| CN112053318A (en) | Two-dimensional PCB defect real-time automatic detection and classification device based on deep learning | |
| CN110443791B (en) | Workpiece detection method and device based on deep learning network | |
| CN113763312A (en) | Detecting defects in semiconductor samples using weak marks | |
| CN110490842A (en) | A kind of steel strip surface defect detection method based on deep learning | |
| CN113393426A (en) | Method for detecting surface defects of rolled steel plate | |
| Wang et al. | Attention-based deep learning for chip-surface-defect detection | |
| CN109711326A (en) | A kind of video object detection method based on shallow-layer residual error network | |
| CN112489042A (en) | Metal product printing defect and surface damage detection method based on super-resolution reconstruction | |
| CN115035092A (en) | Image-based bottle detection method, device, equipment and storage medium | |
| CN110426395B (en) | Method and device for detecting surface of solar EL battery silicon wafer | |
| CN113962951B (en) | Training method and device for detecting segmentation model, and target detection method and device | |
| CN112767345B (en) | DD6 monocrystal superalloy eutectic defect detection and segmentation method | |
| CN113705564A (en) | Pointer type instrument identification reading method | |
| CN108765391A (en) | A kind of plate glass foreign matter image analysis methods based on deep learning | |
| CN111707672A (en) | Method for detecting surface defects of wind power rotary supporting piece | |
| CN115601760A (en) | Defect evaluation method for first flexo printing piece | |
| CN109886912A (en) | A kind of thrust bearing retainer detection method of surface flaw | |
| CN113592853A (en) | Method for detecting surface defects of protofilament fibers based on deep learning | |
| CN113901947A (en) | Intelligent identification method for tire surface flaws under small sample |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |