CN112633327B - Staged metal surface defect detection method, system, medium, equipment and application - Google Patents
Staged metal surface defect detection method, system, medium, equipment and application Download PDFInfo
- Publication number
- CN112633327B CN112633327B CN202011395512.4A CN202011395512A CN112633327B CN 112633327 B CN112633327 B CN 112633327B CN 202011395512 A CN202011395512 A CN 202011395512A CN 112633327 B CN112633327 B CN 112633327B
- Authority
- CN
- China
- Prior art keywords
- layer
- areas
- training
- defect
- metal surface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of metal defect detection, and discloses a staged metal surface defect detection method, a staged metal surface defect detection system, a staged metal surface defect detection medium, a staged metal surface defect detection device and application, wherein the k-1 th and k-th layer characteristics of an image are extracted by utilizing a VGG (video graphics group) pre-training model; taking out defect areas in the k-th layer characteristics of all training samples, and determining the minimum characteristic value as a threshold value T; taking out the defect areas in the k-1 layer characteristics of all training samples by using a sliding window of n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas which are equivalent to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as a negative sample set; and sending the feature vector into an SVM, and training to obtain a classifier. The invention avoids the difficulty that the manual design features are not robust and time-consuming; meanwhile, coarse positioning is performed by utilizing deep abundant semantic features, fine positioning is performed by utilizing shallow abundant position features, and detection efficiency is improved under the condition of ensuring accuracy.
Description
Technical Field
The invention belongs to the technical field of metal defect detection, and particularly relates to a staged metal surface defect detection method, a system, a medium, equipment and application.
Background
At present: the quality of the steel is more and more important when the demand is gradually expanded. The surface of the steel plate is easy to be defective due to the influence of factors such as production equipment, steel plate feeding, processing technology, artificial technology, external environment and the like, the appearance, corrosion resistance, abrasion resistance, fatigue limit and the like of the plate are influenced, and even the normal use of the plate is influenced. The defect type (1) is mainly characterized in that the surface of the metal is provided with punctiform, flaky or strip-shaped structures, and the positions and colors of the metal are not specific rules. (2) Bubbles, round bulges with different sizes on the metal surface, can leave irregular gaps after cracking, and influence the appearance. (3) Cracks, cracks with different depths, lengths, widths and shapes are formed on the surface of the metal, severe decarburization phenomenon and partial inclusion are formed at the edge, the physical properties of the metal are seriously damaged, and serious safety accidents can be even caused.
In the early stages of industrial production, the demands on the product are not very high, and manual visual inspection is generally adopted. Since the detection speed and the detection range of the manual spot check method are very limited, and the subjective influence of a person is large, no definite standard exists, the method has been gradually replaced. With the progressive development of machine vision, vision-based solutions are also increasing, and are mainly divided into (1) traditional image methods based on artificial features, which represent a HOG (Histogram ofOriented Gradients) feature+ SVM (Support VectorMachine) detection method proposed by Dalal in 2005, and are initially widely applied to pedestrian detection. And then researchers also put forward HOG features and other artificial features of the improved version, such as LBP texture features and the like, so that the algorithm can adjust a corresponding feature extraction method according to a scene, and the generalization capability is enhanced to a certain extent. But still requires a great deal of effort from researchers to find the appropriate way of feature extraction. (2) The convolutional neural network-based target detection algorithm is certainly better than the computer vision algorithm that manually extracts feature descriptors. Such as the more mature single-stage object detection network YOLOv3, the two-stage object detection network faster-rcnn, etc. However, the target detection network based on the convolutional neural network generally needs a large amount of training data, and the overfitting phenomenon is easy to occur under a small data set. However, defects in industrial scenes tend to have low occurrence probability and high acquisition cost, and a large amount of defect data is difficult to collect.
Through the above analysis, the problems and defects existing in the prior art are as follows:
(1) The existing manual spot check mode is low in efficiency and is easily influenced by subjective factors of people, so that false detection is caused.
(2) The limitation of the traditional design features is too large, often related to specific scenes, and a large amount of effort is required by researchers to debug and continuously optimize, so that the generalization capability is weak, and a large amount of false detection is caused.
(3) The existing deep learning target detection method needs a large amount of sample data, causes an overfitting phenomenon based on the current data training, has poor performance on a verification data set, and has high hardware cost and low operation efficiency of the deep learning.
The difficulty of solving the problems and the defects is as follows: how to select a proper feature extraction mode and how to improve the detection efficiency.
The meaning of solving the problems and the defects is as follows: the metal defect detection can greatly improve the yield of steel or other metal elements, save a great deal of labor cost and promote the progress of the metal manufacturing field.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a staged metal surface defect detection method, a system, a medium, equipment and application.
The invention is realized in that a staged metal surface defect detection method comprises the following steps:
extracting the k-1 th and k-th layer characteristics of the image by utilizing a VGG pre-training model;
taking out defect areas in the k-th layer characteristics of all training samples, and determining the minimum characteristic value as a threshold value T;
taking out the defect areas in the k-1 layer characteristics of all training samples by using a sliding window of n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas which are equivalent to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as a negative sample set;
and sending the feature vector into an SVM, and training to obtain a classifier.
Further, the step of taking out the defective areas in the kth layer of features of all training samples, and the step of determining the minimum feature value as the threshold T specifically includes: depth features of the k-1 layer of each image in the data set extracted using VGG network are denoted as F k-1 And the depth characteristic of the kth layer is denoted as F k The method comprises the steps of carrying out a first treatment on the surface of the Only a feature extraction part is reserved by adopting a VGG16 network structure; the whole network consists of 13 convolution layers and 5 pooling layers, the size of the feature map of each pooling layer is reduced by 1/4 of that of the previous layer, the deeper the layer number, the richer the semantic features are, and the shallower the layer number, the richer the position information is; features were extracted using VGG pre-training model from Imagenet, let k=3.
Further to，F k-1 And F is equal to k All are three-dimensional matrixes, and each channel is summed to obtain a two-dimensional matrix which is respectively marked as F' k-1 With F' k The method comprises the steps of carrying out a first treatment on the surface of the The conversion formula is as follows:
wherein N is the number of characteristic channels of the k-1 layer, M is the number of characteristic channels of the k layer, and x and y are indexes of the first two dimensions of the characteristic diagram respectively.
Further, the defect area in the k layer characteristic of all training samples is taken out, the minimum characteristic value is defined as a threshold value T, and the minimum characteristic value is defined as F' k Extracting local feature graphs of all defect areas in the training set, and obtaining the minimum feature value in all the local feature graphs, namely a threshold value T;
1) The coordinates of the defect at the upper left corner and the lower right corner in the original image are represented by [ (x 1, y 1), (x 2, y 2) ], and the defect positions in the feature image are represented by [ (x 1', y 1'), (x 2', y 2') ] and have the following correspondence:
n'＝[n/2^k] n∈[x1,y1,x2,y2]；
k represents the number of feature layers selected, [ ] represents a downward rounding.
2) And storing the minimum value of each defect position in the feature map into a set S, and marking the minimum value in the final set S as a threshold value T.
Further, the step of extracting defective areas in the k-1 layer features of all training samples by using a sliding window of n×n, expanding the defective areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously extracting background areas corresponding to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a negative sample set specifically comprises the following steps: at F' k-1 Extracting local feature graphs of all defect areas in the training set to form a positive sample, extracting local feature graphs of partial background areas to form a negative sample, and sending the negative sample to an SVM to obtain a detector;
1) Selecting a sliding window s1 with n x n, sliding on an original image, wherein the sliding step length is n/2, and if the IOU of the s1 and the defect position is greater than 0.7, the sliding window is considered as a positive sample; if the IOU value is less than 0.3, the sliding window is considered a negative example. The definition of the two areas IOU is as follows:
2) Find out the sample at F' k-1 The characteristic patterns in the model (a) are expanded into one-dimensional vectors according to rows, samples with the same number as positive sample sets are randomly sampled from the negative sample sets to form new negative sample sets, and the new negative sample sets and the new positive sample sets are sent to SVM training to obtain classifier models;
3) Parameter setting of a trainer:
kernel: linear core
Penalty term C:1.7
Sliding window size: 32
Sliding step length: 16.
further, the staged metal surface defect detection method further comprises: in the test stage, the depth characteristic of the k-1 layer of each image in the VGG network extraction data set is repeatedly used and marked as F k-1 And the depth characteristic of the kth layer is denoted as F k And F k-1 And F is equal to k Are three-dimensional matrices, and each channel is summed to obtain a two-dimensional matrix which is the characteristic F 'of the k-1 layer respectively' k-1 And feature F 'of the kth layer' k The method comprises the steps of carrying out a first treatment on the surface of the Defining a two-dimensional matrix of the same size as the kth layer as D, matrix D reflecting the region of significance of the sample:
the salient region of D is mapped to region D 'on the k-1 layer feature in feature map F' k-1 In the D' region, extracting the characteristics by utilizing a sliding window with n x n, expanding the characteristics into a one-dimensional vector, and sending the one-dimensional vector into a training SVM (support vector machine) discriminator to judge whether the sliding window has defects or not; marking all sliding windows with defects, and combining the areas with excessive overlapped areas to finish the task of detecting the metal defects.
It is a further object of the present invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:
extracting the k-1 th and k-th layer characteristics of the image by utilizing a VGG pre-training model;
taking out defect areas in the k-th layer characteristics of all training samples, and determining the minimum characteristic value as a threshold value T;
taking out the defect areas in the k-1 layer characteristics of all training samples by using a sliding window of n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas which are equivalent to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as a negative sample set;
and sending the feature vector into an SVM, and training to obtain a classifier.
Another object of the present invention is to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
extracting the k-1 th and k-th layer characteristics of the image by utilizing a VGG pre-training model;
taking out defect areas in the k-th layer characteristics of all training samples, and determining the minimum characteristic value as a threshold value T;
taking out the defect areas in the k-1 layer characteristics of all training samples by using a sliding window of n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas which are equivalent to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as a negative sample set;
and sending the feature vector into an SVM, and training to obtain a classifier.
Another object of the present invention is to provide a metal defect detection information data processing terminal for implementing the staged metal surface defect detection method.
It is another object of the present invention to provide a staged metal surface defect detection system for implementing the staged metal surface defect detection method, the staged metal surface defect detection system comprising:
the image feature extraction module is used for extracting the k-1 th and k-th layer features of the image by utilizing the VGG pre-training model;
the defect region extraction module is used for extracting defect regions in the k-th layer characteristics of all training samples and determining the minimum characteristic value as a threshold value T;
the sample set forming module is used for taking out defect areas in k-1 layer characteristics of all training samples by using a sliding window of n x n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as positive sample sets, simultaneously taking out background areas with the same number as the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as negative sample sets;
the classifier acquisition module is used for sending the feature vector into the SVM and training to obtain the classifier.
By combining all the technical schemes, the invention has the advantages and positive effects that: the invention discloses a staged metal surface defect detection method based on VGG depth features and SVM, which is used for solving the problem of difficult feature selection in the existing defect detection method. According to the invention, the VGG pre-training model is adopted to extract the depth characteristics of the target, so that the difficulty that the characteristics of manual design are not robust and time-consuming is avoided; meanwhile, coarse positioning is performed by utilizing deep abundant semantic features, fine positioning is performed by utilizing shallow abundant position features, and detection efficiency is improved under the condition of ensuring accuracy. The SVM algorithm introduced into machine learning classifies the input features, and has stronger robustness than a neural network under the condition of small samples.
Compared with the prior art, the invention has the following advantages:
(1) According to the invention, the VGG network is introduced to extract the target depth characteristic, and the characteristic is not required to be designed manually.
(2) According to the invention, the SVM based on the structural risk minimization theory is adopted as the defect classifier, so that the input feature vector is accurately classified, the detection precision of the model is improved, and the risk of overfitting is reduced.
(3) The invention improves the detection efficiency by utilizing a staged detection method of deep layer characteristic coarse positioning and shallow layer characteristic fine positioning.
Table 1 comparison of the process of the invention with the prior art:
drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly explain the drawings needed in the embodiments of the present application, and it is obvious that the drawings described below are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for staged metal surface defect detection provided by an embodiment of the present invention.
FIG. 2 is a schematic diagram of a staged metal surface defect detection system according to an embodiment of the present invention;
in fig. 2: 1. an image feature extraction module; 2. a defect region extraction module; 3. a sample set forming module; 4. and a classifier acquisition module.
Fig. 3 is a flowchart of an implementation of a staged metal surface defect detection method provided by an embodiment of the present invention.
Fig. 4 is a schematic diagram of a VGG feature extraction structure according to an embodiment of the invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Aiming at the problems existing in the prior art, the invention provides a staged metal surface defect detection method, a staged metal surface defect detection system, a staged metal surface defect detection medium, a staged metal surface defect detection device and a staged metal surface defect detection application, and the invention is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the method for detecting the metal surface defects by stages provided by the invention comprises the following steps:
s101: extracting the k-1 th and k-th layer characteristics of the image by utilizing a VGG pre-training model;
s102: taking out defect areas in the k-th layer characteristics of all training samples, and determining the minimum characteristic value as a threshold value T;
s103: taking out the defect areas in the k-1 layer characteristics of all training samples by using a sliding window of n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas which are equivalent to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as a negative sample set;
s104: and sending the feature vector into an SVM, and training to obtain a classifier.
Those skilled in the art may also implement other steps in the method for detecting a staged metal surface defect provided by the present invention, and the method for detecting a staged metal surface defect provided by the present invention in fig. 1 is merely one specific embodiment.
As shown in fig. 2, the staged metal surface defect detection system provided by the present invention includes:
the image feature extraction module 1 is used for extracting the k-1 th and k-th layer features of the image by utilizing the VGG pre-training model;
the defect region extraction module 2 is used for extracting defect regions in the k-th layer of features of all training samples and determining the minimum feature value as a threshold value T;
the sample set forming module 3 is used for taking out defect areas in k-1 layer characteristics of all training samples by using a sliding window of n x n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as positive sample sets, simultaneously taking out background areas which are equivalent to the positive samples in number, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as negative sample sets;
and the classifier acquisition module 4 is used for sending the feature vector into the SVM and training to obtain a classifier.
The technical scheme of the invention is further described below with reference to the accompanying drawings.
As shown in fig. 3, the method for detecting the metal surface defects in stages according to the embodiment of the invention includes the following steps: the method specifically comprises the following steps:
step one, acquiring a certain number of metal image data with defects on the surface.
Step two, extracting depth characteristics of a k-1 layer of each image in the data set by utilizing a VGG network and marking the depth characteristics as F k-1 And the depth characteristic of the kth layer is denoted as F k The method comprises the steps of carrying out a first treatment on the surface of the With the VGG16 network architecture, only the feature extraction portion is retained, as shown in fig. 4. The whole network consists of 13 convolution layers and 5 pooling layers, the size of the feature map of each pooling layer is reduced by 1/4 of that of the previous layer, the deeper the layer number, the richer the semantic features are, and the shallower the layer number, the richer the position information is. The VGG pre-training model from Imagenet is used in the present invention to extract features, let k=3.
Step three, F k-1 And F is equal to k All are three-dimensional matrixes, and each channel is summed to obtain a two-dimensional matrix which is respectively marked as F' k-1 With F' k The method comprises the steps of carrying out a first treatment on the surface of the The conversion formula is as follows:
wherein N is the number of characteristic channels of the k-1 layer, M is the number of characteristic channels of the k layer, and x and y are indexes of the first two dimensions of the characteristic diagram respectively.
Step four, at F' k Extracting local feature graphs of all defect areas in the training set, and obtaining the minimum feature value in all the local feature graphs, namely a threshold value T;
1) The coordinates of the defect at the upper left corner and the lower right corner in the original image are represented by [ (x 1, y 1), (x 2, y 2) ], and the defect positions in the feature image are represented by [ (x 1', y 1'), (x 2', y 2') ] and have the following correspondence:
n'＝[n/2^k] n∈[x1,y1,x2,y2]；
k represents the number of feature layers selected, [ ] represents a downward rounding.
2) And storing the minimum value of each defect position in the feature map into a set S, and marking the minimum value in the final set S as a threshold value T.
Step fiveIn F' k-1 And extracting the local feature graphs of all the defect areas in the training set to form a positive sample, extracting the local feature graphs of part of the background areas to form a negative sample, and sending the negative sample to an SVM to obtain a detector.
1) Selecting a sliding window s1 with n x n, sliding on an original image, wherein the sliding step length is n/2, and if the IOU of the s1 and the defect position is greater than 0.7, the sliding window is considered as a positive sample; if the IOU value is less than 0.3, the sliding window is considered a negative example. The definition of the two areas IOU is as follows:
2) Find out the sample at F' k-1 And expands into a one-dimensional vector by rows, the dimensions of the vector naturally remain consistent since all samples are generated by sliding windows of the same size. In general, the number of positive examples will be much smaller than the number of negative examples, and the model detection performance obtained by unbalanced data training will be poor. The new negative sample set is formed by randomly sampling the same number of samples as the positive sample set from the negative sample set, and the new negative sample set and the positive sample set are sent to SVM training to obtain a classifier model.
3) Parameter setting of a trainer:
kernel: linear core
Penalty term C:1.7
Sliding window size: 32
Sliding step length: 16.
step six, in the test stage, repeating the step 2 and the step 3 to obtain the characteristic F 'of the k-1 layer' k-1 And feature F 'of the kth layer' k . Defining a two-dimensional matrix D with the same size as the k layer, wherein the matrix D reflects the salient region of the sample, namely the region with defects, so as to reduce the search range of a sliding window:
step sevenThe salient region of D of step six is mapped to region D' on the k-1 layer feature. In the feature map F' k-1 In the D' region of (2), the sliding window of n is utilized to take out the characteristics, the characteristics are expanded into one-dimensional vectors, and the one-dimensional vectors are sent into the SVM discriminator trained in the step 5, so that whether the sliding window has defects is judged. All defective sliding windows are marked, and the areas with excessive overlapping areas can be combined. Thus, the task of metal defect detection is completed.
The technical effects of the present invention will be described in detail with reference to simulation.
1. Simulation conditions
The simulation experiment of the invention is completed by applying Python language on a PC with CPU of Intel (R) Core (TM) i5-4590, CPU3.30GHz, RAM 16.00GB and Windows 10 operating system.
2. Simulation experiment contents
The training data of the invention are 200 single-channel metal images with defects, and the test data are 50 images with defects and 50 images without defects. In the SVM training stage, positive and negative samples are divided by taking a sliding window as a basic unit, and the positive samples (defects) are 276 samples according to the mode of the fifth step, wherein the number of the negative samples is far greater than that of the positive samples, so 276 samples are randomly extracted from the positive samples to serve as a negative sample set. The model performance was tested after training. In order to verify the effectiveness of the present invention, a comparison experiment was designed to compare with the method of the present invention, and the results are shown in table 1, except YOLOv3, which is a detection method based on a sliding window.
Table 1 comparison of the process of the invention with the prior art:
from the above results, it can be seen that the network-based feature extraction method (VGG) has higher detection accuracy than the conventional feature extraction methods (HOG, LBP), but takes a longer time. The invention greatly reduces the area to be detected by utilizing the strategy of staged detection, thereby reducing the detection time and improving the efficiency; the coarse positioning mode has obvious effect, and the detection precision is higher than that of the mode of checking all areas, which indicates that certain interference exists in the areas beyond the coarse positioning. Overall, the invention significantly improves the accuracy of metal defect detection.
It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.
Claims (6)
1. A staged metal surface defect detection method, the staged metal surface defect detection method comprising:
extracting the k-1 th and k-th layer characteristics of the image by utilizing a VGG pre-training model;
taking out defect areas in the k-th layer characteristics of all training samples, and determining the minimum characteristic value as a threshold value T;
taking out the defect areas in the k-1 layer characteristics of all training samples by using a sliding window of n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas which are equivalent to the number of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as a negative sample set;
the feature vector is sent to an SVM, and a classifier is obtained through training;
the step of taking out the defect areas in the k-th layer characteristics of all training samples, and the step of determining the minimum characteristic value as the threshold value T specifically comprises the following steps: depth features of the k-1 layer of each image in the data set extracted using VGG network are denoted as F k-1 And the depth characteristic of the kth layer is denoted as F k The method comprises the steps of carrying out a first treatment on the surface of the Only a feature extraction part is reserved by adopting a VGG16 network structure; the whole network consists of 13 convolution layers and 5 pooling layers, the size of the feature map of each pooling layer is reduced by 1/4 of that of the previous layer, the deeper the layer number, the richer the semantic features are, and the shallower the layer number, the richer the position information is; extracting features using a VGG pre-training model from Imagenet, let k=3;
F k-1 and F is equal to k All are three-dimensional matrixes, and each channel is summed to obtain a two-dimensional matrix which is respectively marked as F' k-1 With F' k The method comprises the steps of carrying out a first treatment on the surface of the The conversion formula is as follows:
wherein N is the number of characteristic channels of the k-1 layer, M is the number of characteristic channels of the k layer, and x and y are indexes of the first two dimensions of the characteristic diagram respectively;
the defective region in the k layer characteristic of all training samples is taken out, and the minimum characteristic value is defined as a threshold value T, namely F' k Extracting local feature graphs of all defect areas in the training set, and obtaining the minimum feature value in all the local feature graphs, namely a threshold T, wherein the method specifically comprises the following steps:
1) The coordinates of the defect at the upper left corner and the lower right corner in the original image are represented by [ (x 1, y 1), (x 2, y 2) ], and the defect positions in the feature image are represented by [ (x 1', y 1'), (x 2', y 2') ] and have the following correspondence:
n'＝[n/2^k] n∈[x1,y1,x2,y2]；
k represents the number of selected feature layers, [ ] represents a downward rounding;
2) Storing the minimum value of each defect position in the feature map into a set S, and marking the minimum value in the final set S as a threshold value T;
the staged metal surface defect detection method further comprises the steps of: in the test stage, the depth characteristic of the k-1 layer of each image in the VGG network extraction data set is repeatedly used and marked as F k-1 And the depth characteristic of the kth layer is denoted as F k ，F k-1 And F is equal to k All are three-dimensional matrixes, and each channel is summed to obtain a two-dimensional matrix which is respectively marked as F' k-1 With F' k The method comprises the steps of carrying out a first treatment on the surface of the Defining a two-dimensional matrix of the same size as the kth layer as D, matrix D reflecting the region of significance of the sample:
the salient region of D is mapped to region D 'on the k-1 layer feature in feature map F' k-1 In the D' region, extracting the characteristics by utilizing a sliding window with n x n, expanding the characteristics into a one-dimensional vector, and sending the one-dimensional vector into a training SVM (support vector machine) discriminator to judge whether the sliding window has defects or not; marking all sliding windows with defects, merging the areas with excessive overlapped areas, and completing the task of detecting the metal defects.
2. The method for detecting defects on a metal surface by stages according to claim 1, wherein the step of taking out the defective areas in the k-1 layer feature of all training samples by using a sliding window of n x n, expanding the defective areas into a one-dimensional vector by rows, marking the one-dimensional vector as a positive sample set, and simultaneously taking out the background areas corresponding to the number of the positive samples, expanding the background areas into a one-dimensional vector by rows, marking the one-dimensional vector as a negative sample set comprises the following steps: at F' k-1 Extracting local feature graphs of all defect areas in the training set to form a positive sample, extracting local feature graphs of partial background areas to form a negative sample, and sending the negative sample to an SVM to obtain a detector;
1) Selecting a sliding window s1 with n x n, sliding on an original image, wherein the sliding step length is n/2, and if the IOU of the s1 and the defect position is greater than 0.7, the sliding window is considered as a positive sample; if the IOU value is less than 0.3, the sliding window is considered as a negative example, and the definition of the two regions IOU is as follows:
2) Find out the sample at F' k-1 The characteristic patterns in the model (a) are expanded into one-dimensional vectors according to rows, samples with the same number as positive sample sets are randomly sampled from the negative sample sets to form new negative sample sets, and the new negative sample sets and the new positive sample sets are sent to SVM training to obtain classifier models;
3) Parameter setting of a trainer:
kernel: linear core
Penalty term C:1.7
Sliding window size: 32
Sliding step length: 16.
3. a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the staged metal surface defect detection method of any of claims 1-2.
4. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the staged metal surface defect detection method of any of claims 1-2.
5. A metal defect detection information data processing terminal, characterized in that the metal defect detection information data processing terminal is used for implementing the staged metal surface defect detection method according to any one of claims 1-2.
6. A staged metal surface defect detection system for implementing the staged metal surface defect detection method of any of claims 1-2, the staged metal surface defect detection system comprising:
the image feature extraction module is used for extracting the k-1 th and k-th layer features of the image by utilizing the VGG pre-training model;
the defect region extraction module is used for extracting defect regions in the k-th layer characteristics of all training samples and determining the minimum characteristic value as a threshold value T;
the sample set forming module is used for taking out defect areas in k-1 layer characteristics of all training samples by using a sliding window of n x n, expanding the defect areas into one-dimensional vectors according to rows, marking the one-dimensional vectors as positive sample sets, simultaneously taking out background areas with the same number as the positive samples, expanding the background areas into one-dimensional vectors according to rows, and marking the one-dimensional vectors as negative sample sets;
the classifier acquisition module is used for sending the feature vector into the SVM and training to obtain the classifier.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011395512.4A CN112633327B (en) | 2020-12-02 | 2020-12-02 | Staged metal surface defect detection method, system, medium, equipment and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011395512.4A CN112633327B (en) | 2020-12-02 | 2020-12-02 | Staged metal surface defect detection method, system, medium, equipment and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112633327A CN112633327A (en) | 2021-04-09 |
| CN112633327B true CN112633327B (en) | 2023-06-30 |
Family
ID=75307530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011395512.4A Active CN112633327B (en) | 2020-12-02 | 2020-12-02 | Staged metal surface defect detection method, system, medium, equipment and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112633327B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554631B (en) * | 2021-07-30 | 2024-02-20 | 西安电子科技大学 | Chip surface defect detection method based on improved network |
| CN114332495B (en) * | 2021-12-22 | 2024-10-15 | 携程旅游信息技术（上海）有限公司 | Image feature quantization method, system, device and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103996209A (en) * | 2014-05-21 | 2014-08-20 | 北京航空航天大学 | Infrared vessel object segmentation method based on salient region detection |
| RU2570592C1 (en) * | 2014-10-27 | 2015-12-10 | Алексей Николаевич Кузьмин | Method of detecting and analysing acoustic emission signals |
| CN105574527A (en) * | 2015-12-14 | 2016-05-11 | 北京工业大学 | Quick object detection method based on local feature learning |
| CN107491762A (en) * | 2017-08-23 | 2017-12-19 | 珠海安联锐视科技股份有限公司 | A kind of pedestrian detection method |
| CN108304873A (en) * | 2018-01-30 | 2018-07-20 | 深圳市国脉畅行科技股份有限公司 | Object detection method based on high-resolution optical satellite remote-sensing image and its system |
| CN109685030A (en) * | 2018-12-29 | 2019-04-26 | 哈尔滨理工大学 | A kind of mug rim of a cup defects detection classification method based on convolutional neural networks |
| CN109934147A (en) * | 2019-03-05 | 2019-06-25 | 北京联合大学 | Object detection method, system and device based on deep neural network |
| CN110084131A (en) * | 2019-04-03 | 2019-08-02 | 华南理工大学 | A kind of semi-supervised pedestrian detection method based on depth convolutional network |
| CN110276754A (en) * | 2019-06-21 | 2019-09-24 | 厦门大学 | A kind of detection method of surface flaw, terminal device and storage medium |
| CN110766643A (en) * | 2019-10-28 | 2020-02-07 | 电子科技大学 | Microaneurysm detection method facing fundus images |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| EP3633605A1 (en) * | 2017-05-22 | 2020-04-08 | Canon Kabushiki Kaisha | Information processing device, information processing method, and program |
| CN111209848A (en) * | 2020-01-03 | 2020-05-29 | 北京工业大学 | Real-time fall detection method based on deep learning |
| WO2020206861A1 (en) * | 2019-04-08 | 2020-10-15 | 江西理工大学 | Yolo v3-based detection method for key object at transportation junction |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10242266B2 (en) * | 2016-03-02 | 2019-03-26 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for detecting actions in videos |
| EP3596449A4 (en) * | 2017-03-14 | 2021-01-06 | University of Manitoba | Structure defect detection using machine learning algorithms |
| US10607119B2 (en) * | 2017-09-06 | 2020-03-31 | Kla-Tencor Corp. | Unified neural network for defect detection and classification |
-
2020
- 2020-12-02 CN CN202011395512.4A patent/CN112633327B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103996209A (en) * | 2014-05-21 | 2014-08-20 | 北京航空航天大学 | Infrared vessel object segmentation method based on salient region detection |
| RU2570592C1 (en) * | 2014-10-27 | 2015-12-10 | Алексей Николаевич Кузьмин | Method of detecting and analysing acoustic emission signals |
| CN105574527A (en) * | 2015-12-14 | 2016-05-11 | 北京工业大学 | Quick object detection method based on local feature learning |
| EP3633605A1 (en) * | 2017-05-22 | 2020-04-08 | Canon Kabushiki Kaisha | Information processing device, information processing method, and program |
| CN107491762A (en) * | 2017-08-23 | 2017-12-19 | 珠海安联锐视科技股份有限公司 | A kind of pedestrian detection method |
| CN108304873A (en) * | 2018-01-30 | 2018-07-20 | 深圳市国脉畅行科技股份有限公司 | Object detection method based on high-resolution optical satellite remote-sensing image and its system |
| CN109685030A (en) * | 2018-12-29 | 2019-04-26 | 哈尔滨理工大学 | A kind of mug rim of a cup defects detection classification method based on convolutional neural networks |
| CN109934147A (en) * | 2019-03-05 | 2019-06-25 | 北京联合大学 | Object detection method, system and device based on deep neural network |
| CN110084131A (en) * | 2019-04-03 | 2019-08-02 | 华南理工大学 | A kind of semi-supervised pedestrian detection method based on depth convolutional network |
| WO2020206861A1 (en) * | 2019-04-08 | 2020-10-15 | 江西理工大学 | Yolo v3-based detection method for key object at transportation junction |
| CN110276754A (en) * | 2019-06-21 | 2019-09-24 | 厦门大学 | A kind of detection method of surface flaw, terminal device and storage medium |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN110766643A (en) * | 2019-10-28 | 2020-02-07 | 电子科技大学 | Microaneurysm detection method facing fundus images |
| CN111209848A (en) * | 2020-01-03 | 2020-05-29 | 北京工业大学 | Real-time fall detection method based on deep learning |
Non-Patent Citations (4)
| Title |
|---|
| Clothing Images Attributes Classification Based on Deep Neural Network;Ning Lv 等;《2019 IEEE International Conference on Smart Internet of Things (SmartIoT)》;20191114;全文 * |
| Face Recognition Algorithm Based on VGG Network Model and SVM;Hongling Chen 等;《Journal of Physics: Conference Series》;20191231;第1229卷;第1-8页 * |
| 基于多特征的SVM多分类PCB焊点缺陷检测方法;Chen Shouhong 等;《激光杂志》;20191231;第21-26页 * |
| 改进的VGG网络的二极管玻壳图像缺陷检测;杜超等;《图学学报》;20191215(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112633327A (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111860171B (en) | Method and system for detecting irregular-shaped target in large-scale remote sensing image | |
| CN111753828B (en) | Natural scene horizontal character detection method based on deep convolutional neural network | |
| CN109190752A (en) | The image, semantic dividing method of global characteristics and local feature based on deep learning | |
| CN110569738B (en) | Natural scene text detection method, equipment and medium based on densely connected network | |
| CN111753986B (en) | Dynamic test method and device for deep learning model | |
| CN112633327B (en) | Staged metal surface defect detection method, system, medium, equipment and application | |
| CN109816634B (en) | Detection method, model training method, device and equipment | |
| CN106557740A (en) | The recognition methods of oil depot target in a kind of remote sensing images | |
| CN112749673A (en) | Method and device for intelligently extracting stock of oil storage tank based on remote sensing image | |
| CN111754502A (en) | Method for detecting surface defects of magnetic core based on fast-RCNN algorithm of multi-scale feature fusion | |
| CN111127454A (en) | Method and system for generating industrial defect sample based on deep learning | |
| CN117911350A (en) | PCB surface defect detection method based on improvement YOLOv-tiny | |
| CN116958052A (en) | Printed circuit board defect detection method based on YOLO and attention mechanism | |
| Li et al. | An improved PCB defect detector based on feature pyramid networks | |
| Jin et al. | PCB bare board defect detection based on improved YOLOv5s | |
| CN116630989A (en) | Visual fault detection method and system for intelligent ammeter, electronic equipment and storage medium | |
| Dong | A pixel-wise framework based on convolutional neural network for surface defect detection | |
| Xie et al. | Defect detection of printed circuit board based on small target recognition network | |
| CN111259974B (en) | Surface defect positioning and classifying method for small-sample flexible IC substrate | |
| CN113989742A (en) | Nuclear power station plant pedestrian detection method based on multi-scale feature fusion | |
| CN113496159A (en) | Multi-scale convolution and dynamic weight cost function smoke target segmentation method | |
| CN111369508A (en) | Defect detection method and system for metal three-dimensional lattice structure | |
| CN113095359B (en) | Method and system for detecting radiographic image marking information | |
| Poxi et al. | Bushing Surface Defect Detection Method Based on Improved YOLOX | |
| CN118279243A (en) | Flexible IC substrate surface defect identification method based on improved FCOS network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |