CN110853015A - Aluminum profile defect detection method based on improved Faster-RCNN - Google Patents

Aluminum profile defect detection method based on improved Faster-RCNN Download PDF

Info

Publication number
CN110853015A
CN110853015A CN201911099831.8A CN201911099831A CN110853015A CN 110853015 A CN110853015 A CN 110853015A CN 201911099831 A CN201911099831 A CN 201911099831A CN 110853015 A CN110853015 A CN 110853015A
Authority
CN
China
Prior art keywords
defect
aluminum profile
image
rcnn
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911099831.8A
Other languages
Chinese (zh)
Inventor
徐向纮
陈坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Jiliang University
Original Assignee
China Jiliang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Jiliang University filed Critical China Jiliang University
Priority to CN201911099831.8A priority Critical patent/CN110853015A/en
Publication of CN110853015A publication Critical patent/CN110853015A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30136—Metal

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Chemical & Material Sciences (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an aluminum profile defect detection method based on improved Faster-RCNN, which comprises the following steps: selecting a plurality of aluminum defect images, performing data enhancement aiming at the phenomenon of unbalanced distribution of a data set to set parameters of a Faster-RCNN model, preprocessing the aluminum defect images, training a defect data set on a Faster-RCNN network, storing a final detection model, and detecting the defect data of the aluminum profile by using the trained improved Faster-RCNN model. The improvement points of the invention on the fast-RCNN model are as follows: 1. aiming at the extreme length-width ratio phenomenon of the defects of the aluminum profile, a characteristic pyramid method is adopted for optimization; 2. aiming at the phenomenon of irregular defect shapes, deformation convolution is adopted to learn more defect shapes; 3. aiming at the phenomenon of feature loss of a small target in the feature extraction process, the survival rate of target defect features in a high-level network is improved by adopting hole convolution; 4. aiming at the phenomenon of inaccurate positioning of small defects, a ROI Pooling method is adopted to obtain more accurate positioning information.

Description

Aluminum profile defect detection method based on improved Faster-RCNN
Technical Field
The invention relates to the fields of computer vision, deep learning, pattern recognition and the like, in particular to an aluminum profile defect detection method based on an improved Faster-RCNN model.
Background
Aluminum profiles are widely used in the fields of transportation, construction, industrial manufacturing, and the like as one of important materials in modern industry, and the quality of the aluminum profiles needs to be detected before the aluminum profiles are put into use.
The existing aluminum profile quality detection method is to sample and detect products through manual visual comparison and hand touch feeling.
However, the above method has at least the following disadvantages: 1. the problem of easy fatigue exists in the manual work, and errors are easy to occur in the face of long-time detection; 2. the artificial judgment standard has subjectivity, and similar samples can be judged differently at different time intervals; 3. the manual detection efficiency is very low, and real-time detection cannot be carried out.
Disclosure of Invention
The invention provides a defect target detection method, which introduces a Faster-RCNN model and adopts various methods for improvement aiming at difficult samples in the defect detection process of aluminum profiles, and can solve the problems of poor reliability and low efficiency in manual detection of surface defects.
In order to achieve the purpose, the invention adopts the following technical scheme, and specifically comprises the following steps:
step S1: making a data set of a Faster-RCNN model, and specifically taking a standard format of Pascal Voc2007 as a template;
step S2: setting parameters of a fast-RCNN model;
step S3: preprocessing a defect image of the aluminum profile;
step S4: training an aluminum profile data set based on a fast-RCNN model, and optimizing the model by adopting various methods aiming at difficult samples;
step S5: inputting a test image, and detecting by using a trained improved Faster-RCNN model.
Further, the step S1 specifically includes:
step S11: shooting defect images of different aluminum profiles, including various single-defect images, composite multi-defect images and non-defect images, making a data set sample, and dividing a training data set and a test data set according to a proportion;
step S12: the data set samples are scaled to be uniform in size, a nearest neighbor interpolation method is adopted, and the following formula is adopted:
dsty＝dstw/srcw*srcy
dstx＝dsth/srch*srcx
f(dstx,dsty)＝f(srcx,srcy)
in the above formula, dstxRepresents the abscissa, dst, of the scaled imageyRepresenting the ordinate, dst, of the scaled imagewRepresenting the width, dst, of the scaled imagehRepresenting the height of the zoomed image, srcxRepresenting the abscissa, src, of the original imageyRepresenting the ordinate, src, of the original imagewWidth, src, of the original imagehRepresenting the height of the original image, f (dst)x,dsyy) A pixel value at a pixel point representing the scaled image, f (src)x,srcy) Representing original image pixel (src)x,srcy) The pixel value of (d);
step S13: acquiring the picture name, the defect type, the upper left corner coordinate and the lower right corner coordinate of the defect of the image data set sample by using a marking tool, and storing the acquired data into an xml file;
step S14: making the coordinate information generated in the step into an xml file according to an xml format in an options file of a Voc2007 data set;
step S15: correspondingly generating a training set train.txt, a verification set val.txt, a training verification set train.txt and a test set text.txt in the VOC2007 data set according to the xml file;
step S16: replacing the file in JPEGImages in the VOC2007 data set by using the image obtained after the data is augmented in the step S11, replacing the data in the options file in the VOC2007 data set by using the xml file obtained in the step 14, and replacing the txt file in ImageSets by using the txt file obtained in the step 15;
step S17: and downloading the pre-trained model parameters and placing the model parameters under a data folder.
The step S2 further includes:
step S21: modifying parameters related to the total number of categories in the Faster-RCNN model according to the sample distribution of the defect data set;
step S22: setting a class label of a fast-RCNN training model according to the class of the data set sample;
step S23: and performing other parameter configuration including iteration times and step size parameters.
The step S3 further includes:
step S31: reading in an image, including a picture name, coordinates and categories of a GT frame;
step S32: for defect images which are difficult to collect or have a small number of images, data amplification is carried out by adopting methods of mirroring, rotating, cutting, translating, adding Gaussian noise and adjusting image brightness and saturation, and image information after amplification is stored;
step S33, the test image is scaled to a uniform size by bilinear interpolation, and the following formula is adopted: p (x + u, y + V) ═ 1-u (1-V) p (x, y) + (1-u) vp (x, y +1) + u (1-V) p (x +1, y) + uvp (x +1, y +1)
Wherein v represents a floating point number with a vertical coordinate v, u represents a horizontal coordinate u, and the values of the floating point number and the horizontal coordinate u are both [0,1 ]; x and y respectively represent horizontal and vertical coordinates, and values are integers; the pixel values of the horizontal and vertical coordinates of the image are respectively the initial x and y are expressed by p (x, y).
The step S4 further includes:
step S41: downloading model parameters obtained by pre-training under the ImageNet data set, and carrying out migration learning initialization parameters;
step S42: inputting a test image, and extracting main features in the image by adopting a feature extraction network;
step S43: generating ROI areas, and distinguishing the ROI areas into a background and a foreground by using a classifier, wherein the ROI areas specifically comprise the following steps: the test image passes through a target detection model to be trained, and an interested area of a sample image containing a detection object is identified; comparing the area with a target area which is marked in advance in the image and contains the detection object, determining the overlapping degree between the detection area and the target area, marking the area with the overlapping degree smaller than a preset threshold value as a background area, and marking the area larger than the threshold value as a foreground area;
step S44: obtaining the scale of the zoom and the translation of the prediction frame, and using the following formula:
tw＝log(w/wb)
th＝log(h/hb)
ti＝(i-ib)/wb
tj＝(j-jb)/hb
wherein, wbWidth, h, of the anchorbHigh, t representing achroboxwRepresenting the scaling of the prediction box in height, thRepresenting the scaling scale of the prediction frame in height, i representing the central abscissa of the prediction frame, i.e. representing the central ordinate of the prediction frame, w representing the width of the prediction frame, h representing the high of the prediction frame, ibDenotes the central abscissa, j, of the anchorbDenotes the center ordinate, t, of the anchoriRepresenting the translation scale, t, of the prediction box on the abscissajRepresenting the translation scale of the prediction frame on the ordinate;
s45, obtaining the scaling and translation dimensions of the calibration frame by adopting the same method in the step S44;
step S46, correcting the position of the target through the translation scale and the scaling scale to obtain a suggestion frame, and meanwhile, removing the suggestion frame which is too small and exceeds the boundary according to the method in the step 43;
step S47: initializing the RPN by using parameters obtained by training the Faster-RCNN network, then obtaining the overall loss of the RPN, and finely adjusting the RPN, wherein the formula is as follows:
Lcls(pi,pi *)＝-log[pi *pi+(1-pi *)(1-pi)]
Lreg(ti,ti *)＝R(ti-ti *)
Figure RE-GDA0002303663240000041
in the above formula, i is an integer, piDenotes the probability, p, of the ith Anchor prediction being the targeti *Represents the GT prediction probability, t, corresponding to the ith anchori＝{tx,ty,tw,thDenotes a vector of four parameterized coordinates of the prediction box, ti *＝{tx *,ty *,tw *,th *Is the coordinate vector of the calibration frame corresponding to the positive anchor, Lcls(pi,pi *) Represents a classification loss, Lreg(ti,ti *) Representing the regression loss, R representing the Smooth L1 function, NclsNormalized value representing cls term is the size of mini-batch, NregIndicating the normalization of the reg term to the number of anchor positions, L ({ p)i},{ti}) represents a loss function;
step S48: generating an RPN network prediction frame of a defect image by using the training;
step S49: inputting the last layer of feature map generated by the convolutional layer, and normalizing each rectangular frame on the feature map by using an ROI Pooling layer; wherein the ROI Pooling back propagation formula is as follows:
Figure RE-GDA0002303663240000042
step S410: classifying the result after pooling by softmax, and calculating the probability of various categories; wherein, the softmax classification method comprises the following steps:
Figure RE-GDA0002303663240000043
for j＝1…k
wherein, sigma (z)jRepresenting the probability that the image belongs to the jth class;
step S411: aiming at the phenomenon that the details of the aluminum profile defect image are lost rapidly, the survival rate of the target features in a high-rise network is improved by adopting a hole convolution method, wherein the hole convolution calculation method comprises the following steps:
Figure RE-GDA0002303663240000044
in the above-mentioned hole convolution calculation, o represents the size of the characteristic diagram after hole convolution, i represents the size of the input hole convolution, s represents the step length, k is the size of the original convolution kernel, and the value of the hyper-parameter (d-1) is the number of the filled-in empty lattices;
step S412: calculating the translation and scaling scale again, and finely correcting the position to obtain a target detection frame;
step S413: changing the dynamic adjustment learning rate and the iteration times according to the training loss and the change of the accuracy, and retraining until reaching the accuracy threshold or ending when reaching the iteration times;
step S414: and obtaining an aluminum profile defect detection model, and storing the model in a model folder.
The step S5 further includes:
step S51: inputting a test image, and reading data in a BGR (texture mapping) format of the image;
step S52: performing feature extraction on the input image by using a convolution neural model to generate a feature map of a suspected defect image;
step S53: sliding on the feature map by using a sliding window to generate a plurality of suggestion areas;
step S54: mapping the plurality of suggested regions to the last layer of the convolutional layer, namely a full-link layer, classifying and calculating the probability that the image belongs to each class;
step S55: using a RoI Pooling layer to enable each output box to generate a defect image feature map with the same size;
step S56: and classifying the images according to the characteristic diagram and giving the coordinates of the defect positions.
Summarizing the steps: the method introduces a fast-RCNN model, and adopts a data amplification method to amplify data aiming at the data unbalance phenomenon of the defect data of the aluminum profile in training; aiming at the extreme length-width ratio phenomenon of the defect, FPN (feature pyramid network) is adopted to enhance the lower layer feature weight; aiming at the phenomenon that important features of a receptive field at the highest layer of a defect image are lost after the features of a basic model are extracted, the survival rate of target defects in a high-layer network is improved by adopting hole convolution; aiming at the condition that the false detection is caused by inaccurate positioning of the small defects, the ROI Align is adopted to replace the ROI Pooling positioning algorithm, and more accurate positioning information is obtained.
Compared with the prior art, the invention has the following advantages:
the invention adopts a data augmentation technology, and still has better detection effect under the condition of insufficient data quantity;
compared with the traditional machine learning method and the traditional machine vision method, the method does not need to perform specific characteristic extraction action, and has good applicability to the defects of dynamic variability and different forms;
compared with the traditional manual detection method, the method has obvious efficiency improvement, certain guarantee on accuracy rate and capability of rapidly detecting whether the defects exist or not and the type conditions;
the invention has good portability, can be embedded into the system after the model training is finished, can be combined with a defect detection system, and communicates the detected defect condition with a PLC or a mechanical arm to finish the operation of real-time classification;
the method has the advantages of on-line training and model updating, can complete the training of a new model in real time aiming at the defects of a new type, and can flexibly cope with different conditions;
the invention can be transplanted to be applied to the detection of similar defects, such as cloth defects, glass appearance defects and the like, and has certain reference significance.
Drawings
FIG. 1 is a schematic flow chart of the aluminum profile defect detection method based on the improved Faster-RCNN.
Detailed Description
The technical scheme of the invention is specifically explained below with reference to the accompanying drawings.
As shown in FIG. 1, the aluminum profile defect detection method based on improved Faster-RCNN comprises the following steps:
step S1: making an aluminum profile defect data set in a VOC2007 format:
the step S1 specifically includes:
step S11: downloading a VOC2007 data set and moving the data set to a data folder;
step S12: analyzing the distribution of samples aiming at the samples of the aluminum profile defect image data set, performing data enhancement on the samples with less quantity, including performing data enhancement by adopting methods of mirroring, rotating, cutting, translating, adding Gaussian noise and adjusting the brightness and saturation of the image, and moving the image into a JPEGImages folder;
step S13: labeling the data set by using a labeling tool, wherein the data set comprises the name of an image, the category of a target, an upper left corner coordinate and a lower right corner coordinate, generating xml files, and replacing all the xml files with files in an options folder;
step S14: correspondingly generating txt files with names of train, test, train val and val according to the generated xml file, and replacing the txt file in the Main folder in the VOC2007 data set by the four txt files;
step S15: model parameters are initialized, and a pre-trained model ResNet101 is downloaded and placed under the data folder.
Step S2: aluminum profile defect detection training based on improved Faster-RCNN;
the step S2 specifically includes:
step S21: initializing a feature extraction network, including initializing a ResNet101 network, improving an original fast-RCNN model aiming at an extreme length-width ratio phenomenon presented by a defect image, adding an FPN (feature pyramid) network, adding a deformation convolution method aiming at the characteristic of irregular shape of the defect image, and adding a cavity convolution method aiming at the phenomenon of quick loss of defect features;
step S22: configuring parameters and acquiring a training image; reading in image data according to a specified format, wherein the image data comprises a name, coordinates of a GT frame and category information;
step S23: initializing the RPN network by using ImageNet model parameters;
step S24: configuring model parameters including learning rate, iteration times and step length related parameters;
step S25: initially scaling the training images to a uniform size;
step S26: generating a prediction box using the RPN network;
step S27: reading GT frame and prediction frame information;
step S28: obtaining the scaling translation scale of the prediction frame and the calibration frame;
step S29: acquiring a relatively accurate prediction frame, and finely adjusting the RPN to generate a new prediction frame;
step S210: limiting the occurrence of a prediction box by using an NMS (non-maximum suppression) method to generate a simplified prediction box;
step S211: carrying out classification detection on the targets in the prediction frame;
step S212: and adjusting the learning rate and the iteration times, refining the target position, and then training again until the ideal accuracy is reached or the specified iteration times are reached.
Step S3: preprocessing a defect test image of the aluminum profile:
the step S3 specifically includes:
step S31: inputting a test image, and reading in image acquisition data;
step S32: normalizing the test image to a uniform size;
step S33: and adjusting the test image channel to the specified channel.
Step S4: testing an aluminum profile defect detection model based on improved Faster-RCNN;
the step S4 includes:
step S41: inputting a test image and zooming to a specified size;
step S42: carrying out feature extraction on the image by using a base network to generate a feature map;
step S43: mapping the high-dimensional data to a low-dimensional data;
step S44: generating a plurality of suggestion windows using an RPN network of NMSs (non-maximum suppression);
step S45: mapping the generated suggestion window to a full connection layer;
step S46: generating feature maps with the same size;
step S47: and (5) carrying out classification and position regression calculation on the feature map.

Claims (7)

1. An aluminum profile defect detection method based on improved Faster-RCNN is characterized by comprising the following steps:
(1) data acquisition: shooting a proper amount of aluminum profile defect images, including all defect types, and classifying the defect images according to a single-defect image, a multi-defect image and a non-defect image classification method;
(2) data processing: for the types with few defect samples, the number of the pictures is amplified by adopting a data enhancement method, and all the pictures are processed into a unified set size;
(3) label making: marking the defect picture by adopting a marking tool;
(4) model training: inputting the label data and the pictures into a fast-RCNN network for training by adopting a batch iteration mode to obtain a defect detection model;
(5) defect detection: and inputting the test picture into a detection model, and detecting the type probability and the position of the defect.
2. The aluminum profile defect detection method based on the improved Faster-RCNN according to the claim 1, characterized in that the step (1) uses an industrial camera to shoot the aluminum profile defects, so that the shooting angle is kept as vertical as possible, and the shot aluminum profile defect image should contain all defects as much as possible, including scratch, variegated color, orange peel, jet flow, paint bubble, crater, non-conducting, dirty point, corner leakage bottom and non-defective image as a data set sample.
3. The aluminum profile defect detection method based on the improved Faster-RCNN according to claim 1, characterized in that the step (2) is used for amplifying the data set sample by adopting methods including image translation, rotation, mirror image, cutting, scaling and noise addition for the aluminum profile defect image.
4. According to the model training method of step (4) in claim 1, in order to improve the accuracy of detecting the defect target by taking the aluminum profile as an object, the invention adopts the following methods to improve the fast-RCNN model:
(1) aiming at the phenomenon of the extreme length-width ratio of the defect image, improving the original model by adopting a characteristic pyramid network;
(2) aiming at the phenomenon of irregular shape of the defect image, improving the original model by adopting a deformable convolution method;
(3) aiming at the phenomenon of rapid loss of defects, an original model is improved by adopting a hole convolution method, wherein the calculation method of the hole convolution is as follows:
Figure FDA0002269502140000011
in the above-described hole convolution calculation, o represents the size of the feature map after hole convolution, i represents the size of the input hole convolution, s represents the step length, k represents the size of the original convolution kernel, and the value of the hyper-parameter (d-1) is the number of filled-in spaces.
5. The defect detection method according to step 4 in claim 1, inputting a final layer feature map generated by the convolutional layer, and normalizing each rectangular frame on the feature map by using a ROIPooling layer; wherein the ROIPooling back propagation formula is as follows:
Figure FDA0002269502140000021
6. classifying the result after pooling in the step 5 by softmax, and solving the probability of various categories; wherein, the softmax classification method comprises the following steps:
Figure FDA0002269502140000022
wherein, sigma (z)jIndicating the probability that the image belongs to class j.
7. The aluminum profile defect detection method based on improved Faster-RCNN as claimed in claim 1, wherein the step 5 comprises the following steps:
(1) inputting the aluminum profile defect image test set picture and the label into the trained model for detection;
(2) the detection output result is the probability that the detected image belongs to each type of defect, and the position of the defect type with the highest probability is automatically framed;
(3) after the detection is finished, the average precision and the call-back rate of all the test pictures are automatically calculated.
CN201911099831.8A 2019-11-12 2019-11-12 Aluminum profile defect detection method based on improved Faster-RCNN Pending CN110853015A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911099831.8A CN110853015A (en) 2019-11-12 2019-11-12 Aluminum profile defect detection method based on improved Faster-RCNN

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911099831.8A CN110853015A (en) 2019-11-12 2019-11-12 Aluminum profile defect detection method based on improved Faster-RCNN

Publications (1)

Publication Number Publication Date
CN110853015A true CN110853015A (en) 2020-02-28

Family

ID=69600075

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911099831.8A Pending CN110853015A (en) 2019-11-12 2019-11-12 Aluminum profile defect detection method based on improved Faster-RCNN

Country Status (1)

Country Link
CN (1) CN110853015A (en)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111523540A (en) * 2020-04-17 2020-08-11 中南大学 Metal surface defect detection method based on deep learning
CN111583198A (en) * 2020-04-23 2020-08-25 浙江大学 Insulator picture defect detection method combining FasterR-CNN + ResNet101+ FPN
CN111598861A (en) * 2020-05-13 2020-08-28 河北工业大学 Improved Faster R-CNN model-based non-uniform texture small defect detection method
CN111696077A (en) * 2020-05-11 2020-09-22 余姚市浙江大学机器人研究中心 Wafer defect detection method based on wafer Det network
CN111798447A (en) * 2020-07-18 2020-10-20 太原理工大学 Deep learning plasticized material defect detection method based on fast RCNN
CN111951212A (en) * 2020-04-08 2020-11-17 北京交通大学 Method for identifying defects of contact network image of railway
CN111950488A (en) * 2020-08-18 2020-11-17 山西大学 Improved fast-RCNN remote sensing image target detection method
CN112113978A (en) * 2020-09-22 2020-12-22 成都国铁电气设备有限公司 Vehicle-mounted tunnel defect online detection system and method based on deep learning
CN112508090A (en) * 2020-12-04 2021-03-16 重庆大学 External package defect detection method
CN112699498A (en) * 2021-03-23 2021-04-23 中国空气动力研究与发展中心计算空气动力研究所 Jet flow simulation shock wave rapid discrimination method based on discontinuity characteristics of normalized physical quantity
CN113012153A (en) * 2021-04-30 2021-06-22 武汉纺织大学 Aluminum profile flaw detection method
CN113256623A (en) * 2021-06-29 2021-08-13 南昌工程学院 FPC defect detection method based on improved MASK RCNN
CN113435409A (en) * 2021-07-23 2021-09-24 北京地平线信息技术有限公司 Training method and device of image recognition model, storage medium and electronic equipment
CN113469997A (en) * 2021-07-19 2021-10-01 京东科技控股股份有限公司 Method, device, equipment and medium for detecting plane glass
CN114397306A (en) * 2022-03-25 2022-04-26 南方电网数字电网研究院有限公司 Power grid grading ring hypercomplex category defect multi-stage model joint detection method
CN115272293A (en) * 2022-08-29 2022-11-01 新极技术(北京)有限公司 Strip steel surface defect detection method and system
CN116589171A (en) * 2023-07-14 2023-08-15 江西省博信玻璃有限公司 Intelligent tempering method and system with automatic glass detection function

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108334955A (en) * 2018-03-01 2018-07-27 福州大学 Copy of ID Card detection method based on Faster-RCNN
CN109299688A (en) * 2018-09-19 2019-02-01 厦门大学 Ship Detection based on deformable fast convolution neural network
CN109448365A (en) * 2018-10-16 2019-03-08 北京航空航天大学 Across the scale space base land regions road traffic system integrated supervision method of one kind
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
KR20190088089A (en) * 2017-12-26 2019-07-26 세종대학교산학협력단 Apparatus and method for detecting defects on welding surface
CN110264466A (en) * 2019-06-28 2019-09-20 广州市颐创信息科技有限公司 A kind of reinforcing bar detection method based on depth convolutional neural networks

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190088089A (en) * 2017-12-26 2019-07-26 세종대학교산학협력단 Apparatus and method for detecting defects on welding surface
CN108334955A (en) * 2018-03-01 2018-07-27 福州大学 Copy of ID Card detection method based on Faster-RCNN
CN109299688A (en) * 2018-09-19 2019-02-01 厦门大学 Ship Detection based on deformable fast convolution neural network
CN109448365A (en) * 2018-10-16 2019-03-08 北京航空航天大学 Across the scale space base land regions road traffic system integrated supervision method of one kind
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
CN110264466A (en) * 2019-06-28 2019-09-20 广州市颐创信息科技有限公司 A kind of reinforcing bar detection method based on depth convolutional neural networks

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
ROSS GIRSHICK: "Fast R-CNN", 《2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION》 *
SHAOQING REN ET AL.: "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 *
秦方: "基于计算机视觉的行人检测与人数统计算法研究", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》 *
陈久红， 张海玉: "基于深度学习的教室人数统计系统设计", 《软件导刊》 *
高志强 等: "《深度学习从入门到实践》", 30 June 2018, 中国铁道出版社 *

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111951212A (en) * 2020-04-08 2020-11-17 北京交通大学 Method for identifying defects of contact network image of railway
CN111523540A (en) * 2020-04-17 2020-08-11 中南大学 Metal surface defect detection method based on deep learning
CN111583198A (en) * 2020-04-23 2020-08-25 浙江大学 Insulator picture defect detection method combining FasterR-CNN + ResNet101+ FPN
CN111696077A (en) * 2020-05-11 2020-09-22 余姚市浙江大学机器人研究中心 Wafer defect detection method based on wafer Det network
CN111598861B (en) * 2020-05-13 2022-05-03 河北工业大学 Improved Faster R-CNN model-based non-uniform texture small defect detection method
CN111598861A (en) * 2020-05-13 2020-08-28 河北工业大学 Improved Faster R-CNN model-based non-uniform texture small defect detection method
CN111798447A (en) * 2020-07-18 2020-10-20 太原理工大学 Deep learning plasticized material defect detection method based on fast RCNN
CN111798447B (en) * 2020-07-18 2023-03-10 太原理工大学 Deep learning plasticized material defect detection method based on fast RCNN
CN111950488A (en) * 2020-08-18 2020-11-17 山西大学 Improved fast-RCNN remote sensing image target detection method
CN111950488B (en) * 2020-08-18 2022-07-19 山西大学 Improved Faster-RCNN remote sensing image target detection method
CN112113978A (en) * 2020-09-22 2020-12-22 成都国铁电气设备有限公司 Vehicle-mounted tunnel defect online detection system and method based on deep learning
CN112508090A (en) * 2020-12-04 2021-03-16 重庆大学 External package defect detection method
CN112699498B (en) * 2021-03-23 2021-05-25 中国空气动力研究与发展中心计算空气动力研究所 Jet flow simulation shock wave rapid discrimination method based on discontinuity characteristics of normalized physical quantity
CN112699498A (en) * 2021-03-23 2021-04-23 中国空气动力研究与发展中心计算空气动力研究所 Jet flow simulation shock wave rapid discrimination method based on discontinuity characteristics of normalized physical quantity
CN113012153A (en) * 2021-04-30 2021-06-22 武汉纺织大学 Aluminum profile flaw detection method
CN113256623A (en) * 2021-06-29 2021-08-13 南昌工程学院 FPC defect detection method based on improved MASK RCNN
CN113469997A (en) * 2021-07-19 2021-10-01 京东科技控股股份有限公司 Method, device, equipment and medium for detecting plane glass
CN113469997B (en) * 2021-07-19 2024-02-09 京东科技控股股份有限公司 Method, device, equipment and medium for detecting plane glass
CN113435409A (en) * 2021-07-23 2021-09-24 北京地平线信息技术有限公司 Training method and device of image recognition model, storage medium and electronic equipment
CN114397306A (en) * 2022-03-25 2022-04-26 南方电网数字电网研究院有限公司 Power grid grading ring hypercomplex category defect multi-stage model joint detection method
CN115272293A (en) * 2022-08-29 2022-11-01 新极技术(北京)有限公司 Strip steel surface defect detection method and system
CN116589171A (en) * 2023-07-14 2023-08-15 江西省博信玻璃有限公司 Intelligent tempering method and system with automatic glass detection function
CN116589171B (en) * 2023-07-14 2024-01-09 江西省博信玻璃有限公司 Intelligent tempering method and system with automatic glass detection function

Similar Documents

Publication Publication Date Title
CN110853015A (en) Aluminum profile defect detection method based on improved Faster-RCNN
CN109711474B (en) Aluminum product surface defect detection algorithm based on deep learning
CN108960245B (en) Tire mold character detection and recognition method, device, equipment and storage medium
CN110389127B (en) System and method for identifying metal ceramic parts and detecting surface defects
CN109859171B (en) Automatic floor defect detection method based on computer vision and deep learning
CN113160192B (en) Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background
CN112598001B (en) Automatic ship water gauge reading identification method based on multi-model fusion
CN107609525B (en) Remote sensing image target detection method for constructing convolutional neural network based on pruning strategy
CN111062915B (en) Real-time steel pipe defect detection method based on improved YOLOv3 model
CN111325203B (en) American license plate recognition method and system based on image correction
CN109509187B (en) Efficient inspection algorithm for small defects in large-resolution cloth images
CN109145830B (en) Intelligent water gauge identification method
CN113469177B (en) Deep learning-based drainage pipeline defect detection method and system
CN108918536B (en) Tire mold surface character defect detection method, device, equipment and storage medium
CN110927171A (en) Bearing roller chamfer surface defect detection method based on machine vision
CN108827969A (en) Metal parts surface defects detection and recognition methods and device
CN111127417B (en) Printing defect detection method based on SIFT feature matching and SSD algorithm improvement
CN112232263B (en) Tomato identification method based on deep learning
CN112907519A (en) Metal curved surface defect analysis system and method based on deep learning
CN108985337A (en) A kind of product surface scratch detection method based on picture depth study
CN113469951B (en) Hub defect detection method based on cascade region convolutional neural network
CN112365497A (en) High-speed target detection method and system based on Trident Net and Cascade-RCNN structures
CN112991271B (en) Aluminum profile surface defect visual detection method based on improved yolov3
CN106340007A (en) Image processing-based automobile body paint film defect detection and identification method
CN109584206B (en) Method for synthesizing training sample of neural network in part surface flaw detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200228