CN115147363A - Image defect detection and classification method and system based on deep learning algorithm - Google Patents

Image defect detection and classification method and system based on deep learning algorithm Download PDF

Info

Publication number
CN115147363A
CN115147363A CN202210726506.5A CN202210726506A CN115147363A CN 115147363 A CN115147363 A CN 115147363A CN 202210726506 A CN202210726506 A CN 202210726506A CN 115147363 A CN115147363 A CN 115147363A
Authority
CN
China
Prior art keywords
image
training
classification
defect detection
defect
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210726506.5A
Other languages
Chinese (zh)
Inventor
朱保明
许杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Nanovision Technology Co ltd
Original Assignee
Shenzhen Nanovision Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Nanovision Technology Co ltd filed Critical Shenzhen Nanovision Technology Co ltd
Priority to CN202210726506.5A priority Critical patent/CN115147363A/en
Publication of CN115147363A publication Critical patent/CN115147363A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The application discloses an image defect detection and classification method and system based on a deep learning algorithm, wherein the method comprises the following steps: acquiring images to obtain training images, wherein the training images comprise qualified samples and unqualified samples; classifying the training images to obtain classification types, wherein the classification types comprise qualified sample types and unqualified sample types, and the unqualified sample types comprise defect types; preprocessing the training images, and obtaining an image data set according to the image data and the classification type of the preprocessed training images; training through an image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model; and when the image to be detected is obtained, processing the image to be detected through the image defect detection and classification model to obtain the defect detection and classification result of the image to be detected. The defect detection and classification difficulty is reduced, and the defect detection and classification accuracy is improved.

Description

Image defect detection and classification method and system based on deep learning algorithm
Technical Field
The application relates to the field of image defect detection, in particular to an image defect detection and classification method and system based on a deep learning algorithm.
Background
The image defect detection is the most complex and difficult requirement in the machine vision requirement, and mainly needs to ensure the stability and precision of detection and realize the universality of defect detection in the project development process, common image defects comprise concave-convex defects, stain defects, scratches and the like, and the types of the image defects are various. The image defect detection mainly aims at industrial scenes, and the requirements and key points of different industries are different, so that defect algorithms used by different industries are quite different, and the image defect detection is bound by an industrial scene non-standard system.
The existing image defect detection implementation process comprises the following steps: 1. performing Blob feature extraction, wherein the adopted image processing technology comprises image segmentation, morphological operation, connectivity analysis, feature value calculation and scene description; 2. carrying out template matching and searching for similar parts; 3. the luminosity stereo detects the notch or dent on the surface of the object by utilizing shadow; 4. measuring and fitting, namely measuring the size of a target object, measuring the distance and detecting the integrity of the object to realize defect detection; 5. machine learning, designing and analyzing algorithms that allow computers to learn automatically, object detection and object recognition.
The image preprocessing steps in the existing image defect detection method are various and have strong pertinence and poor robustness; the multiple algorithms are large in calculation amount and cannot accurately detect the size and the shape of the defect, the debugging difficulty is large, the detection algorithm is unstable after the image is changed, and then parameters need to be adjusted repeatedly, the probability of false detection of complex defects is high, and the compatibility is poor. Making defect detection and classification difficult and inaccurate.
Disclosure of Invention
In order to solve the problems of high difficulty and inaccuracy of image defect detection and classification, the application provides an image defect detection and classification method based on a deep learning algorithm.
In a first aspect, the present application provides an image defect detection and classification method based on a deep learning algorithm, which adopts the following technical scheme:
acquiring an image to obtain a training image, wherein the training image comprises a qualified sample and an unqualified sample;
classifying the training images to obtain classification types, wherein the classification types comprise qualified sample types and unqualified sample types, and the unqualified sample types comprise defect types;
preprocessing the training images, and obtaining an image data set according to the image data and classification types of the preprocessed training images;
training through an image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model;
and when the image to be detected is obtained, processing the image to be detected through the image defect detection and classification model to obtain the defect detection and classification result of the image to be detected.
Optionally, classifying the training images to obtain classification types, including:
classifying the training images according to a qualified standard to obtain a qualified sample type or an unqualified sample type;
obtaining an original image corresponding to the unqualified sample type in the training image, and processing the defects in the original image to obtain a result image;
classifying the result graph according to preset defect types, and determining the defect types corresponding to the unqualified sample types, wherein the defect types comprise white foreign matters, black foreign matters, peeling, scratches, bridges, plating leakage and heterochrosis;
and synthesizing the qualified sample types or the defect types corresponding to the unqualified sample types of all the training images to obtain the classification types.
Optionally, the preprocessing is performed on the training image, and an image data set is obtained according to the image data and the classification type of the preprocessed training image, where the method includes:
preprocessing the training image by an image preprocessing technology to obtain image data of the preprocessed training image, wherein the image preprocessing technology comprises turning transformation, translation transformation, scale transformation, contrast transformation, noise disturbance, rotation transformation and segmentation;
and obtaining an image data set according to the image data and the classification type of the preprocessed training image.
Optionally, training is performed through an image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model, including:
dividing an image data set into a training set and a verification set;
constructing and obtaining an initial convolutional neural network model by adopting a convolutional neural network based on a deep learning algorithm;
and training the initial convolutional neural network model by setting hyper-parameters and utilizing a training set to obtain an image defect detection and classification model.
Optionally, the hyper-parameters include learning rate, iteration number and training batch size,
through setting up the hyper-parameter, utilize the training set to train initial convolution neural network model, obtain image defect detection and classification model, include:
setting a learning rate according to a gradient descent method;
setting the training times of a training set according to a preset training standard to obtain iteration times;
setting the size of a training batch according to the memory requirement, wherein the size of the training batch is the number of images for each training;
and inputting the training set into the initial convolutional neural network model for training according to the learning rate, the iteration times and the training batch size to obtain an image defect detection and classification model.
Optionally, the training set is input to the initial convolutional neural network model for training according to the learning rate, the iteration number and the training batch size, so as to obtain an image defect detection and classification model, which includes:
selecting a single training set with the corresponding number of images from the training set according to the size of the training batch;
according to image data and classification categories of training images corresponding to a single training set, presetting N anchor frames with different sizes, and setting N corresponding labels, wherein N is a positive integer greater than or equal to 1;
inputting the single training set into an initial convolutional neural network model for training to obtain N prediction frames with different scales, wherein the scale of each prediction frame is the same as that of a corresponding anchor frame;
calculating to obtain total loss values of all anchor frames and corresponding prediction frames;
and performing iterative training on the initial convolutional neural network model according to the total loss value, the learning rate and the iteration times to obtain an image defect detection and classification model.
Optionally, the calculating to obtain the total loss values of all anchor frames and corresponding prediction frames includes:
obtaining the predicted coordinate data and the predicted defect coordinate data of the target prediction frame, wherein the predicted coordinate data comprises a predicted central coordinate value (a)
Figure 100002_DEST_PATH_IMAGE001
) Predicting width coordinate value
Figure 100002_DEST_PATH_IMAGE002
And predicting the height coordinate value
Figure 100002_DEST_PATH_IMAGE003
；
Acquiring actual coordinate data and actual defect coordinate data of an anchor frame corresponding to the target prediction frame, wherein the actual coordinate data comprises an actual central coordinate value (
Figure 100002_DEST_PATH_IMAGE004
) Actual width coordinate value
Figure 100002_DEST_PATH_IMAGE005
And actual height coordinate value
Figure 100002_DEST_PATH_IMAGE006
；
Obtaining the width and height coordinate error of the defect according to the predicted defect coordinate data and the actual defect coordinate data
Figure 100002_DEST_PATH_IMAGE007
；
Obtaining a prediction confidence for a target prediction box
Figure 100002_DEST_PATH_IMAGE008
And actual confidence of corresponding anchor frame
Figure 100002_DEST_PATH_IMAGE009
；
Substituting the predicted coordinate data and the actual coordinate data into a coordinate error loss function
Figure 100002_DEST_PATH_IMAGE010
And calculating to obtain the coordinate loss value,
Figure 100002_DEST_PATH_IMAGE011
representing division of a training image into
Figure 100002_DEST_PATH_IMAGE012
A grid of a plurality of grids, each grid having a grid,
Figure 100002_DEST_PATH_IMAGE013
representing each mesh generation
Figure 41618DEST_PATH_IMAGE013
Training candidate frames of the anchor frames to obtain outsourcing rectangular frames;
substituting defect width and height coordinate errors, prediction confidence coefficient and actual confidence coefficient into a confidence coefficient error loss function
Figure 100002_DEST_PATH_IMAGE014
Calculating to obtain a confidence coefficient loss value;
introducing defect width and height coordinate errors into classification error loss function
Figure 100002_DEST_PATH_IMAGE015
Calculating to obtain a classification error loss value;
and calculating the sum of the coordinate loss value, the confidence coefficient loss value and the classification error loss value to obtain a total loss value.
Optionally, the method further comprises:
verifying the accuracy of the defect detection and classification results of the image defect detection and classification model through a verification set;
when the accuracy is not lower than a preset threshold value, determining that the verification is passed;
and when the accuracy is lower than a preset threshold value, determining that the verification is not passed.
In a second aspect, the present application provides an image defect detecting and classifying system based on a deep learning algorithm, which adopts the following technical scheme:
the training image acquisition module is used for acquiring images to obtain training images, and the training images comprise qualified samples and unqualified samples;
the training image classification module is used for classifying the training images to obtain classification types, wherein the classification types comprise a qualified sample type and an unqualified sample type, and the unqualified sample type comprises a defect type;
the training image processing module is used for preprocessing the training images and obtaining an image data set according to the image data and the classification types of the preprocessed training images;
the model training module is used for training through an image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model;
and the defect detection and classification module is used for processing the image to be detected through the image defect detection and classification model when the image to be detected is obtained, so as to obtain the defect detection and classification result of the image to be detected.
In summary, the present application includes the following advantageous technical effects:
the training image is required to be preprocessed firstly, an image data set is obtained according to the image data and classification type of the preprocessed training image, then the image data set and a deep learning algorithm convolutional neural network are trained to obtain an image defect detection and classification model, when the image to be detected is required to be subjected to defect detection and classification, the image to be detected is only required to be input into the image defect detection and classification model to output a defect detection and classification result, image preprocessing and algorithm calculation are not required, and the difficulty of defect detection and classification is reduced; and in the training process of the image defect detection and classification model, a deep learning algorithm convolutional neural network is adopted, and model parameters do not need to be adjusted repeatedly according to image changes, so that the defect detection and classification are more accurate.
Drawings
Fig. 1 is a schematic flowchart of an image defect detection and classification method based on a deep learning algorithm according to the present application.
Fig. 2 is a schematic flowchart of classifying training images according to the present application.
Fig. 3 is a flowchart illustrating a training process of the image defect detection and classification model according to the present application.
Fig. 4 is a schematic structural diagram of the image defect detecting and classifying system based on the deep learning algorithm according to the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The embodiment of the application discloses an image defect detection and classification method based on a deep learning algorithm.
Referring to fig. 1, the method includes:
and 101, acquiring an image to obtain a training image.
When image acquisition is performed in a specific application scene, the light source is controlled to be turned on and an image acquisition signal is sent to the camera, and product images are acquired through the camera.
And 102, classifying the training images to obtain classification types.
The process of classifying the training images is shown in fig. 2, and the specific steps include:
201, classifying the training images according to a qualified standard to obtain a qualified sample type or an unqualified sample type;
for the training images, the qualified products are marked as qualified samples, and the unqualified products are marked as unqualified samples, so that one training image can be classified into a qualified sample type or an unqualified sample type;
202, obtaining an original image corresponding to the unqualified sample type in the training image, and processing the defects in the original image to obtain a result image;
defects in the original image of an unqualified sample type in the training image may not be obvious, so that highlighting processing needs to be carried out to obtain a result image;
203, classifying the result graph according to the preset defect type, determining a defect type corresponding to the unqualified sample type;
counting defects possibly existing in the product to form preset defect types, and classifying the result chart according to the preset defect types, wherein the defect types comprise white foreign matters, black foreign matters, peeling, scratches, bridges, plating leakage and different colors;
and 204, synthesizing the defect types corresponding to the qualified sample types or the unqualified sample types of all the training images to obtain a classification type.
And 103, preprocessing the training image, and obtaining an image data set according to the image data and the classification type of the preprocessed training image.
Preprocessing a training image by an image preprocessing technology to obtain image data of the preprocessed training image, wherein the image preprocessing technology comprises turning transformation, translation transformation, scale transformation, contrast transformation, noise disturbance, rotation transformation and segmentation; and obtaining an image data set according to the image data and the classification type of the preprocessed training image. Image pre-processing is to increase the training data set and thereby reduce model overfitting.
And 104, training through an image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model.
The training process of the image defect detection and classification model is shown in fig. 3, and the specific steps include:
301, dividing an image data set into a training set and a verification set;
the proportion of the training set to the validation set is 8;
302, constructing and obtaining an initial convolutional neural network model by adopting a convolutional neural network based on a deep learning algorithm;
deep learning is a general term of a type of pattern analysis method, and a typical deep learning algorithm is a Convolutional Neural Network (CNN) to construct an initial Convolutional Neural Network model;
303, training the initial convolutional neural network model by setting hyper-parameters and utilizing a training set to obtain an image defect detection and classification model.
In the training process, a hyper-parameter is required to be set to train the model, the hyper-parameter comprises a learning rate, iteration times and a training batch size, the learning rate is set according to a gradient descent method, the training times of a training set are set according to a preset training specification to obtain the iteration times, the training batch size is set according to the memory requirement, the training batch size is the number of images trained each time, as the larger the batch size is, the faster the convergence is, but the larger the memory consumption is, the proper batch size is required to be selected according to the memory requirement;
selecting a single training set with the corresponding number of images from the training set according to the size of the training batch;
according to image data and classification categories of training images corresponding to a single training set, presetting N anchor frames with different sizes, and setting N corresponding labels, wherein N is a positive integer greater than or equal to 1, and N =3 can be set in practical application;
inputting the single training set into an initial convolutional neural network model for training to obtain N prediction frames with different scales, wherein the scale of each prediction frame is the same as that of a corresponding anchor frame;
obtaining the predicted coordinate data and the predicted defect coordinate data of the target prediction frame, wherein the predicted coordinate data comprises a predicted central coordinate value (a)
Figure DEST_PATH_IMAGE016
) Predicting width coordinate value
Figure DEST_PATH_IMAGE017
And predicting the height coordinate value
Figure DEST_PATH_IMAGE018
；
Acquiring actual coordinate data and actual defect coordinate data of an anchor frame corresponding to the target prediction frame, wherein the actual coordinate data comprises an actual central coordinate value (a)
Figure 805044DEST_PATH_IMAGE004
) Actual width coordinate value
Figure DEST_PATH_IMAGE019
And actual height coordinate value
Figure DEST_PATH_IMAGE020
；
Obtaining the defect width and height coordinate error according to the predicted defect coordinate data and the actual defect coordinate data
Figure 796658DEST_PATH_IMAGE007
；
Obtaining a prediction confidence for a target prediction box
Figure 938926DEST_PATH_IMAGE008
And the actual confidence of the corresponding anchor frame
Figure 823706DEST_PATH_IMAGE009
；
Substituting the predicted coordinate data and the actual coordinate data into a coordinate error loss function
Figure 418635DEST_PATH_IMAGE010
And calculating to obtain the coordinate loss value,
Figure DEST_PATH_IMAGE021
representing division of a training image into
Figure 7748DEST_PATH_IMAGE012
A grid of a plurality of grids, each grid having a grid,
Figure 879889DEST_PATH_IMAGE013
representing each mesh generation
Figure 466728DEST_PATH_IMAGE013
Training candidate frames of the anchor frames to obtain outsourcing rectangular frames;
substituting defect width and height coordinate errors, prediction confidence coefficient and actual confidence coefficient into a confidence coefficient error loss function
Figure DEST_PATH_IMAGE022
Calculating to obtain a confidence coefficient loss value;
introducing defect width and height coordinate errors into classification error loss function
Figure DEST_PATH_IMAGE023
The classification error loss value is obtained through calculation, and the cross entropy function is used, so that compared with the traditional sigmoid function, the cross entropy function has the advantages of fast convergence and fast weight updating of the convolution layer;
calculating the sum of the coordinate loss value, the confidence coefficient loss value and the classification error loss value to obtain a total loss value;
and performing iterative training on the initial convolutional neural network model according to the total loss value, the learning rate and the iteration times to obtain an image defect detection and classification model, wherein the iteration times reach the last time when the training is finished and the labeling is finished.
And 105, when the image to be detected is obtained, processing the image to be detected through the image defect detection and classification model to obtain the defect detection and classification result of the image to be detected.
After the training of the image defect detection and classification model is completed, when the image to be detected is acquired and needs to be subjected to defect detection and classification, the image to be detected is subjected to defect detection and classification through the image defect detection and classification model, and a defect detection and classification result of the image to be detected is obtained.
The implementation principle of the embodiment is as follows:
the training image is required to be preprocessed firstly, an image data set is obtained according to the image data and classification type of the preprocessed training image, then the image data set and a deep learning algorithm convolutional neural network are trained to obtain an image defect detection and classification model, when the image to be detected is required to be subjected to defect detection and classification, the image to be detected is only required to be input into the image defect detection and classification model to output a defect detection and classification result, image preprocessing and algorithm calculation are not required, and the difficulty of defect detection and classification is reduced; and in the training process of the image defect detection and classification model, a deep learning algorithm convolutional neural network is adopted, and model parameters do not need to be adjusted repeatedly according to image changes, so that the defect detection and classification are more accurate.
In the above embodiment shown in fig. 1, after the image defect detection and classification model training is completed, model verification is further performed, which includes:
verifying the accuracy of the defect detection and classification results of the image defect detection and classification model through a verification set;
when the accuracy is not lower than a preset threshold value, determining that the verification is passed;
and when the accuracy is lower than a preset threshold value, determining that the verification is not passed.
The implementation principle of the embodiment is as follows: the images in the verification set are input into the image defect detection and classification model one by one, defect detection and classification results are output, actual defects and classification of the images in the verification set are compared, the numerical value of the accuracy is determined, the numerical value is compared with the preset threshold, in order to guarantee practicability, the preset threshold can be set to be 90% higher, the verification is passed only when the accuracy is not lower than 90%, and the verification is not passed when the accuracy is lower than 90%.
The above is an embodiment of the image defect detecting and classifying method based on the deep learning algorithm of the present application, and an image defect detecting and classifying system based on the deep learning algorithm is described below by the embodiment. As shown in fig. 4, the present application provides an image defect detecting and classifying system based on a deep learning algorithm, which includes:
a training image acquisition module 401, configured to acquire a training image by performing image acquisition, where the training image includes a qualified sample and an unqualified sample;
a training image classification module 402, configured to classify the training images to obtain classification types, where the classification types include a qualified sample type and an unqualified sample type, and the unqualified sample type includes a defect type;
a training image processing module 403, configured to pre-process a training image, and obtain an image data set according to image data and classification types of the pre-processed training image;
the model training module 404 is configured to train through an image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model;
and the defect detecting and classifying module 405 is configured to, when the image to be detected is obtained, process the image to be detected through the image defect detecting and classifying model to obtain a defect detecting and classifying result of the image to be detected.
According to the image defect detection and classification system based on the deep learning algorithm, the training image needs to be preprocessed, an image data set is obtained according to the image data and the classification type of the preprocessed training image, the image data set and the deep learning algorithm convolutional neural network are trained to obtain an image defect detection and classification model, when the image to be detected needs to be detected to be subjected to defect detection and classification, only the image to be detected needs to be input into the image defect detection and classification model, the defect detection and classification result is output, image preprocessing and algorithm calculation are not needed, and the difficulty of defect detection and classification is reduced; and in the training process of the image defect detection and classification model, a deep learning algorithm convolutional neural network is adopted, and model parameters do not need to be adjusted repeatedly according to image changes, so that the defect detection and classification are more accurate.
The foregoing is a preferred embodiment of the present application and is not intended to limit the scope of the application in any way, and any features disclosed in this specification (including the abstract and drawings) may be replaced by alternative features serving equivalent or similar purposes, unless expressly stated otherwise. That is, unless expressly stated otherwise, each feature is only an example of a generic series of equivalent or similar features.

Claims (9)

1. An image defect detection and classification method based on a deep learning algorithm is characterized by comprising the following steps:
acquiring an image to obtain a training image, wherein the training image comprises a qualified sample and an unqualified sample;
classifying the training images to obtain classification types, wherein the classification types comprise qualified sample types and unqualified sample types, and the unqualified sample types comprise defect types;
preprocessing the training image, and obtaining an image data set according to the preprocessed image data of the training image and the classification type;
training through the image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model;
and when the image to be detected is obtained, processing the image to be detected through the image defect detection and classification model to obtain the defect detection and classification result of the image to be detected.
2. The method for detecting and classifying image defects based on deep learning algorithm according to claim 1, wherein the classifying the training images to obtain classification types comprises:
classifying the training images according to a qualified standard to obtain a qualified sample type or an unqualified sample type;
obtaining an original image corresponding to the unqualified sample type in the training image, and processing the defects in the original image to obtain a result image;
classifying the result graph according to preset defect types, and determining the defect types corresponding to the unqualified sample types, wherein the defect types comprise white foreign matters, black foreign matters, peeling, scratches, bridges, plating leakage and heterochrosis;
and synthesizing the qualified sample types or the defect types corresponding to the unqualified sample types of all the training images to obtain the classification types.
3. The method for detecting and classifying image defects based on the deep learning algorithm according to claim 2, wherein the preprocessing the training image to obtain an image data set according to the preprocessed image data of the training image and the classification type comprises:
preprocessing the training image by an image preprocessing technology to obtain image data of the preprocessed training image, wherein the image preprocessing technology comprises turnover transformation, translation transformation, scale transformation, contrast transformation, noise disturbance, rotation transformation and segmentation;
and obtaining an image data set according to the preprocessed image data of the training image and the classification type.
4. The method according to claim 1, wherein training through the image data set and the deep learning algorithm convolutional neural network to obtain an image defect detection and classification model comprises:
dividing the image data set into a training set and a verification set;
constructing and obtaining an initial convolutional neural network model by adopting a convolutional neural network based on a deep learning algorithm;
and training the initial convolutional neural network model by setting hyper-parameters and utilizing the training set to obtain an image defect detection and classification model.
5. The method of claim 4, wherein the hyper-parameters include learning rate, iteration number and training batch size,
by setting the hyper-parameters, training the initial convolutional neural network model by using the training set to obtain an image defect detection and classification model, comprising:
setting a learning rate according to a gradient descent method;
setting the training times of the training set according to a preset training standard to obtain iteration times;
setting the size of a training batch according to the memory requirement, wherein the size of the training batch is the number of images for each training;
and inputting the training set into the initial convolutional neural network model for training according to the learning rate, the iteration times and the training batch size to obtain an image defect detection and classification model.
6. The method according to claim 5, wherein the inputting the training set into the initial convolutional neural network model for training according to the learning rate, the iteration number and the training batch size to obtain an image defect detection and classification model comprises:
selecting a single training set with a corresponding image quantity from the training set according to the size of the training batch;
according to the image data and classification category of the training image corresponding to the single training set, presetting N anchor frames with different sizes, and setting N corresponding labels, wherein N is a positive integer greater than or equal to 1;
inputting the single training set into the initial convolutional neural network model for training to obtain N prediction frames with different scales, wherein the scale of each prediction frame is the same as that of a corresponding anchor frame;
calculating to obtain total loss values of all anchor frames and corresponding prediction frames;
and performing iterative training on the initial convolutional neural network model according to the total loss value, the learning rate and the iteration times to obtain an image defect detection and classification model.
7. The image defect detecting and classifying method based on the deep learning algorithm of claim 6, wherein the calculating the total loss value of all anchor frames and corresponding prediction frames includes:
obtaining predicted coordinate data of the target prediction frame and predicted defect coordinate data, wherein the predicted coordinate data comprises a predicted central coordinate value (a: (a))
Figure DEST_PATH_IMAGE001
) Predicting width coordinate value
Figure DEST_PATH_IMAGE002
And predicting the height coordinate value
Figure DEST_PATH_IMAGE003
；
Acquiring actual coordinate data and actual defect coordinate data of an anchor frame corresponding to the target prediction frame, wherein the actual coordinate data comprises an actual central coordinate value (c:)
Figure DEST_PATH_IMAGE004
) Actual width coordinate value
Figure DEST_PATH_IMAGE005
And actual height coordinate value
Figure DEST_PATH_IMAGE006
；
Obtaining the width and height coordinate error of the defect according to the predicted defect coordinate data and the actual defect coordinate data
Figure DEST_PATH_IMAGE007
；
Obtaining a prediction confidence of the target prediction box
Figure DEST_PATH_IMAGE008
And the actual confidence of the corresponding anchor frame
Figure DEST_PATH_IMAGE009
；
Substituting the predicted coordinate data and the actual coordinate data into a coordinate error loss function
Figure DEST_PATH_IMAGE010
Calculating to obtain a coordinate loss value, said
Figure DEST_PATH_IMAGE011
Representing division of a training image into
Figure DEST_PATH_IMAGE012
A grid of
Figure DEST_PATH_IMAGE013
Representing each mesh generation
Figure 979244DEST_PATH_IMAGE013
Training candidate frames of each anchor frame to obtain an outsourcing rectangular frame;
introducing the defect width and height coordinate error, the prediction confidence coefficient and the actual confidence coefficient into a confidence coefficient error loss function
Figure DEST_PATH_IMAGE014
Calculating to obtain a confidence coefficient loss value;
introducing the defect width and height coordinate error into a classification error loss function
Figure DEST_PATH_IMAGE015
Calculating to obtain a classification error loss value;
and calculating the sum of the coordinate loss value, the confidence coefficient loss value and the classification error loss value to obtain a total loss value.
8. The method for detecting and classifying image defects based on deep learning algorithm according to claim 4, further comprising:
verifying the accuracy of the defect detection and classification result of the image defect detection and classification model through the verification set;
when the accuracy is not lower than a preset threshold value, determining that the verification is passed;
and when the accuracy is lower than a preset threshold value, determining that the verification is not passed.
9. An image defect detection and classification system based on a deep learning algorithm, comprising:
the training image acquisition module is used for acquiring images to obtain training images, and the training images comprise qualified samples and unqualified samples;
the training image classification module is used for classifying the training images to obtain classification types, wherein the classification types comprise qualified sample types and unqualified sample types, and the unqualified sample types comprise defect types;
the training image processing module is used for preprocessing the training image and obtaining an image data set according to the preprocessed image data of the training image and the classification type;
the model training module is used for training through the image data set and a deep learning algorithm convolutional neural network to obtain an image defect detection and classification model;
and the defect detection and classification module is used for processing the image to be detected through the image defect detection and classification model when the image to be detected is obtained, so as to obtain the defect detection and classification result of the image to be detected.
CN202210726506.5A 2022-06-24 2022-06-24 Image defect detection and classification method and system based on deep learning algorithm Pending CN115147363A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210726506.5A CN115147363A (en) 2022-06-24 2022-06-24 Image defect detection and classification method and system based on deep learning algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210726506.5A CN115147363A (en) 2022-06-24 2022-06-24 Image defect detection and classification method and system based on deep learning algorithm

Publications (1)

Publication Number Publication Date
CN115147363A true CN115147363A (en) 2022-10-04

Family

ID=83408441

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210726506.5A Pending CN115147363A (en) 2022-06-24 2022-06-24 Image defect detection and classification method and system based on deep learning algorithm

Country Status (1)

Country Link
CN (1) CN115147363A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115656189A (en) * 2022-12-12 2023-01-31 易斯德(福建)智能科技有限公司 Defect detection method and device based on luminosity stereo and deep learning algorithm
CN115731228A (en) * 2022-11-30 2023-03-03 杭州数途信息科技有限公司 Gold-plated chip defect detection system and method
WO2024044942A1 (en) * 2022-08-30 2024-03-07 宁德时代新能源科技股份有限公司 Point inspection method and device for visual inspection system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
CN111612784A (en) * 2020-06-01 2020-09-01 南通大学 Steel plate surface defect detection method based on classification-first YOLO network
CN111612758A (en) * 2020-05-19 2020-09-01 南京航空航天大学 Tunnel surface defect detection method based on convolutional neural network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
CN111612758A (en) * 2020-05-19 2020-09-01 南京航空航天大学 Tunnel surface defect detection method based on convolutional neural network
CN111612784A (en) * 2020-06-01 2020-09-01 南通大学 Steel plate surface defect detection method based on classification-first YOLO network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZYW2002: ""目标检测——yolov3论文精读"", 《HTTPS://BLOG.CSDN.NET/ZYW2002/ARTICLE/DETAILS/122917498》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024044942A1 (en) * 2022-08-30 2024-03-07 宁德时代新能源科技股份有限公司 Point inspection method and device for visual inspection system
CN115731228A (en) * 2022-11-30 2023-03-03 杭州数途信息科技有限公司 Gold-plated chip defect detection system and method
CN115731228B (en) * 2022-11-30 2023-08-18 杭州数途信息科技有限公司 Gold-plated chip defect detection system and method
CN115656189A (en) * 2022-12-12 2023-01-31 易斯德(福建)智能科技有限公司 Defect detection method and device based on luminosity stereo and deep learning algorithm
CN115656189B (en) * 2022-12-12 2023-03-17 易斯德(福建)智能科技有限公司 Defect detection method and device based on luminosity stereo and deep learning algorithm

Similar Documents

Publication Publication Date Title
Zhou et al. Automated visual inspection of glass bottle bottom with saliency detection and template matching
CN108918536B (en) Tire mold surface character defect detection method, device, equipment and storage medium
CN115147363A (en) Image defect detection and classification method and system based on deep learning algorithm
JP5546317B2 (en) Visual inspection device, visual inspection discriminator generation device, visual inspection discriminator generation method, and visual inspection discriminator generation computer program
CN106600600A (en) Wafer defect detection method based on characteristic matching
CN114897816B (en) Mask R-CNN mineral particle identification and particle size detection method based on improved Mask
Wang et al. Surface defects detection using non-convex total variation regularized RPCA with kernelization
CN109284779A (en) Object detecting method based on the full convolutional network of depth
CN116188475B (en) Intelligent control method, system and medium for automatic optical detection of appearance defects
Eshkevari et al. Automatic dimensional defect detection for glass vials based on machine vision: A heuristic segmentation method
CN112365497A (en) High-speed target detection method and system based on Trident Net and Cascade-RCNN structures
CN111914902B (en) Traditional Chinese medicine identification and surface defect detection method based on deep neural network
Li et al. SPM-IS: An auto-algorithm to acquire a mature soybean phenotype based on instance segmentation
CN114694178A (en) Method and system for monitoring safety helmet in power operation based on fast-RCNN algorithm
CN111815573B (en) Coupling outer wall detection method and system based on deep learning
CN113420619A (en) Remote sensing image building extraction method
Chao-Ching et al. Machine vision and deep learning based rubber gasket defect detection
CN115599844A (en) Visual detection method for misloading and neglected loading of airplane airfoil connecting piece
CN112037221A (en) Multi-domain co-adaptation training method for cervical cancer TCT slice positive cell detection model
CN111178405A (en) Similar object identification method fusing multiple neural networks
CN105654042B (en) The proving temperature character identifying method of glass-stem thermometer
CN113962980A (en) Glass container flaw detection method and system based on improved YOLOV5X
CN117910073A (en) Artwork package design optimization system and method based on 3D printing technology
CN116843615B (en) Lead frame intelligent total inspection method based on flexible light path
CN117011274A (en) Automatic glass bottle detection system and method thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination