CN114549493A - Magnetic core defect detection system and method based on deep learning - Google Patents

Magnetic core defect detection system and method based on deep learning Download PDF

Info

Publication number
CN114549493A
CN114549493A CN202210186081.3A CN202210186081A CN114549493A CN 114549493 A CN114549493 A CN 114549493A CN 202210186081 A CN202210186081 A CN 202210186081A CN 114549493 A CN114549493 A CN 114549493A
Authority
CN
China
Prior art keywords
magnetic core
image
roi
detection
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210186081.3A
Other languages
Chinese (zh)
Inventor
陈科宇
雷雅彧
刘鹏飞
翁扬凯
吴帅杰
王宪保
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN202210186081.3A priority Critical patent/CN114549493A/en
Publication of CN114549493A publication Critical patent/CN114549493A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36—Sorting apparatus characterised by the means used for distribution
    • B07C5/361—Processing or control devices therefor, e.g. escort memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/155—Segmentation; Edge detection involving morphological operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/168—Segmentation; Edge detection involving transform domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20061—Hough transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The invention discloses a magnetic core defect detection system and method based on deep learning. The trained model is led into the real-time detection module, real-time defect detection can be carried out on the magnetic core on the production line, and the peripheral sorting mechanism sorts the magnetic core according to the defect detection result. After the detection is finished, the detection result including the relevant defect indexes is subjected to statistical analysis, and a corresponding detection report is generated. The algorithm based on deep learning and traditional image processing has higher detection precision; and the model training is easier by combining a full-supervision training mode and a semi-supervision training mode. And a series of functions such as sampling, marking, training, real-time detection, intelligent sorting and the like are integrated, the system integration level is higher, and the usability is stronger.

Description

Magnetic core defect detection system and method based on deep learning
Technical Field
The invention relates to the field of image detection and identification, in particular to a magnetic core defect detection method based on deep learning target detection and identification.
Background
The surface quality of industrial products is an important part of the product quality. However, in the actual production process, various defects are easily generated on the surface of the product. Aiming at the problem of the surface defects, at present, the defect detection of the product surface in industrial production generally adopts a manual detection method. Visual fatigue can occur when people work for a long time, and the classification of defects can be influenced by subjective judgment of people, so that the problems of low efficiency, high cost, low detection precision and the like exist, and the method is not suitable for the requirement of large-scale industrial production.
Magnetic cores are common components in coil transformers. In the actual production or transportation process, due to collision friction and other reasons, the surface of the magnetic core is easily damaged, and defects such as spots, scratches and the like are generated, so that the integrity of the magnetic core is influenced, and the performance of the magnetic core is also greatly influenced.
The conventional magnetic core defect detection method is difficult to classify manually, is high in efficiency, and is a technical problem in how to develop a defect detection system which is safe, reliable, high in detection precision and wide in application range.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a magnetic core defect detection system and method based on deep learning, and the specific technical scheme is as follows:
a deep learning based core defect detection system, the system comprising:
the sampling module is used for receiving and displaying magnetic core images acquired by a plurality of industrial cameras arranged on a magnetic core production line in real time, sequentially carrying out gray processing, denoising and binaryzation on the magnetic core images, then extracting a magnetic core contour, extracting an ROI (region of interest) and storing the magnetic core contour and the ROI into a sample library;
the marking module is used for marking the defects of the ROI of each magnetic core to obtain marking information corresponding to each magnetic core;
the training module is internally provided with a deep neural network model, receives the ROI area of each magnetic core and corresponding marking information, and guides the information into the deep neural network model for training to obtain an optimized deep neural network model;
and the real-time detection module is used for receiving the ROI (region of interest) of the image of the magnetic core to be detected, acquired by the sampling module in real time, inputting the ROI into the optimized deep neural network model to obtain a defect detection result of the magnetic core, and sending the defect detection result of the magnetic core to a corresponding peripheral sorting device to sort the magnetic core.
Furthermore, an image processing algorithm based on self-adaptive threshold segmentation is further built in the real-time detection module, the ROI of the image of the magnetic core to be detected, which is acquired in real time, is detected through the image processing algorithm based on the threshold and the optimized deep neural network model, and the magnetic core is determined to be a defective magnetic core as long as one result shows that the magnetic core is defective.
Furthermore, the real-time detection module can also generate a magnetic core detection report comprising the number of defective products, the defect rate, the number of various defects and the detection speed, and store the detection results of all the magnetic cores into a database; the confidence coefficient threshold value set by the real-time detection module before real-time detection is adjustable;
the detection system also comprises a setting module which is used for setting parameters of the industrial camera and other modules.
Further, the marking module stores the data set into three different formats, namely COCO, VOC or YOLO;
the training module is internally provided with a full-supervision training algorithm and a semi-supervision training algorithm.
A magnetic core defect detection method based on deep learning is realized based on the detection system, and comprises the following steps:
the method comprises the following steps: a sensor arranged on a magnetic core production line sends a collection signal to a plurality of industrial cameras, and the magnetic core images are collected by the plurality of industrial cameras;
step two: the sampling module sequentially performs gray processing, denoising and binarization on magnetic core images acquired by the multi-path industrial camera, extracts a magnetic core contour, extracts an ROI (region of interest) and stores the ROI into a sample library;
step three: the marking module marks the defects of the ROI of each magnetic core to obtain marking information corresponding to each magnetic core;
step four: the training module receives the ROI area of each magnetic core and corresponding marking information, and guides the information into the deep neural network model for training to obtain an optimized deep neural network model;
step five: and the real-time detection module receives the ROI of the image of the magnetic core to be detected, which is acquired by the sampling module in real time, inputs the ROI into the optimized deep neural network model to obtain a defect detection result of the magnetic core, and sends the defect detection result of the magnetic core to a corresponding peripheral sorting device to sort the magnetic core.
Further, the steps are realized by the following sub-steps:
step 2.1: carrying out gray level processing on the magnetic core image by using a weighted average method, wherein the calculation formula is as follows:
gray(i，j)＝(R(i，j)+G(i，j)+B(i，j))/3 (1)
wherein, (i, j) is the coordinate of each pixel point, gray (i, j) is the gray value of each pixel after graying, and R (i, j), G (i, j) and B (i, j) are three channel values of the color image;
step 2.2: denoising the gray scale image obtained in the step 2.1 by using median filtering of a 3 x 3 template, wherein the formula is as follows:
g(x，y)＝med{f(x-k，y-l)，(k，l∈W)} (2)
wherein f (x, y) and g (x, y) are respectively an original image and a processed image; w is a 3 x 3 two-dimensional template;
step 2.3: binarizing the image filtered in the step 2.2 by using an Otsu method, wherein the calculation formula is as follows:
u＝w0*u0+w1*u1 (3)
h＝w0*(u0-u)2+w1*(u1-u)2 (4)
wherein u is the average gray of the image, w0 is the proportion of the foreground pixel points to the whole image, u0 is the average value of the foreground pixel points, w1 is the proportion of the background pixel points to the whole image, u1 is the average value of the background pixel points, and h is the inter-class variance, so that t when h is the maximum is the optimal threshold for segmentation;
step 2.4: and (3) performing closed operation on the binary image obtained in the step (2.3) to reduce the internal contour and avoid repeated detection, wherein the formula is as follows:
Figure BDA0003523468880000031
wherein A is a target image, and B is a structural element;
step 2.5: extracting the magnetic core contour of the binary image processed in the step 2.4 by adopting a Hough transform circle detection method;
step 2.6: and traversing each contour to obtain a minimum external rectangle and vertex coordinates of the contour, and then segmenting the rectangle by utilizing the vertex coordinates to extract an ROI (region of interest) and storing the ROI into a sample library.
Further, in the fourth step, the step when the training module selects the semi-supervised training mode is as follows:
(1) inputting a marked data set and an unmarked data set, and inputting the marked data set into a deep neural network model to train to obtain a classifier;
(2) predicting class labels of all unlabeled data instances by using the trained classifier, and screening according to a screening standard that the confidence coefficient is greater than 0.9 and only one prediction box is used;
(3) marking pseudo class labels on the screened prediction samples, adding the pseudo class labels into a label training set, and then retraining;
(4) and (4) repeating the steps (1) to (3) until the unmarked data set is an empty set or the model loss is not reduced any more, and obtaining the defect detection classifier.
Further, when the data increment of the database reaches a set threshold, automatically triggering a deep neural network model of a training module to perform semi-supervised training; and if the accuracy of the newly obtained model is higher, switching the deep neural network model of the training module into the latest model.
Further, the deep neural network model is a YOLO V3 network, firstly, feature extraction is carried out on a magnetic core image through a backbone network, then, each grid of the obtained feature maps with three different scales is detected, and finally, feature maps with 2 dimensions are output; and selecting a confidence threshold, filtering out a low threshold box, inhibiting by a non-maximum value, and outputting a prediction result of the whole network.
One dimension of the output feature map is the feature map size, and the other dimension is the depth, i.e., B (5+ C), where B is the number of frames detected by each grid, and C is the number of defect classes of the magnetic core, including 4 positions (Ix, Iy, Iw, Ih) and 1 confidence of the magnetic core defects.
The invention has the following beneficial effects:
(1) the invention applies the deep learning target detection algorithm to the problem of automatic defect detection of the magnetic core, and effectively improves the detection speed and the detection rate of the magnetic core.
(2) The invention integrates a series of functions such as sampling, labeling, training, real-time detection and the like into the same system. The defect detection system has the advantages of high integration degree, simplicity in use, easiness in use, no need of related professional knowledge and capability of being applied to the field of defect detection on a large scale.
Drawings
FIG. 1 is a flowchart illustrating the operation of a magnetic core defect detection system of the present invention;
FIG. 2 is a flow chart of the real-time detection module.
FIG. 3 is a schematic diagram of a deep neural network model.
FIG. 4 is a defect map of a magnetic core.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the objects and effects of the present invention will be more apparent, it being understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
As shown in FIG. 1, the magnetic core defect detection system based on deep learning of the present invention includes a sampling module, a labeling module, a training module and a real-time detection module, wherein data links are communicated with each other between the modules to form a closed loop. After the system is arranged, information transmission is realized with the sensor and the multiple industrial cameras arranged on the magnetic core production line.
The sampling module is used for receiving and displaying magnetic core images which are parallelly acquired by a plurality of industrial cameras arranged on a magnetic core production line, sequentially carrying out gray processing, denoising and binaryzation on the magnetic core images, then extracting a magnetic core contour, extracting an ROI (region of interest) and storing the magnetic core contour and the ROI into a sample library. The sampling module can sample multiple industrial cameras concurrently, is high in sampling speed and high in system throughput, can count the number of samples in real time, and stores the sampled magnetic core images in a sample database after preprocessing.
And the marking module visually marks the defects of the ROI of each magnetic core to obtain marking information corresponding to each magnetic core and generate a data set with a specified format, such as three different formats of COCO, VOC or YOLO.
The training module is internally provided with a deep neural network model, a full-supervision training algorithm and a semi-supervision training algorithm, acquires the ROI area of each magnetic core and corresponding marking information from a sample library, introduces the information into the deep neural network model, and selects a proper training mode (such as full supervision or semi supervision) for training to obtain the optimized deep neural network model and the corresponding weight thereof. The training module can also display the training process in real time, generate a training curve, and can automatically interrupt and recover training for secondary training.
As shown in fig. 3, the basic idea of the deep neural network is:
firstly, the magnetic core image is subjected to feature extraction through a backbone network and a residual error network, and each grid of three feature maps with different scales is obtained for detection. The final output signature has 2 dimensions: one is the feature size and the other is the depth, i.e., B (5+ C), where B is the number of frames detected per grid and C is the number of defect classes for the core, including 4 locations (Ix, Iy, Iw, Ih) and 1 confidence of the core defects. And selecting a confidence threshold, filtering out a low threshold box, and outputting the prediction result of the whole network through nms (non-maximum suppression). When the semi-supervised training mode is selected, the specific steps are as follows:
(1) inputting a marked data set and an unmarked data set, and inputting the marked data set into a deep neural network model to train to obtain a classifier;
(2) predicting class labels of all unlabeled data instances by using the trained classifier, and screening according to a screening standard that the confidence coefficient is greater than 0.9 and only one prediction box is used;
(3) marking pseudo class labels on the screened prediction samples, adding the pseudo class labels into a label training set, and then retraining;
(4) and (4) repeating the steps (1) to (3) until the unmarked data set is an empty set or the model loss is not reduced any more, and obtaining the defect detection classifier.
And the real-time detection module is used for loading the model weight of the appointed deep neural network model, acquiring the ROI (region of interest) of the image of the magnetic core to be detected, acquired by the sampling module in real time, from the sample library, inputting the ROI into the optimized deep neural network model to obtain the defect real-time detection result of the magnetic core, and sending the defect detection result of the magnetic core to a corresponding peripheral sorting device to sort the magnetic core. Meanwhile, the real-time monitoring module also adds the sample image obtained in the detection into the database and displays the defect detection result in real time on a system interface. And when the detection is finished, the detection result including the relevant defect indexes is subjected to statistical analysis, and a detection report including parameters such as the number of defective products, the defect rate, the number of various defects, the detection speed and the like is generated. In addition, in order to ensure that all the defective magnetic cores are detected, an image processing algorithm based on a threshold value is also built in the real-time detection module and is used for detecting the defects of the magnetic cores together with the optimized deep neural network model, so that double verification is realized. If there is a result indicating that the core is defective, the core is determined to be defective. Because the system is allowed to have a certain false detection in production, but the condition of missing detection is absolutely not allowed. The flow chart of the real-time detection module is shown in fig. 2.
The confidence threshold set by the real-time detection module before real-time detection is carried out can be adjusted, a user can set the confidence threshold, when the detection result is good, the model can be identified as good and output the detection result only if the confidence score is larger than the set threshold, and if the confidence score is lower than the confidence score threshold, the model outputs the result of the second detection confidence, so that the reliability of the system is ensured.
And the setting module is used for setting parameters of all modules, such as the type of the selected deep neural network model, the training mode and the like, the setting of the starting number of the industrial cameras in butt joint with the detection system, the setting of the number of frames per second and the sampling size of the cameras, the setting of the detection model weight and the model related configuration file, the interface skin setting and the like.
In addition, the magnetic core defect detection system can also realize a model self-updating function, namely, when the data increment of the database reaches a set threshold value, a deep neural network model of a training module is automatically triggered to carry out semi-supervised training; and if the accuracy of the newly obtained model is higher, switching the deep neural network model of the training module into the latest model.
The invention discloses a magnetic core defect detection method based on deep learning, which is realized based on the magnetic core defect detection system, and comprises the following steps:
the method comprises the following steps: when a magnetic core on the conveyor belt passes through a sensor arranged on a magnetic core production line, the sensor triggers a plurality of industrial cameras to acquire magnetic core images;
step two: carrying out preprocessing such as gray processing, denoising and binaryzation on magnetic core images acquired by a plurality of paths of industrial cameras by a sampling module in sequence, then extracting a magnetic core outline, extracting an ROI (region of interest) area and storing the ROI area in a sample library; this step is realized by the following substeps:
step 2.1: carrying out gray processing on the magnetic core image by using a weighted average method, wherein the calculation formula is as follows:
gray(i，j)＝(R(i，j)+G(i，j)+B(i，j))/3 (1)
wherein, (i, j) is the coordinate of each pixel point, gray (i, j) is the gray value of each pixel after graying, and R (i, j), G (i, j) and B (i, j) are three channel values of the color image;
step 2.2: denoising the gray level image obtained in the step 2.1 by using median filtering of a 3 x 3 template, wherein the formula is
g(x，y)＝med{f(x-k，y-l)，(k，l∈W)} (2)
Wherein f (x, y) and g (x, y) are respectively an original image and a processed image; w is a 3 x 3 two-dimensional template;
step 2.3: binarizing the image filtered in the step 2.2 by using an Otsu method, wherein the calculation formula is as follows:
u＝w0*u0+w1*u1 (3)
h＝w0*(u0-u)2+w1*(u1-u)2 (4)
wherein u is the average gray of the image, w0 is the proportion of the foreground pixel points to the whole image, u0 is the average of the foreground pixel points, w1 is the proportion of the background pixel points to the whole image, ul is the average of the background pixel points, and h is the inter-class variance, so that t when h is the maximum is the optimal threshold for segmentation;
step 2.4: and (3) performing closed operation on the binary image obtained in the step (2.3) to reduce the internal contour and avoid repeated detection, wherein the formula is as follows:
Figure BDA0003523468880000061
wherein A is a target image, and B is a structural element;
step 2.5: extracting the magnetic core contour of the binary image processed in the step 2.4 by adopting a Hough transform circle detection method, and specifically comprising the following steps of:
a＝x-rcosθ (6)
b＝y-rsinθ (7)
in a three-dimensional coordinate system formed by abr, a point can uniquely determine a circle, (a, b) represents the center of the circle, r represents the radius, and in a Cartesian xy coordinate system, all circles passing through a certain point are mapped to the abr coordinate system to form a three-dimensional curve. All circles passing through all non-zero pixel points in the xv coordinate system form a plurality of three-dimensional curves in the abr coordinate system.
The equation of a circle for all points on the same circle in the xy coordinate system is the same, and they map to the same point in the abr coordinate system, so that the point in the abr coordinate system should have the total pixel curves of the circle intersect. By determining the number of intersections (accumulations) of each point in abr, points greater than a certain threshold are considered circles.
Step 2.6: and traversing each contour to obtain a minimum external rectangle and vertex coordinates of the contour, and then segmenting the rectangle by utilizing the vertex coordinates to extract an ROI (region of interest) and storing the ROI into a sample library.
Step three: the marking module marks the defects of the ROI of each magnetic core to obtain marking information corresponding to each magnetic core;
step four: receiving the ROI area of each magnetic core and corresponding marking information by a training module, importing the information into a deep neural network model, and training to obtain an optimized deep neural network model; the training mode comprises semi-supervised training and full-supervised training, and the training module adopts the semi-supervised training mode and comprises the following steps:
(1) inputting a marked data set and an unmarked data set, and inputting the marked data set into a deep neural network model to train to obtain a classifier;
(2) predicting class labels of all unlabeled data instances by using the trained classifier, and screening according to a screening standard that the confidence coefficient is greater than 0.9 and only one prediction box is used;
(3) marking pseudo class labels on the screened prediction samples, adding the pseudo class labels into a label training set, and then retraining;
(4) and (4) repeating the steps (1) to (3) until the unmarked data set is an empty set or the model loss is not reduced any more, and obtaining the defect detection classifier.
Step five: the real-time detection module receives the ROI of the image of the magnetic core to be detected, which is acquired by the sampling module in real time, inputs the ROI into the optimized deep neural network model to obtain a defect detection result of the magnetic core, and sends the defect detection result of the magnetic core to a corresponding peripheral sorting device to sort the magnetic core.
When the data increment of the database reaches a set threshold value, automatically triggering a deep neural network model of a training module to perform semi-supervised training; and if the accuracy of the newly obtained model is higher, switching the deep neural network model of the training module into the latest model.
The industrial camera in this embodiment is an area-array industrial camera having 500 ten thousand pixels, which is Haekwondo, and the working distance is set to 13.1cm and the detection field is set to a rectangular area of 36mm 30 mm. The magnetic core is from a manufacturer of a magnetic core in Hangzhou, and the surface defects are mainly spots and scratches, as shown in FIG. 4. The sampling module samples 2000 magnetic core images on a production line, wherein the defect image proportion is twenty-five percent. And then, performing data enhancement on the defect image, and expanding the defect image to 3000 sheets. And labeling the magnetic core data set through a labeling module, importing the labeled information and the magnetic core data set obtained through labeling into a training module for training, performing training iteration for 300 generations, and obtaining an optimized deep neural network model after single-time picture input (batch-size) is 16. The real-time detection module can accurately detect the magnetic core on the production line in real time based on the obtained optimized neural network model, and the actual production requirements of enterprises are met. The detection precision corresponding to each defect type detected by the system and the method of the invention in the embodiment is shown in table 1.
TABLE 1 inspection accuracy table corresponding to each defect type
Identifying classifications Detection accuracy
Front spot 0.921
Spots on the reverse side 0.933
Front scratch 0.968
Back scratch mark 0.93
Good front surface 0.918
Good reverse side 0.939
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and although the invention has been described in detail with reference to the foregoing examples, it will be apparent to those skilled in the art that various changes in the form and details of the embodiments may be made and equivalents may be substituted for elements thereof. All modifications, equivalents and the like which come within the spirit and principle of the invention are intended to be included within the scope of the invention.

Claims (9)

1. A deep learning based core defect detection system, the system comprising:
the sampling module is used for receiving and displaying magnetic core images acquired by a plurality of industrial cameras arranged on a magnetic core production line in real time, sequentially carrying out gray processing, denoising and binaryzation on the magnetic core images, then extracting a magnetic core contour, extracting an ROI (region of interest) and storing the magnetic core contour and the ROI into a sample library;
the marking module is used for marking the defects of the ROI of each magnetic core to obtain marking information corresponding to each magnetic core;
the training module is internally provided with a deep neural network model, receives the ROI area of each magnetic core and corresponding marking information, and guides the information into the deep neural network model for training to obtain an optimized deep neural network model;
and the real-time detection module is used for receiving the ROI (region of interest) of the image of the magnetic core to be detected, acquired by the sampling module in real time, inputting the ROI into the optimized deep neural network model to obtain a defect detection result of the magnetic core, and sending the defect detection result of the magnetic core to a corresponding peripheral sorting device to sort the magnetic core.
2. The magnetic core defect detection system based on deep learning of claim 1, wherein the real-time detection module further embeds an image processing algorithm based on adaptive threshold segmentation, and the threshold-based image processing algorithm and the optimized deep neural network model simultaneously detect the ROI region of the image of the magnetic core to be detected, which is acquired in real time, and the magnetic core is determined to be defective as long as one result indicates that the magnetic core is defective.
3. The deep learning-based magnetic core defect detection system according to claim 1, wherein the real-time detection module is further capable of generating a magnetic core detection report including the number of defective products, the defect rate, the number of various types of defects and the detection speed, and storing the detection results of all the magnetic cores in a database; the confidence coefficient threshold value set by the real-time detection module before real-time detection is adjustable;
the detection system also comprises a setting module which is used for setting parameters of the industrial camera and other modules.
4. The deep learning-based magnetic core flaw detection system of claim 1, wherein the labeling module saves the data set into three different formats, COCO, VOC or YOLO;
the training module is internally provided with a full-supervision training algorithm and a semi-supervision training algorithm.
5. A method for detecting defects of a magnetic core based on deep learning, which is implemented based on the detection system of any one of the preceding claims, and comprises the following steps:
the method comprises the following steps: a sensor arranged on a magnetic core production line sends a collection signal to a plurality of industrial cameras, and the magnetic core images are collected by the plurality of industrial cameras;
step two: the sampling module sequentially performs gray processing, denoising and binarization on magnetic core images acquired by the multi-path industrial camera, extracts a magnetic core contour, extracts an ROI (region of interest) and stores the ROI into a sample library;
step three: the marking module marks the defects of the ROI of each magnetic core to obtain marking information corresponding to each magnetic core;
step four: the training module receives the ROI area of each magnetic core and corresponding marking information, and guides the information into the deep neural network model for training to obtain an optimized deep neural network model;
step five: and the real-time detection module receives the ROI of the image of the magnetic core to be detected, which is acquired by the sampling module in real time, inputs the ROI into the optimized deep neural network model to obtain a defect detection result of the magnetic core, and sends the defect detection result of the magnetic core to a corresponding peripheral sorting device to sort the magnetic core.
6. The deep learning-based core defect inspection method of claim 5, wherein the step is implemented by the following sub-steps:
step 2.1: carrying out gray level processing on the magnetic core image by using a weighted average method, wherein the calculation formula is as follows:
gray(i，j)＝(R(i，j)+G(i，j)+B(i，j))/3 (1)
wherein, (i, j) is the coordinate of each pixel point, gray (i, j) is the gray value of each pixel after graying, and R (i, j), G (i, j) and B (i, j) are three channel values of the color image;
step 2.2: denoising the gray scale image obtained in the step 2.1 by using median filtering of a 3 x 3 template, wherein the formula is as follows:
g(x，y)＝med{f(x-k，y-l)，(k，l∈W)} (2)
wherein f (x, y) and g (x, y) are respectively an original image and a processed image; w is a 3 x 3 two-dimensional template;
step 2.3: binarizing the image filtered in the step 2.2 by using an Otsu method, wherein the calculation formula is as follows:
u＝w0*u0+w1*u1 (3)
h＝w0*(u0-u)2+w1*(u1-u)2 (4)
wherein u is the average gray of the image, w0 is the proportion of the foreground pixel points to the whole image, u0 is the average value of the foreground pixel points, w1 is the proportion of the background pixel points to the whole image, u1 is the average value of the background pixel points, and h is the inter-class variance, so that t when h is the maximum is the optimal threshold for segmentation;
step 2.4: and (3) performing closed operation on the binary image obtained in the step (2.3) to reduce the internal contour and avoid repeated detection, wherein the formula is as follows:
Figure FDA0003523468870000021
wherein A is a target image, and B is a structural element;
step 2.5: extracting the magnetic core contour of the binary image processed in the step 2.4 by adopting a Hough transform circle detection method;
step 2.6: and traversing each contour to obtain a minimum external rectangle and vertex coordinates of the contour, and then segmenting the rectangle by utilizing the vertex coordinates to extract an ROI (region of interest) and storing the ROI into a sample library.
7. The method for detecting defects of magnetic cores based on deep learning of claim 5, wherein in the fourth step, the step when the training module selects the semi-supervised training mode is as follows:
(1) inputting a marked data set and an unmarked data set, and inputting the marked data set into a deep neural network model to train to obtain a classifier;
(2) predicting class labels of all unlabeled data instances by using the trained classifier, and screening according to a screening standard that the confidence coefficient is greater than 0.9 and only one prediction box is used;
(3) marking pseudo class labels on the screened prediction samples, adding the pseudo class labels into a label training set, and then retraining;
(4) and (4) repeating the steps (1) to (3) until the unmarked data set is an empty set or the model loss is not reduced any more, and obtaining the defect detection classifier.
8. The magnetic core defect detection method based on deep learning of claim 5, characterized in that when the data increment of the database reaches a set threshold, the deep neural network model of the training module is automatically triggered to perform semi-supervised training; and if the accuracy of the newly obtained model is higher, switching the deep neural network model of the training module into the latest model.
9. The magnetic core defect detection method based on deep learning of claim 5, wherein the deep neural network model is a YOLO V3 network, the magnetic core image is firstly subjected to feature extraction through a backbone network, then each grid of the obtained three feature maps with different scales is detected, and finally a feature map with 2 dimensions is output; and selecting a confidence threshold, filtering out a low threshold box, inhibiting by a non-maximum value, and outputting a prediction result of the whole network.
One dimension of the output feature map is the feature map size, and the other dimension is the depth, i.e., B (5+ C), where B is the number of frames detected by each grid, and C is the number of defect classes of the magnetic core, including 4 positions (Ix, Iy, Iw, Ih) and 1 confidence of the magnetic core defects.
CN202210186081.3A 2022-02-28 2022-02-28 Magnetic core defect detection system and method based on deep learning Pending CN114549493A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210186081.3A CN114549493A (en) 2022-02-28 2022-02-28 Magnetic core defect detection system and method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210186081.3A CN114549493A (en) 2022-02-28 2022-02-28 Magnetic core defect detection system and method based on deep learning

Publications (1)

Publication Number Publication Date
CN114549493A true CN114549493A (en) 2022-05-27

Family

ID=81678788

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210186081.3A Pending CN114549493A (en) 2022-02-28 2022-02-28 Magnetic core defect detection system and method based on deep learning

Country Status (1)

Country Link
CN (1) CN114549493A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115601618A (en) * 2022-11-29 2023-01-13 浙江华是科技股份有限公司（Cn） Magnetic core defect detection method and system and computer storage medium
CN115615998A (en) * 2022-12-13 2023-01-17 浙江工业大学 Circular magnetic core side defect detection device and method thereof

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115601618A (en) * 2022-11-29 2023-01-13 浙江华是科技股份有限公司（Cn） Magnetic core defect detection method and system and computer storage medium
CN115601618B (en) * 2022-11-29 2023-03-10 浙江华是科技股份有限公司 Magnetic core defect detection method and system and computer storage medium
CN115615998A (en) * 2022-12-13 2023-01-17 浙江工业大学 Circular magnetic core side defect detection device and method thereof

Similar Documents

Publication Publication Date Title
CN109829914B (en) Method and device for detecting product defects
CN111080622B (en) Neural network training method, workpiece surface defect classification and detection method and device
CN108765412B (en) Strip steel surface defect classification method
CN106841209B (en) One kind being based on big data self study chemical fiber wire ingot intelligence appearance detection system and method
CN111517092B (en) Transmission belt tearing detection method
CN109550712A (en) A kind of chemical fiber wire tailfiber open defect detection system and method
CN108090494B (en) Textile defect identification method based on Gabor filter and support vector machine
CN114549493A (en) Magnetic core defect detection system and method based on deep learning
CN104198497A (en) Surface defect detection method based on visual saliency map and support vector machine
CN114972356B (en) Plastic product surface defect detection and identification method and system
CN111242899B (en) Image-based flaw detection method and computer-readable storage medium
CN106355579A (en) Defect detecting method of cigarette carton surface wrinkles
CN111612747B (en) Rapid detection method and detection system for product surface cracks
CN110349125A (en) A kind of LED chip open defect detection method and system based on machine vision
CN111724355A (en) Image measuring method for abalone body type parameters
CN111161237A (en) Fruit and vegetable surface quality detection method, storage medium and sorting device thereof
CN115601355A (en) Method and device for detecting and classifying product surface defects and storage medium
Wah et al. Analysis on feature extraction and classification of rice kernels for Myanmar rice using image processing techniques
CN113850749A (en) Method for training defect detector
CN113870202A (en) Far-end chip defect detection system based on deep learning technology
CN111487192A (en) Machine vision surface defect detection device and method based on artificial intelligence
CN113435460A (en) Method for identifying brilliant particle limestone image
CN111539330B (en) Transformer substation digital display instrument identification method based on double-SVM multi-classifier
CN116245882A (en) Circuit board electronic element detection method and device and computer equipment
CN113822836A (en) Method of marking an image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination