CN111951232A - Metal powder injection molding appearance defect detection method and system - Google Patents
Metal powder injection molding appearance defect detection method and system Download PDFInfo
- Publication number
- CN111951232A CN111951232A CN202010724049.7A CN202010724049A CN111951232A CN 111951232 A CN111951232 A CN 111951232A CN 202010724049 A CN202010724049 A CN 202010724049A CN 111951232 A CN111951232 A CN 111951232A
- Authority
- CN
- China
- Prior art keywords
- layer
- image data
- training
- category
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a metal powder injection molding appearance defect detection method and system, comprising the following steps: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data; preprocessing the image data and expanding the image data to obtain processed image data; labeling the processed image data by using labeling software; converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion; processing image data based on a computer vision deep neural network, and constructing a deep learning model; training a deep learning model by using the divided data sets; carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result; the invention has high stability and high detection rate.
Description
Technical Field
The invention relates to the field of computer vision and deep learning, in particular to a metal powder injection molding appearance defect detection method and system, and more particularly to an MIM metal powder injection molding appearance defect detection method applied to appearance defect detection of metal powder injection molding workpieces.
Background
One of the most important tasks in industrial processes is to ensure the quality of the finished product, in particular the inspection of the surface of the product. In general, the quality inspection of the surface of the product is carried out manually by workshop workers, which is time-consuming, inefficient and severely limits productivity. A few years ago, the traditional machine vision method has been enough to solve these problems, but in the industrial 4.0 environment, the intelligent factory is developing towards the generalization of products, and needs to adapt to the requirements of new products again. The conventional machine vision technology does not have such flexibility because the technical scheme is that the characteristics of manual design can only be applied to a specific field, and once a product line is changed or a production environment is changed, the method needs to be modified again or even overturned for redesign. One solution is to find low points that improve flexibility driven by large amounts of data, and deep learning methods can quickly adapt to new products and detect defects only through a reasonable number of trainings. Compared with the traditional visual scheme, the deep learning method can directly extract data from low-level data and has higher response capability, so that the project of manually learning defect characteristics is completely replaced, the deep learning method is fast suitable for new products, and the method becomes a flexible production line very suitable for production. Nevertheless, the pending problems remain: how to obtain a large amount of annotation data and to be able to be used for practical applications is particularly important for deep learning, since deep models with millions of learnable parameters require thousands of pictures, and in practice image data of this magnitude is very difficult to obtain in an industrial production environment.
The invention researches a deep learning method for surface quality control, in particular to the field of industrial product surface defects. The invention network has good classification performance and meets the marking requirements of the industrial field, the number of training samples and the calculated amount. The data requirements are addressed by using a deep convolutional network of a two-stage architecture. A new network with segmentation and classification effects is designed to meet the requirement of training of small batches of samples with defects and achieve good detection effects.
Deep learning surface foreign object detection is becoming a hot and promising area of research because of their advanced and direct presentation in the field of visual detection. Deep learning techniques have evolved to the techniques best suited to this detection task. They allow inspection systems to learn some examples to achieve inspection goals, and the present invention proposes a segmentation-based deep learning network for inspecting and segmenting surface defects and verified in the specific field of surface scratch inspection. The proposed network structure makes training for small batch samples possible, and is a very important appeal in practical application. A large amount of field data also verifies the network structure labeling progress, the training sample number and the calculation overhead. The training of the model is based on real scene data, the number of samples for each training is only hundreds, wherein the number of defective samples is more than three or forty, so that a model with good effect can be obtained, compared with the requirement of thousands of training samples in a common model frame, the model can better fit the real situation of the output of the intelligent workshop sample training samples, and the deep learning method can get rid of the limitation of insufficient training samples and can be really applied to the ground.
Patent document CN106770307A (application number: 201610853917.5) discloses an electronic product appearance surface defect detection device and a detection method thereof, which includes a detection object shell, a high-precision CCD camera is arranged at the upper end of a double-telephoto lens, a combined illumination system coaxial light source is arranged at the lower end of the double-telephoto lens, a combined illumination system ball integral light source is arranged at the lower end of the combined illumination system coaxial light source, and the detection object shell is placed in the combined illumination system ball integral light source, relating to CCD image acquisition, optical illumination, image processing and surface defect detection technologies.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method and a system for detecting the appearance defects of metal powder injection molding.
The invention provides a metal powder injection molding appearance defect detection method, which comprises the following steps:
step M1: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data;
step M2: preprocessing the image data and expanding the image data to obtain processed image data;
step M3: labeling the processed image data by using labeling software;
step M4: converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion;
step M5: processing image data based on a computer vision deep neural network, and constructing a deep learning model;
step M6: training a deep learning model by using the divided data sets;
step M7: carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
the deep learning model is a mathematical model established between the image and the label, and carries out category judgment and position positioning on a certain object in the input image.
Preferably, the image data expansion in the step M2 includes: image rotation, random cropping, random boosting of gaussian noise, image scaling and/or slight projective transformation.
Preferably, the deep learning model in the step M5 includes a segmentation network and a classification network;
the classification category of each pixel point in the segmentation network learning image comprises a background pixel category and a defect pixel category;
the classification network judges each pixel point in the extracted background area and defect area on the basis of network segmentation to give the possibility that each pixel point belongs to the background pixel category and the defect pixel category, namely confidence.
Preferably, the deep learning model in the step M5 includes an input layer, a convolutional layer, a pooling layer, a feature fusion layer, a category judgment model layer and an output layer;
the convolution layer performs characteristic extraction on the input layer, filters useless information and retains characteristic effective information;
the pooling layer performs dimension reduction processing on the effective information reserved after the convolution layer processing;
the characteristic fusion layer is used for performing cross-layer connection on different layers with the same dimensionality;
the category judgment layer quantizes the feature information obtained by the feature fusion layer into a probability value of a certain category;
the output layer outputs a vector [ m, n, c, s ] serving as a result after passing through the convolution layer, the pooling layer, the feature fusion layer and the category judgment layer, and represents a category confidence coefficient of each pixel value in an image, wherein m represents the image width, n represents the image height, and c represents the category; s represents the confidence.
Preferably, the step M6 includes: training all images in a training set in the divided data set, respectively training good images and defect images in the images during training, and stopping training when the difference between a prediction result and a real result is not obviously reduced after the training is carried out for a preset time.
The invention provides a metal powder injection molding appearance defect detection system, which comprises:
module M1: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data;
module M2: preprocessing the image data and expanding the image data to obtain processed image data;
module M3: labeling the processed image data by using labeling software;
module M4: converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion;
module M5: processing image data based on a computer vision deep neural network, and constructing a deep learning model;
module M6: training a deep learning model by using the divided data sets;
module M7: carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
the deep learning model is a mathematical model established between the image and the label, and carries out category judgment and position positioning on a certain object in the input image.
Preferably, the image data expansion in the module M2 includes: image rotation, random cropping, random boosting of gaussian noise, image scaling and/or slight projective transformation.
Preferably, the deep learning model in the module M5 includes a segmentation network and a classification network;
the classification category of each pixel point in the segmentation network learning image comprises a background pixel category and a defect pixel category;
the classification network judges each pixel point in the extracted background area and defect area on the basis of network segmentation to give the possibility that each pixel point belongs to the background pixel category and the defect pixel category, namely confidence.
Preferably, the deep learning model in the module M5 includes an input layer, a convolutional layer, a pooling layer, a feature fusion layer, a category judgment model layer and an output layer;
the convolution layer performs characteristic extraction on the input layer, filters useless information and retains characteristic effective information;
the pooling layer performs dimension reduction processing on the effective information reserved after the convolution layer processing;
the characteristic fusion layer is used for performing cross-layer connection on different layers with the same dimensionality;
the category judgment layer quantizes the feature information obtained by the feature fusion layer into a probability value of a certain category;
the output layer outputs a vector [ m, n, c, s ] serving as a result after passing through the convolution layer, the pooling layer, the feature fusion layer and the category judgment layer, and represents a category confidence coefficient of each pixel value in an image, wherein m represents the image width, n represents the image height, and c represents the category; s represents the confidence.
Preferably, said module M6 comprises: training all images in a training set in the divided data set, respectively training good images and defect images in the images during training, and stopping training when the difference between a prediction result and a real result is not obviously reduced after the training is carried out for a preset time.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention can effectively solve the problems of insufficient training samples and unbalanced positive and negative samples in industrial defect detection, and the trained model can achieve a good precision ratio and recall ratio;
2. the method utilizes marking software to mark image data, marks minor and tiny defects which are lower than standard defects as defects besides marking out the defect information identified by quality inspection during marking, ensures recall ratio, and ensures that marked data comprises original images and label files which describe key defect target information of the images;
3. the deep learning model is composed of a segmentation network and a classification network two-stage, and mainly comprises a convolution module, a pooling module, a feature fusion module, a category judgment module and an output module. The network model is designed in such a way, mainly aiming at solving the problems of difficult collection of industrial image samples and small quantity of defect samples, pixel information of the area where the defect is located is used for replacing the defect individual and sending the defect individual into network training, and the quantity of training samples is invisibly and greatly increased;
4. processing the characteristic graph by combining multiple downsampling with an algorithm of a large convolution kernel, realizing network design of a target of increasing a receptive field under a high-resolution image, and solving the problems of fine appearance defects and difficulty in extraction of industrial products;
5. the invention provides a new network with segmentation and classification effects to meet the requirement of training of small batches of samples with defects, and the invention can obtain good detection effect while fitting the shortage of training samples in an intelligent workshop, so that a deep learning method can get rid of the limitation of the shortage of training samples and can be really applied to the ground;
6. compared with the traditional manual detection mode, the MIM metal powder injection molding appearance defect detection method has high stability and high detection rate.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flow chart of the appearance defect detection of the present invention;
fig. 2 is a schematic diagram of a detection network structure according to the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
Example 1
The invention provides a metal powder injection molding appearance defect detection method, which comprises the following steps:
step M1: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data;
step M2: preprocessing the image data and expanding the image data to obtain processed image data;
step M3: labeling the processed image data by using labeling software;
step M4: converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion;
step M5: processing image data based on a computer vision deep neural network, and constructing a deep learning model;
step M6: training a deep learning model by using the divided data sets;
step M7: carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
the deep learning model is a mathematical model established between the image and the label, and carries out category judgment and position positioning on a certain object in the input image.
Specifically, the image data expansion in the step M2 includes: image rotation, random cropping, random boosting of gaussian noise, image scaling and/or slight projective transformation.
Specifically, the deep learning model in the step M5 includes a segmentation network and a classification network;
the classification category of each pixel point in the segmentation network learning image comprises a background pixel category and a defect pixel category;
the classification network judges each pixel point in the extracted background area and defect area on the basis of network segmentation to give the possibility that each pixel point belongs to the background pixel category and the defect pixel category, namely confidence.
Specifically, the deep learning model in the step M5 includes an input layer, a convolutional layer, a pooling layer, a feature fusion layer, a category judgment model layer, and an output layer;
the convolution layer performs characteristic extraction on the input layer, filters useless information and retains characteristic effective information;
the pooling layer performs dimension reduction processing on the effective information reserved after the convolution layer processing;
the characteristic fusion layer is used for performing cross-layer connection on different layers with the same dimensionality;
the category judgment layer quantizes the feature information obtained by the feature fusion layer into a probability value of a certain category;
the output layer outputs a vector [ m, n, c, s ] serving as a result after passing through the convolution layer, the pooling layer, the feature fusion layer and the category judgment layer, and represents a category confidence coefficient of each pixel value in an image, wherein m represents the image width, n represents the image height, and c represents the category; s represents the confidence and the result of the output layer indicates the probability s that each pixel in an image belongs to the class c, expressed as a matrix m, n, c, s.
Specifically, the step M6 includes: training all images in a training set in the divided data set, respectively training good images and defect images in the images during training, and stopping training when the difference between a prediction result and a real result is not obviously reduced after the training is carried out for a preset time.
The invention provides a metal powder injection molding appearance defect detection system, which comprises:
module M1: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data;
module M2: preprocessing the image data and expanding the image data to obtain processed image data;
module M3: labeling the processed image data by using labeling software;
module M4: converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion;
module M5: processing image data based on a computer vision deep neural network, and constructing a deep learning model;
module M6: training a deep learning model by using the divided data sets;
module M7: carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
the deep learning model is a mathematical model established between the image and the label, and carries out category judgment and position positioning on a certain object in the input image.
Specifically, the image data expansion in the module M2 includes: image rotation, random cropping, random boosting of gaussian noise, image scaling and/or slight projective transformation.
Specifically, the deep learning model in the module M5 includes a segmentation network and a classification network;
the classification category of each pixel point in the segmentation network learning image comprises a background pixel category and a defect pixel category;
the classification network judges each pixel point in the extracted background area and defect area on the basis of network segmentation to give the possibility that each pixel point belongs to the background pixel category and the defect pixel category, namely confidence.
Specifically, the deep learning model in the module M5 includes an input layer, a convolutional layer, a pooling layer, a feature fusion layer, a category judgment model layer, and an output layer;
the convolution layer performs characteristic extraction on the input layer, filters useless information and retains characteristic effective information;
the pooling layer performs dimension reduction processing on the effective information reserved after the convolution layer processing;
the characteristic fusion layer is used for performing cross-layer connection on different layers with the same dimensionality;
the category judgment layer quantizes the feature information obtained by the feature fusion layer into a probability value of a certain category;
the output layer outputs a vector [ m, n, c, s ] serving as a result after passing through the convolution layer, the pooling layer, the feature fusion layer and the category judgment layer, and represents a category confidence coefficient of each pixel value in an image, wherein m represents the image width, n represents the image height, and c represents the category; s represents the confidence and the result of the output layer indicates the probability s that each pixel in an image belongs to the class c, expressed as a matrix m, n, c, s.
Specifically, the module M6 includes: training all images in a training set in the divided data set, respectively training good images and defect images in the images during training, and stopping training when the difference between a prediction result and a real result is not obviously reduced after the training is carried out for a preset time.
Example 2
Example 2 is a modification of example 1
In order to more clearly illustrate the purpose, technical scheme and application value of the invention, the invention will be further described in detail with reference to the accompanying drawings and implementation items. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, the present invention provides a method for detecting appearance defects of metal powder injection molding, comprising:
carrying out appropriate data preprocessing and expansion on the collected good product image and defect image of the industrial field metal powder injection molding workpiece;
specifically, the method comprises the following steps:
the quantity of the collected good images and the quantity of the collected defect images are more than 400, wherein the proportion of the good images to the defect images is close to 3: 1.
the data expansion technology comprises image rotation, random clipping, random Gaussian noise addition, image scaling, slight transmission transformation and the like.
Marking the image data by using marking software, wherein in the marking process, besides marking the defect information identified by quality inspection, the slight and tiny defects lower than the standard defects are marked as defects together so as to ensure the recall ratio, and the marked data should contain an original image and a label file describing the key defect target information of the image;
specifically, the marking method comprises the steps of marking an image by using self-researched marking software Markimage, drawing an approximate handwriting graph of elongated defects such as scratches, cracks and the like with different thicknesses along the shape and the outline of the defects by using a painting brush function in the software, recording a point set passed by the handwriting and the thickness value of the painting brush, storing the point set and the thickness value of the painting brush in an xml file as a description file of the marked defects, and distinguishing the marked defect types by using a defect type field.
Carrying out data preprocessing and file format conversion on the manually marked image data set into a data set conforming to model training, and dividing the data set into a training set, a verification set and a test set;
specifically, when data preprocessing and file format conversion are carried out, an 'xml' file is read and analyzed, data information containing the same 'defect type' field in the file is extracted every time, the data information comprises a point set and a line width of a painting brush, then a pure black background picture with the same width and the same height as an original image is generated, a defect outline is drawn out by the pure white painting brush by utilizing the clicking and the line width of the painting brush extracted by the 'xml' and is stored as a mask picture as a label picture corresponding to the original image, the naming rule of the mask picture is 'original picture name' + '_ mask.png', and the storage path is consistent with the original image path; marking the good images to directly generate pure black background images with the same width and height as the original images as label images, wherein the storage path is consistent with the original image path; after all the images are labeled, the images and the labels are labeled according to the following steps of 8: 1: the 1 scale is divided into a training set, a validation set and a test set.
Designing and building a deep learning model for detecting the defects of the workpiece;
specifically, the deep learning model is composed of a segmentation network and a classification network two-stage, and mainly comprises a convolution module, a pooling module, a feature fusion module, a category judgment module and an output module. The network model is designed in such a way, mainly aiming at solving the problems of difficult collection of industrial image samples and small quantity of defect samples, pixel information of the area where the defect is located is used for replacing the defect individual to be sent to network training, and the quantity of training samples is invisibly and greatly increased.
Dividing classification categories to which each pixel point in the network learning image belongs, expressing the pixel category as a background by 0, expressing the pixel category as a defect by 1, and dividing an image into a background area and a defect area according to the categories; the classification network judges each pixel point in the extracted background area and defect area on the basis of the segmentation network to give the possibility that each pixel point belongs to a certain category, namely confidence.
The convolution layer is characterized in that the input layer is subjected to characteristic extraction, part of useless information is filtered, and most of effective information of the characteristics is reserved; the pooling layer is characterized in that the input layer is reduced to reduce the calculated amount; the characteristic fusion layer is characterized in that different layers with the same dimensionality are connected in a cross-layer mode to obtain richer characteristic information; the category judgment layer is characterized in that the feature information obtained by the feature fusion layer is quantized into a probability value of a certain category; the output layer features output vectors [ m, n, c, s ] after convolution pooling feature fusion and the like, and represent the category and confidence of each pixel value in an image as a result.
Circularly training a deep learning model by using the divided data sets;
specifically, the deep learning training method is characterized in that all images in a well-divided training set folder are trained, the number of training steps is set to be more than 1000, good images and defect images in a training set are trained in a single-double alternative mode during training, and a model corresponding to the number of steps is stopped to be output during training until loss is not reduced obviously and serves as an output model of the training.
Carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
when the appearance defects of the real scene workpiece are detected, the output model is used for detecting the workpiece formed by injecting the metal powder, and the result is output according to the vector [ m, n, c, s ].
Example 3
The invention provides a method for marking the appearance defects of metal powder injection molding, which is characterized by comprising the following steps of N1: drawing the appearance defect of metal powder injection molding along the shape and the outline of the defect;
step N2: recording a point set and a wire frame through which a drawing note passes, and storing the point set and the wire frame into a description file for marking defects;
step N3: distinguishing the marked defect types by using the defect type field;
step N4: reading and analyzing a description file for marking the defects, and extracting data information containing fields with the same defect type;
step N5: and drawing the defect outline by using the extracted data information, and storing the defect outline as a mask image as a label image corresponding to the original image.
Preferably, the step N1 includes: elongated defects including scratches and cracks having different thicknesses are drawn along the shape and contour of the defect.
Preferably, the step N2 includes: the point set and the brush thickness value which describe the passing of the note are recorded and saved in an XML file to be used as a description file for marking the defect.
Preferably, the step N4 includes: reading and analyzing the description file for marking the defects, and extracting data information containing fields with the same defect type, wherein the data information comprises a point set and a line frame of the brush.
Preferably, the step N5 includes: and generating a pure black background picture with the same width and height as the original picture, drawing the defect outline by using a pure white brush pen by using the point set and the line width extracted by the XML, and storing the defect outline as a mask picture as a label picture corresponding to the original picture.
The invention provides a marking system for metal powder injection molding appearance defects, which comprises
Module N1: drawing the appearance defect of metal powder injection molding along the shape and the outline of the defect;
module N2: recording a point set and a wire frame through which a drawing note passes, and storing the point set and the wire frame into a description file for marking defects;
module N3: distinguishing the marked defect types by using the defect type field;
module N4: reading and analyzing a description file for marking the defects, and extracting data information containing fields with the same defect type;
module N5: and drawing the defect outline by using the extracted data information, and storing the defect outline as a mask image as a label image corresponding to the original image.
Preferably, the module N1 includes: elongated defects including scratches and cracks having different thicknesses are drawn along the shape and contour of the defect.
Preferably, the module N2 includes: the point set and the brush thickness value which describe the passing of the note are recorded and saved in an XML file to be used as a description file for marking the defect.
Preferably, the module N4 includes: reading and analyzing the description file for marking the defects, and extracting data information containing fields with the same defect type, wherein the data information comprises a point set and a line frame of the brush.
Preferably, the module N5 includes: and generating a pure black background picture with the same width and height as the original picture, drawing the defect outline by using a pure white brush pen by using the point set and the line width extracted by the XML, and storing the defect outline as a mask picture as a label picture corresponding to the original picture.
Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (10)
1. A metal powder injection molding appearance defect detection method is characterized by comprising the following steps:
step M1: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data;
step M2: preprocessing the image data and expanding the image data to obtain processed image data;
step M3: labeling the processed image data by using labeling software;
step M4: converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion;
step M5: processing image data based on a computer vision deep neural network, and constructing a deep learning model;
step M6: training a deep learning model by using the divided data sets;
step M7: carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
the deep learning model is a mathematical model established between the image and the label, and carries out category judgment and position positioning on a certain object in the input image.
2. The method for detecting the appearance defects of metal powder injection molding according to claim 1, wherein the step M2 of expanding the image data comprises: image rotation, random cropping, random boosting of gaussian noise, image scaling and/or slight projective transformation.
3. The method for detecting the appearance defects of metal powder injection molding according to claim 1, wherein the deep learning model in the step M5 includes a segmentation network and a classification network;
the classification category of each pixel point in the segmentation network learning image comprises a background pixel category and a defect pixel category;
the classification network judges each pixel point in the extracted background area and defect area on the basis of network segmentation to give the possibility that each pixel point belongs to the background pixel category and the defect pixel category, namely confidence.
4. The method for detecting the appearance defects of the metal powder injection molding according to claim 3, wherein the deep learning model in the step M5 comprises an input layer, a convolution layer, a pooling layer, a feature fusion layer, a class determination model layer and an output layer;
the convolution layer performs characteristic extraction on the input layer, filters useless information and retains characteristic effective information;
the pooling layer performs dimension reduction processing on the effective information reserved after the convolution layer processing;
the characteristic fusion layer is used for performing cross-layer connection on different layers with the same dimensionality;
the category judgment layer quantizes the feature information obtained by the feature fusion layer into a probability value of a certain category;
the output layer outputs a vector [ m, n, c, s ] serving as a result after passing through the convolution layer, the pooling layer, the feature fusion layer and the category judgment layer, and each pixel value in an image is represented to obtain a category confidence coefficient; where m represents the image width, n represents the image height, c represents the category; s represents the confidence.
5. The method for detecting the appearance defects of metal powder injection molding according to claim 1, wherein the step M6 includes: training all images in a training set in the divided data set, respectively training good images and defect images in the images during training, and stopping training when the difference between a prediction result and a real result is not obviously reduced after the training is carried out for a preset time.
6. A metal powder injection molding appearance defect detection system, comprising:
module M1: collecting good product images and defect images of industrial field metal powder injection molding workpieces according to a preset proportion to obtain image data;
module M2: preprocessing the image data and expanding the image data to obtain processed image data;
module M3: labeling the processed image data by using labeling software;
module M4: converting the file format of the labeled image data set into an image data set conforming to deep learning model training, and dividing the data set into a training set, a verification set and a test set according to a preset proportion;
module M5: processing image data based on a computer vision deep neural network, and constructing a deep learning model;
module M6: training a deep learning model by using the divided data sets;
module M7: carrying out appearance defect detection on a real scene workpiece by using a deep learning model, and judging and quantifying a detection result;
the deep learning model is a mathematical model established between the image and the label, and carries out category judgment and position positioning on a certain object in the input image.
7. The metal powder injection molding appearance defect detection system of claim 6, wherein the image data expansion in module M2 comprises: image rotation, random cropping, random boosting of gaussian noise, image scaling and/or slight projective transformation.
8. The metal powder injection molding appearance defect detection system of claim 6, wherein the deep learning model in module M5 includes a segmentation network and a classification network;
the classification category of each pixel point in the segmentation network learning image comprises a background pixel category and a defect pixel category;
the classification network judges each pixel point in the extracted background area and defect area on the basis of network segmentation to give the possibility that each pixel point belongs to the background pixel category and the defect pixel category, namely confidence.
9. The metal powder injection molding appearance defect detection system of claim 8, wherein the deep learning model in module M5 includes an input layer, a convolution layer, a pooling layer, a feature fusion layer, a class determination model layer, and an output layer;
the convolution layer performs characteristic extraction on the input layer, filters useless information and retains characteristic effective information;
the pooling layer performs dimension reduction processing on the effective information reserved after the convolution layer processing;
the characteristic fusion layer is used for performing cross-layer connection on different layers with the same dimensionality;
the category judgment layer quantizes the feature information obtained by the feature fusion layer into a probability value of a certain category;
the output layer outputs a vector [ m, n, c, s ] serving as a result after passing through the convolution layer, the pooling layer, the feature fusion layer and the category judgment layer, and represents a category confidence coefficient of each pixel value in an image, wherein m represents the image width, n represents the image height, and c represents the category; s represents the confidence.
10. The metal powder injection molding appearance defect detection system of claim 6, wherein the module M6 comprises: training all images in a training set in the divided data set, respectively training good images and defect images in the images during training, and stopping training when the difference between a prediction result and a real result is not obviously reduced after the training is carried out for a preset time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010724049.7A CN111951232A (en) | 2020-07-24 | 2020-07-24 | Metal powder injection molding appearance defect detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010724049.7A CN111951232A (en) | 2020-07-24 | 2020-07-24 | Metal powder injection molding appearance defect detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111951232A true CN111951232A (en) | 2020-11-17 |
Family
ID=73339003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010724049.7A Pending CN111951232A (en) | 2020-07-24 | 2020-07-24 | Metal powder injection molding appearance defect detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111951232A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113240665A (en) * | 2021-06-04 | 2021-08-10 | 同济大学 | Industrial automatic surface defect detection method based on deep learning |
| CN113392915A (en) * | 2021-06-23 | 2021-09-14 | 宁波聚华光学科技有限公司 | Industrial part defect detection method based on deep learning |
| CN113466235A (en) * | 2021-03-19 | 2021-10-01 | 江苏立讯机器人有限公司 | Visual inspection module, defect inspection workstation and defect inspection method |
| CN113554589A (en) * | 2021-06-04 | 2021-10-26 | 奕目(上海)科技有限公司 | Semiconductor chip gold wire segmentation method and system based on deep learning |
| CN113592828A (en) * | 2021-08-03 | 2021-11-02 | 南京市特种设备安全监督检验研究院 | Nondestructive testing method and system based on industrial endoscope |
| CN113705531A (en) * | 2021-09-10 | 2021-11-26 | 北京航空航天大学 | Method for identifying alloy powder inclusions based on microscopic imaging |
| WO2022120665A1 (en) * | 2020-12-09 | 2022-06-16 | 电子科技大学 | Capacitance defect intelligent detection method based on deep learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107607554A (en) * | 2017-09-26 | 2018-01-19 | 天津工业大学 | A kind of Defect Detection and sorting technique of the zinc-plated stamping parts based on full convolutional neural networks |
| CN110232675A (en) * | 2019-03-28 | 2019-09-13 | 昆明理工大学 | Grain surface defects detection and segmenting device and method under a kind of industrial environment |
| CN110443791A (en) * | 2019-08-02 | 2019-11-12 | 西安工程大学 | A kind of workpiece inspection method and its detection device based on deep learning network |
| CN210322816U (en) * | 2019-05-30 | 2020-04-14 | 常州微亿智造科技有限公司 | Intelligent AI appearance detection system |
| CN111179229A (en) * | 2019-12-17 | 2020-05-19 | 中信重工机械股份有限公司 | Industrial CT defect detection method based on deep learning |
-
2020
- 2020-07-24 CN CN202010724049.7A patent/CN111951232A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107607554A (en) * | 2017-09-26 | 2018-01-19 | 天津工业大学 | A kind of Defect Detection and sorting technique of the zinc-plated stamping parts based on full convolutional neural networks |
| CN110232675A (en) * | 2019-03-28 | 2019-09-13 | 昆明理工大学 | Grain surface defects detection and segmenting device and method under a kind of industrial environment |
| CN210322816U (en) * | 2019-05-30 | 2020-04-14 | 常州微亿智造科技有限公司 | Intelligent AI appearance detection system |
| CN110443791A (en) * | 2019-08-02 | 2019-11-12 | 西安工程大学 | A kind of workpiece inspection method and its detection device based on deep learning network |
| CN111179229A (en) * | 2019-12-17 | 2020-05-19 | 中信重工机械股份有限公司 | Industrial CT defect detection method based on deep learning |
Non-Patent Citations (3)
| Title |
|---|
| LONG JON ET AL: "Fully convolutional network for semantic segmentation", 《2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 乐毅等: "《深度学习 Caffe之经典模型详解与实战》", 31 December 2016 * |
| 梁智聪: "基于卷积神经网络的工件表面缺陷检测系统", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022120665A1 (en) * | 2020-12-09 | 2022-06-16 | 电子科技大学 | Capacitance defect intelligent detection method based on deep learning |
| CN113466235A (en) * | 2021-03-19 | 2021-10-01 | 江苏立讯机器人有限公司 | Visual inspection module, defect inspection workstation and defect inspection method |
| CN113240665A (en) * | 2021-06-04 | 2021-08-10 | 同济大学 | Industrial automatic surface defect detection method based on deep learning |
| CN113554589A (en) * | 2021-06-04 | 2021-10-26 | 奕目(上海)科技有限公司 | Semiconductor chip gold wire segmentation method and system based on deep learning |
| CN113554589B (en) * | 2021-06-04 | 2023-05-02 | 奕目(上海)科技有限公司 | Deep learning-based semiconductor chip gold thread segmentation method and system |
| CN113392915A (en) * | 2021-06-23 | 2021-09-14 | 宁波聚华光学科技有限公司 | Industrial part defect detection method based on deep learning |
| CN113592828A (en) * | 2021-08-03 | 2021-11-02 | 南京市特种设备安全监督检验研究院 | Nondestructive testing method and system based on industrial endoscope |
| CN113705531A (en) * | 2021-09-10 | 2021-11-26 | 北京航空航天大学 | Method for identifying alloy powder inclusions based on microscopic imaging |
| CN113705531B (en) * | 2021-09-10 | 2023-06-20 | 北京航空航天大学 | Identification method of alloy powder inclusions based on microscopic imaging |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111951232A (en) | Metal powder injection molding appearance defect detection method and system | |
| CN108074231B (en) | Magnetic sheet surface defect detection method based on convolutional neural network | |
| CN109190752B (en) | Image semantic segmentation method based on global features and local features of deep learning | |
| CN106875381B (en) | Mobile phone shell defect detection method based on deep learning | |
| CN110992317B (en) | PCB defect detection method based on semantic segmentation | |
| Saberironaghi et al. | Defect detection methods for industrial products using deep learning techniques: A review | |
| CN108960245B (en) | Tire mold character detection and recognition method, device, equipment and storage medium | |
| CN109615016B (en) | Target detection method of convolutional neural network based on pyramid input gain | |
| CN111179251A (en) | Defect detection system and method based on twin neural network and by utilizing template comparison | |
| Zheng et al. | Research on detecting bearing-cover defects based on improved YOLOv3 | |
| CN113643268B (en) | Industrial product defect quality inspection method and device based on deep learning and storage medium | |
| CN116188475B (en) | Intelligent control method, system and medium for automatic optical detection of appearance defects | |
| CN111932511B (en) | Electronic component quality detection method and system based on deep learning | |
| CN111582126B (en) | Pedestrian re-recognition method based on multi-scale pedestrian contour segmentation fusion | |
| Tao et al. | Industrial weak scratches inspection based on multifeature fusion network | |
| CN112766110A (en) | Training method of object defect recognition model, object defect recognition method and device | |
| CN113177518A (en) | Vehicle weight identification method recommended by weak supervision area | |
| CN109034121B (en) | Commodity identification processing method, device, equipment and computer storage medium | |
| CN111161213A (en) | Industrial product defect image classification method based on knowledge graph | |
| CN115272204A (en) | Bearing surface scratch detection method based on machine vision | |
| CN114494780A (en) | Semi-supervised industrial defect detection method and system based on feature comparison | |
| CN115035082B (en) | Method for detecting defects of transparent parts of aircraft based on YOLOv4 improved algorithm | |
| CN116052222A (en) | Cattle face recognition method for naturally collecting cattle face image | |
| Zhou et al. | DeepInspection: Deep learning based hierarchical network for specular surface inspection | |
| CN114972316A (en) | Battery case end surface defect real-time detection method based on improved YOLOv5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201117 |