CN113902710A - Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm - Google Patents
Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm Download PDFInfo
- Publication number
- CN113902710A CN113902710A CN202111185034.9A CN202111185034A CN113902710A CN 113902710 A CN113902710 A CN 113902710A CN 202111185034 A CN202111185034 A CN 202111185034A CN 113902710 A CN113902710 A CN 113902710A
- Authority
- CN
- China
- Prior art keywords
- model
- abnormal
- image
- detecting
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 121
- 230000007547 defect Effects 0.000 title claims abstract description 120
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000002159 abnormal effect Effects 0.000 claims abstract description 100
- 238000012549 training Methods 0.000 claims abstract description 58
- 238000013135 deep learning Methods 0.000 claims abstract description 44
- 230000005856 abnormality Effects 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 15
- 238000010586 diagram Methods 0.000 claims description 11
- 239000013598 vector Substances 0.000 claims description 11
- 230000002950 deficient Effects 0.000 claims description 8
- 238000004821 distillation Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 7
- 239000002344 surface layer Substances 0.000 abstract description 6
- 238000013480 data collection Methods 0.000 abstract description 5
- 239000010410 layer Substances 0.000 description 17
- 238000005516 engineering process Methods 0.000 description 13
- 238000004519 manufacturing process Methods 0.000 description 7
- 238000013136 deep learning model Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013140 knowledge distillation Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000012372 quality testing Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000004753 textile Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
Abstract
The invention belongs to the technical field of deep learning anomaly detection, and discloses a method and a system for detecting surface defects of industrial parts based on an anomaly detection algorithm, wherein images of products to be detected are obtained, and surface layer image data of the good products to be detected are collected and shot under an industrial camera; inputting the image into a teacher-student anomaly detection network model for training; detecting the defect data and the good product data by using a model output after training is finished; returning the detection result to the client, and displaying the detection result by the client; and transmitting the detected abnormal region and the size of the region to a client of the software, and judging whether the defect needs to be eliminated or not through an area threshold on the software. The detection of abnormal areas on the surface of most parts with simple surface structures can be met. Compared with target detection, the method saves a large amount of model training and data collection time, and has great promotion effect on the project to be promoted and accepted as soon as possible.
Description
Technical Field
The invention belongs to the technical field of deep learning anomaly detection, and particularly relates to a method and a system for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm.
Background
At present, in the field of industrial defect detection, a defect identification technology by utilizing deep learning target detection tends to be mature, but the defect of target detection also comes along with the mature technology, because the number of defects of parts is small, and the types of defects are uncertain, in a conventional deep learning target detection algorithm, a large amount of defect data needs to be provided for a neural network to learn, but because the imaging of the defects has influence factors such as positions, shapes, light sources and the like, different factors can be combined into various defects, and a very small number of unknown defects with large forms can occur, so that the learning of an AI target detection algorithm becomes abnormally difficult. And the collection of real defects of industrial parts is difficult, the delivery date of the equipment can be continuously delayed, and the production side can continuously invest in labor-saving cost.
Aiming at the defects of most industrial parts, the quality inspection is mostly carried out by adopting a manual naked eye quality inspection mode. However, this method has the following drawbacks: the efficiency is low: the efficiency of checking parts actually tests the proficiency of one person, the higher detection efficiency can be realized if the working time is longer, but the personal fatigue and lazy inertia can be increased along with the increase of the working time, so that the detection efficiency of quality testing personnel can be reduced; risk of missed detection: as the working time increases, the personal attention also decreases, the risk of missed detection is brought, the machine is not tired, and the problem does not exist; hard to define: because the sizes of industrial defects are all in millimeter level, the defects in millimeter level are difficult to be distinguished manually by naked eyes; the quantitative analysis is difficult: the manual defect judgment cannot carry out data statistics, and the factory intelligence is hindered; the labor cost is high: enterprises pay corresponding expenses for manual quality inspection with high uncertainty continuously, and the cost benefit is low; furthermore, the traditional visual algorithm is adopted, the writing limitation of the manual non-standardized algorithm is large, the defect judgment is carried out only by judging the gray value or the area, and the misjudgment risk is increased; and the algorithm compiling process is complex, the universality is not high, if the models are many, algorithm engineers are consumed for compiling the algorithm for each model, and the labor cost is high. And if the deep learning target detection method is adopted, the detection capability will be lost when new characteristic defects are encountered because the defect data collection is difficult and the defect type number is fixed.
Based on the method, after learning training is carried out on data of the non-defective part, an output model is deployed on equipment, defect detection is carried out on image data shot by a camera on the equipment, and detection results are screened according to the area of an abnormal area and the size of an abnormal value.
Through the above analysis, the problems and defects of the prior art are as follows:
(1) defect unknown: only known defect classes can be continuously increased, and if defect classes of unknown types appear in the future, the equipment loses the function, and immeasurable loss is brought to a production side.
(2) Defect collection is difficult: the defect data set is difficult to collect, the difference between artificial or synthetic defects and real defects is large, low-quality sample data exists, and the data collection period is long.
(3) Low-frequency defect interception is difficult: even if the defect is known and has a sufficient category of data set, the defect with characteristics which are not similar to the category appears, and a missing detection situation may occur.
The difficulty in solving the above problems and defects is: due to the limitation of the target detection range, the device can hardly intercept the defects of unknown categories or subclasses of the defects of the known categories in the data set. Low frequency samples are difficult to collect and difficult for algorithms to detect.
The significance of solving the problems and the defects is as follows: through the deep learning anomaly detection algorithm, the problems that the low-frequency defect data set is difficult to collect and the unknown class defects are difficult to intercept are solved. The high standard of zero missing detection in the industrial field is further. The project cycle is effectively reduced, and benefits are brought to enterprises more quickly.
Disclosure of Invention
In order to overcome the problems in the related art, the disclosed embodiment of the invention provides a method and a system for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm. The technical scheme is as follows:
the method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm comprises the following steps:
acquiring images of parts with simple structures, such as steel plates and textiles, on an object to be detected; collecting surface image data of a good product to be detected, which is shot under an industrial camera, and acquiring defect-free image data of the part through detection equipment;
inputting the image into a teacher-student anomaly detection network model for training, so that the model extracts and learns the characteristics of the image of the non-defective part, and finally outputting a model for modeling the current task after the training is finished;
step three, deploying the model generated after training on detection equipment, and detecting parts on the assembly line; after receiving the image, the model deduces whether each small area in the image is abnormal or not, and if the model detects the abnormality, the coordinates, the area and the abnormal score of the defect position of the equipment client are returned;
after receiving the detection result, the client judges whether the defect needs to be eliminated or not through an area threshold and an abnormal score on software, and displays the detection result larger than the threshold; the subsequent software transmits the rejection signal to a mechanical plc terminal;
and step five, judging whether the part is to be removed to a defective material channel or not after the plc receives a signal from the software.
In one embodiment, the second step specifically includes:
step 2.1, distilling the collected part data set on the basis of a reasnet18 pre-training model by taking a resnet18 pre-training model as a distillation network of the teacher, so that the teacher model inherits the characteristics of the resnet18 and has the characteristics of same feature extraction capacity and smaller body size;
step 2.2: training three student models in the collected data set for the distilled teacher model;
step 2.3: detecting defects by using a teacher model and three student models obtained by training as detection modules of an algorithm; and performing feature extraction, comparison and judgment on the image, and outputting the position of the abnormal area and the size of the abnormal value in the image.
In one embodiment, in step 2.2, the obtained good product data is trained by using a teacher model, and three student models having the same network structure as the teacher model are taught in the training process, that is, the predicted output of the teacher model is used as the group route of the output of the student model, and then the network nodes in the three student models are updated by back propagation, so that the output of the student model is similar to the output of the teacher model, and the condition that the four have similar outputs to the same characteristic diagram in the training process is ensured.
In one embodiment, in step 2.3, a specific method for detecting defects includes: and reasoning a patch area on the characteristic diagram, if the characteristic vectors output by the four models are similar, the patch area can be regarded as a qualified patch area, and if the characteristic vectors are not similar, the patch area is regarded as an abnormal patch area.
In one embodiment, in step 2.3, a specific method for detecting defects includes: calculating the average value output by the three students, comparing the average value with a teacher model, and considering the average value as an abnormal patch area if the phase difference is dozens; by calculating the variance of the three student outputs, if the variance is large, the three student outputs are considered to be not similar and are considered to be a patch area with an exception.
In one embodiment, the resnet18 pre-trained model in the distillation network consists of 2 convolutional layers of size 3 × 3 and number 64 convolutional kernels; 2 convolutional layers of size 3 × 3 and number 128 convolutional kernels; 2 convolutional layers of size 3 × 3 and number 256 convolutional kernels; 2 convolution layers with the size of 3 multiplied by 3 and the number of 512 convolution kernels are formed by finally outputting 1 multiplied by 1 dimension through full-connection linear transformation;
in one embodiment, the specific construction process of the resnet18 pre-training model in the distillation network is as follows:
performing feature extraction on a region with the size of 65 x 65 pixel values in an original input image through three groups of convolution, activation and pooling modules, performing two-layer convolution, inputting the result into a full-connection layer, and reducing the output dimension to 1 x 128 feature vectors;
by comparing the outputs of the resnet18 pre-trained model and the teacher model, the teacher model can finally inherit the feature extraction capability of the resnet18 pre-trained model through training.
In one embodiment, the anomaly region includes: position information, size information, and abnormality score information of the abnormal region.
Another object of the present invention is to provide a system for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm, which implements the method for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm, the system comprising:
the model training module is used for acquiring a normal sample image of the data to be detected, inputting the normal sample image into a teacher-student anomaly detection algorithm, training and generating a model;
the anomaly detection model is used for acquiring an image of a part to be detected, inputting the image into an anomaly detection algorithm based on deep learning, and detecting and positioning an anomaly region; and detecting the obtained image by using the trained model, and if an abnormal region is found, reasoning the position and the abnormal value of the abnormal region on the surface of the part.
In one embodiment, the system for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm further comprises:
and the defect judgment module is used for jointly determining whether the area is a defect or not through the set area and the abnormal value according to the area size and the abnormal value size of the abnormal area calculated by the algorithm.
And the data analysis module is used for judging the abnormality of the part according to the size of the abnormal area and the abnormal value acquired from the model. And if the area and the abnormal value are larger than the set value, the part is considered to have the abnormal area.
By combining all the technical schemes, the invention has the advantages and positive effects that:
first, the detection of abnormal areas on the surface of most parts with uniform and simple surface structures can be met.
And secondly, compared with target detection, a large amount of model training and data collection time is saved, and great promotion effect is provided for project promotion and acceptance as soon as possible.
Thirdly, the qualitative rejection of defect classes may not be accurate for the classes with similar features, and it is difficult to train classes. This system will not have such a problem, and only the characteristics of the defect are evaluated abnormally, and the high abnormal score can be regarded as a defect. And target detection does not have the capability of detecting the characteristics of unseen new classes or defined classes.
Fourthly, reasoning time is fast, the number of parts in an industrial scene is large, production beat is compact, the detection rate of the anomaly detection system is fast, and the requirements of most projects are met.
Fifthly, aiming at the defect of large area abnormity, the detection accuracy can reach 95%.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a flowchart of a method for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm according to an embodiment of the present invention.
Fig. 2 is a logic diagram of algorithm anomaly detection in the method for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm according to the embodiment of the present invention.
Fig. 3 is a logic diagram of algorithm anomaly detection in the method for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm according to the embodiment of the present invention.
FIG. 4 is a diagram illustrating the image detection effect of the surface of the heat sink according to the embodiment of the present invention;
wherein, a is an image obtained by mapping an abnormal region to an original image after the detected image is post-processed, and b is an abnormal detection image.
FIG. 5 is an image effect diagram of the inner wall of the coupling teeth of the transmission provided by the embodiment of the invention;
where a is an original gray scale image and b is an abnormality detection image.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather should be construed as broadly as the present invention is capable of modification in various respects, all without departing from the spirit and scope of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. As used herein, the terms "vertical," "horizontal," "left," "right," and the like are for purposes of illustration only and are not intended to represent the only embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like. The image target detection technology in the embodiment of the invention applies an artificial intelligence technology, and for convenience of understanding, the following explains terms related to the embodiment of the invention:
firstly, deep learning: the method is a new research direction in the field of machine learning, and refers to the internal rule and the expression level of learning sample data, a proper amount of neuron calculation nodes and a multilayer operation hierarchical structure are established through design, and a function relationship from input to output is established through network learning and tuning, so that the realistic incidence relationship is approached as much as possible;
II, modeling: the method belongs to a deep learning model for simulating the brain structure in the field of deep learning. In the field of deep learning, models are often used to model more complex tasks. The scale of the deep learning model, including depth, width and calculation mode, can be set according to specific tasks. The deep learning model has strong learning and expression capabilities, and is widely applied to the fields of natural understanding, machine vision, advertisement putting and the like.
Thirdly, training: the training process of deep learning is a process of iteratively adjusting each weight parameter in the deep learning model according to the input and output of the training sample until the model converges, which is also referred to as a learning process of the deep learning model. The training sample is labeled data, the input of the training sample comprises the data and labeled content, and the output of the training sample refers to a prediction result obtained after the model infers the sample. And updating the weight parameters of the model by calculating the difference between the marked content and the prediction result so as to reduce the difference between the prediction result and the marked content. Each iteration performs the operations of calculating the differences and updating the weights of the parameters.
Fourthly, a Teacher-student algorithm: the method is a visual anomaly detection algorithm, and is used for detecting the area which does not accord with the integral sample distribution of the model in the image, and performing abnormal score evaluation on the abnormal area to realize the detection of the abnormal area.
The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm provides a new detection scheme for the object to be detected with consistent image feature expression after the camera is imaged, can well solve the problem of missed detection under the condition of insufficient defect data set, can reduce the data accumulation requirement due to the deep learning requirement, and reduces the missed detection risk caused by the lack of data quantity of various unknown defects.
The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm comprises the following steps:
s101: acquiring an image of a simple-structure part on an object to be detected, collecting surface image data of a good part to be detected, which is shot by an industrial camera, and acquiring defect-free data of the corresponding part through detection equipment;
s102: inputting the image into a teacher-student anomaly detection network model for training, detecting an anomaly region of the whole image, and positioning the anomaly region; enabling the model to remember the non-defective features of the part and produce a model;
s103: detecting the defect data and the good product data by using a model produced after training is finished, and reasoning whether each small area in the image is abnormal or not after the model receives the image; if the model detects the abnormity, the defect area and the abnormal score of the defect area are returned to the client;
s104: returning the detection result to the client, and displaying the detection result by the client; transmitting the abnormal region and the size of the region detected in the step 2 to a client of the software, and judging whether the defect needs to be eliminated or not through an area threshold on the software;
s105: and the client judges whether the defect needs to be eliminated or not according to the defect size and the abnormal point value returned by the model.
In step S102, inputting the image into a teacher-student abnormality detection network model for training, detecting an abnormal region of the whole image, and positioning the abnormal region; the specific method comprises the following steps:
step 2.1: the knowledge distillation was performed using the published Resnet18 pre-trained model with robust feature extraction on a common dataset, and yielded a small volume of a model that inherited the same strong feature extraction capability, the teacher model.
Step 2.2: the acquired non-defective product data are trained by using the teacher model, three student models with the same network structure are taught in the training process, namely the predicted output of the teacher model is used as the Ground route output by the student models, then the network nodes in the three student models are updated by back propagation, and finally the output of the student models is similar to the output of the teacher model.
Step 2.3: finally, three student models obtained through training and the teacher model are used for defect detection, and through reasoning a patch area on the feature diagram at the same time, if feature vectors output by the four models are similar, the three models can be regarded as an ok patch, and if the feature vectors are not similar, the three models are regarded as an abnormal patch area; the method specifically comprises the following steps:
calculating the average value output by the three students, comparing the average value with a teacher model, and considering an abnormal patch if the phase difference is dozen; calculating the variance of the outputs of the three students, and if the variance is large, considering that the outputs of the three students are not similar and considering that the outputs of the three students are a patch with abnormal features; the en-route abnormal area is determined through the two modes.
In one embodiment, the distillation model resnet18 in step 2.1 is composed of 2 convolutional layers with a size of 3 × 3 and a number of 64 convolutional kernels, 2 convolutional layers with a size of 3 × 3 and a number of 128 convolutional kernels, 2 convolutional layers with a size of 3 × 3 and a number of 256 convolutional kernels, two convolutional layers with a size of 3 × 3 and a number of 512 convolutional kernels, and the final output dimension is 1 × 1 through full-connected linear transformation.
The reference model for distilling it consists of 5 convolutional layers and 3 max pooling layers, with one fully connected layer that varies linearly. Five convolutional layers include 3 5 × 5, 128 in number, one 4 × 4 256 in number, one 1 × 1 in number of 1 convolutional kernels for dimensionality reduction, and 3 2 × 2 max pooling layers for feature amplification.
The specific process is as follows: the method comprises the steps of performing feature extraction on a region with the size of 65 x 65 pixel values in an original input image through three groups of convolution, activation and pooling modules, performing two-layer convolution, inputting the result into a full-connection layer, and reducing the output dimension to 1 x 128 feature vectors. By comparing the outputs of resnet18 and the teacher model, the teacher model can finally inherit the feature extraction capability of resnet18 through training.
In one embodiment, the teacher model trains three student models in this dataset. The structure of the Teacher model is the same as that of the student model, and the condition that the four have similar outputs to the same characteristic diagram in the training process is ensured.
In S104, the abnormality information includes a position, a size, and an abnormality score of the abnormal region.
The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm can meet the requirement of detecting the anomaly areas on the surface of the part, which is uniform and simple in most surface layer structures, and has uniform and regular shapes as the detection effect of the surface image of the radiator shown in figure 4; the image a in fig. 4 is an image in which the detected image is post-processed to map the abnormal area to the original image, and the image b in fig. 4 is an abnormal detection detected image, by performing abnormal value calculation on each block of the image, the abnormal area of the image can be detected, and the effect of dividing the abnormal defect area mask can be achieved.
The upper image is a radiator surface image, the shape of the upper image is uniform and regular, the left side is an image of a detected image which is subjected to post-processing to map an abnormal area to an original image, the right side is an abnormal detection image, the abnormal area of the image can be detected by calculating the abnormal value of each area on the image, and the effect of dividing the abnormal defect area mask can be achieved.
The invention also provides a system for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm, which comprises the following steps:
and the model training module is used for acquiring a normal sample image of the data to be detected, inputting the normal sample image into a teacher-student anomaly detection algorithm, training and generating a model.
The anomaly detection model is used for acquiring an image of a part to be detected, inputting the image into an anomaly detection algorithm based on deep learning, and detecting and positioning an anomaly region; and detecting the obtained image by using the trained model, and if an abnormal region is found, reasoning the position and the abnormal value of the abnormal region on the surface of the part.
And the defect judgment module is used for jointly determining whether the area is a defect or not through the set area and the abnormal value according to the area size and the abnormal value size of the abnormal area calculated by the algorithm.
And the data analysis module is used for judging the abnormality of the part according to the size of the abnormal area and the abnormal value acquired from the model. And if the area and the abnormal value are larger than the set value, the part is considered to have the abnormal area.
The system for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm detects the anomalous region on the surface of the part under the condition of no defect data and without longer training period and data set accumulation.
Example (b):
the method comprises the steps that firstly, data collection is carried out on models to be detected, parts enter equipment and then arrive at an abnormal detection station, the parts are shot through a camera of a fixed machine position, the image areas of the parts obtained after shooting are guaranteed to be fixed in position and close in characteristics.
And step two, transmitting the image data acquired in the step 1 to an equipment client, and transmitting the image data to an anomaly detection algorithm for defect detection after the client receives the image.
And step three, after the position and the abnormal value of the abnormal area are deduced by the abnormal detection algorithm, returning the json string forming form to the client on the equipment.
And step four, screening the abnormal region size and the value returned by the algorithm by the client on the equipment, firstly judging the abnormal value size, if the abnormal value size is larger than the set value of the equipment, continuously judging the area of the abnormal region, and if the abnormal value size is larger than the set threshold value of the equipment, determining that the abnormal region area is a defect.
As a preferred embodiment of the present invention, before step S01, the method further includes the following steps of generating a system detection module: and (3) acquiring the part image in advance, manually confirming that no abnormity exists on the image, inputting the image and the labeled data into a network in the second step for model training, distilling out a teacher model, training a student model through the distilled teacher model, enabling three student models to learn deep feature distribution of the teacher model in the abnormal-free data, and judging a field area in the image through the distilled teacher model and the student model.
In step two, the model obtaining manner for detection is as shown in the figure, and specifically includes:
step 2.1: the knowledge distillation was performed using the published Resnet18 pre-trained model with robust feature extraction on a common dataset, and yielded a small volume of a model that inherited the same strong feature extraction capability, the teacher model.
Step 2.2: the acquired non-defective product data are trained by using the teacher model, three student models with the same network structure are taught in the training process, namely the predicted output of the teacher model is used as the Ground route of the output of the student model, then the network nodes in the three student models are updated by back propagation, and finally the output of the student model is similar to the output of the teacher model.
Step 2.3: and finally, carrying out defect detection by using three student models and a teacher model obtained through training, and simultaneously reasoning a patch region on the feature diagram, wherein if feature vectors output by the four models are similar, the patch can be regarded as an ok patch, and if the feature vectors are not similar, the patch region with abnormality is regarded as an ok patch.
In step three, the model performs inference prediction to generate four results, as shown in fig. 2, one teacher model and three yolo models. The abnormal area is judged by two aspects.
Step 3.1: by calculating the output errors of the teacher model and the three student models, the student networks cannot regress the output of the teacher network for the abnormal regions in the graph because the student networks do not see descriptors output by the teacher network in the abnormal regions during training. Therefore, if the output error of both is larger, the region is considered to be an abnormal region.
Step 3.2: when the three student models reason each patch in the graph, as the student networks learn the feature output of the same teacher model in a normal sample, the prediction of the student networks in regions without abnormality is similar, the results predicted by regions which are not seen in training are different, the output variance of the three student models can be calculated based on the result, and if the calculated variance is large, the output of the three student models is proved to be different, so that the student networks are considered to be abnormal regions.
For solving the problems in the prior art, the embodiment provides a method for detecting an industrial part under non-defective data, which is applied to an online detection scene of a part surface layer on a production line, and the invention is described in detail below.
Acquiring surface layer images of parts at all angles on each machine position camera, respectively sending the surface layer images to each operation server through a data distributor for operation processing, and inputting the images received by each server into a teacher-student-based anomaly detection algorithm for detecting defect areas on the images;
each server outputs the detection output result through gathering, and the output data comprises: and the position coordinates and the area of the defect region on the surface of the part are larger than the abnormal defect fraction. After receiving the data of each server, the detection system can count the number of each defect and remove the defect according to a preset defect parameter threshold value.
In the embodiment of the invention, when the surface image of the part is acquired, the part image on the production line can be synchronously acquired on line, so that the part on the production line is detected on line. Due to the fact that the reasoning time of the anomaly detection algorithm is short, the defect detection of most industrial scenes can be met, and the influence of the detection process on the industrial production speed is greatly reduced.
The defects in the embodiment of the invention comprise various abnormal conditions possibly appearing on the surface layer of the part, such as floating foreign matters falling on the part, collision and material shortage caused by a production process, rusting and the like, and the existing visual inspection by naked eyes has the risk of visual fatigue and the possibility of missing inspection. And the traditional visual algorithm is difficult to compile and has poor stability. Deep learning visual inspection requires a large amount of defect data for support, the period is long, and the defect types are relatively fixed. The algorithm in the embodiment of the invention overcomes the difficulties, greatly reduces various defects of defect detection in the industrial scene, and improves the capability of deep learning of visual detection defects in the industrial scene.
The deep learning algorithm teacher-student is adopted as a main algorithm of defect detection, a resnet50 algorithm with powerful feature extraction capability is used as a distillation network of a teacher model, so that the obtained small teacher network can also have efficient feature extraction capability and rapid reasoning capability, and the student network can also have the same feature extracted from the data set as the teacher model by teaching three student networks, so that the industrial scene efficient detection is met. And the size and the abnormality score value of the abnormal region can be calculated through regression of the abnormal region by the model.
In the embodiment of the invention, the anomaly detection model is obtained by deep learning and model training in advance. Specifically, a plurality of training abnormal-free samples are obtained in advance and manually confirmed; inputting the training sample into a teacher-student-based anomaly detection algorithm for training, and adjusting model parameters aiming at modified parts and defects to obtain the anomaly detection model
As shown in the image of the inner wall of the gear box combined tooth in fig. 5, the graph a in fig. 5 is an original gray scale graph, the graph b is an abnormality detection image, the right side in the graph b is an index for measuring an abnormal area, the more abnormal area has darker color, the graph can show that the type detection of the model does not carry out classification qualification on the defects, and only the area which is considered to be large in abnormality is detected
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure should be limited only by the attached claims.
Claims (10)
1. A method for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm is characterized by comprising the following steps of:
acquiring an image of a simple-structure part on an object to be detected, collecting surface image data of a good product to be detected, which is shot by an industrial camera, and acquiring defect-free data of corresponding parts through detection equipment;
inputting the image into a teacher-student abnormality detection network model for training, detecting an abnormal region of the whole image, and positioning; enabling the model to remember the non-defective features of the part and produce a model;
step three, detecting the defect data and the good product data by using a model produced after training is finished; after the model receives the image, whether each small area in the inference image is abnormal or not is judged; if the model detects the abnormity, the defect area and the abnormity score are returned to the client;
step four, returning the detection result to the client, and displaying the detection result by the client; the detected abnormal region and the size of the region are transmitted to a client of the software, and whether the defect needs to be eliminated is judged through an area threshold on the software;
and step five, the client judges whether the defect needs to be eliminated or not according to the defect size and the abnormal score value returned by the model.
2. The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm according to claim 1, wherein the second step specifically comprises the following steps:
step 2.1, distilling the collected part data set on the basis of a reasnet18 pre-training model by taking a resnet18 pre-training model as a distillation network of the teacher, so that the teacher model inherits the characteristics of the resnet18 and has the characteristics of same feature extraction capacity and smaller body size;
step 2.2: training three student models in the collected data set for the distilled teacher model;
step 2.3: detecting defects by using a teacher model and three student models obtained by training as detection modules of an algorithm; and performing feature extraction, comparison and judgment on the image, and outputting the position of the abnormal area and the size of the abnormal value in the image.
3. The method for detecting the defects of the surface of the industrial part based on the deep learning anomaly detection algorithm is characterized in that in step 2.2, a teacher model is used for training acquired good product data, three student models with the same network structure are taught in the training process, namely the predicted output of the teacher model is used as the group channel of the output of the student model, then the network nodes in the three student models are updated through back propagation, finally the output of the student model is similar to the output of the teacher model, and the condition that the four models have similar output to the same characteristic diagram in the training process is ensured.
4. The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm according to claim 2, wherein in the step 2.3, the specific method for detecting the defects comprises the following steps: and reasoning a patch area on the characteristic diagram, if the characteristic vectors output by the four models are similar, the patch area can be regarded as a qualified patch area, and if the characteristic vectors are not similar, the patch area is regarded as an abnormal patch area.
5. The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm according to claim 2, wherein in the step 2.3, the specific method for detecting the defects comprises the following steps: calculating the average value output by the three students, comparing the average value with a teacher model, and considering the average value as an abnormal patch area if the phase difference is dozens; by calculating the variance of the three student outputs, if the variance is large, the three student outputs are considered to be not similar and are considered to be a patch area with an exception.
6. The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm is characterized in that a resnet18 pre-training model in the distillation network consists of 2 convolution layers with the size of 3 x 3 and the number of 64 convolution kernels; 2 convolutional layers of size 3 × 3 and number 128 convolutional kernels; 2 convolutional layers of size 3 × 3 and number 256 convolutional kernels; 2 convolution layers with the size of 3 multiplied by 3 and the number of 512 convolution kernels are formed by 1 multiplied by 1 of the final output dimension through full-connection linear transformation.
7. The method for detecting the defects on the surface of the industrial part based on the deep learning anomaly detection algorithm as claimed in claim 5, wherein the specific construction process of the resnet18 pre-training model in the distillation network is as follows:
performing feature extraction on a region with the size of 65 x 65 pixel values in an original input image through three groups of convolution, activation and pooling modules, performing two-layer convolution, inputting the result into a full-connection layer, and reducing the output dimension to 1 x 128 feature vectors;
by comparing the outputs of the resnet18 pre-trained model and the teacher model, the teacher model can finally inherit the feature extraction capability of the resnet18 pre-trained model through training.
8. The method for detecting defects on the surface of an industrial part based on the deep learning anomaly detection algorithm according to claim 1, wherein the anomaly region comprises: position information, size information, and abnormality score information of the abnormal region.
9. A system for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm, which implements the method for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm according to any one of claims 1 to 8, wherein the system for detecting defects on the surface of an industrial part based on a deep learning anomaly detection algorithm comprises:
the model training module is used for acquiring a normal sample image of the data to be detected, inputting the normal sample image into a teacher-student anomaly detection algorithm, training and generating a model;
the anomaly detection model is used for acquiring an image of a part to be detected, inputting the image into an anomaly detection algorithm based on deep learning, and detecting and positioning an anomaly region; and detecting the obtained image by using the trained model, and if an abnormal region is found, reasoning the position and the abnormal value of the abnormal region on the surface of the part.
10. The system of claim 9, wherein the system further comprises:
the defect judgment module is used for jointly determining whether the area is a defect or not through the set area and the magnitude of the abnormal value according to the magnitude of the area and the magnitude of the abnormal value of the abnormal area calculated by the algorithm;
and the data analysis module is used for judging the abnormality of the part according to the size and the abnormal value of the abnormal region acquired from the model, and if the area and the abnormal value are larger than a set value, the part is considered to have the abnormal region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111185034.9A CN113902710A (en) | 2021-10-12 | 2021-10-12 | Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111185034.9A CN113902710A (en) | 2021-10-12 | 2021-10-12 | Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113902710A true CN113902710A (en) | 2022-01-07 |
Family
ID=79191465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111185034.9A Pending CN113902710A (en) | 2021-10-12 | 2021-10-12 | Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113902710A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115471645A (en) * | 2022-11-15 | 2022-12-13 | 南京信息工程大学 | Knowledge distillation anomaly detection method based on U-shaped student network |
| WO2024000372A1 (en) * | 2022-06-30 | 2024-01-04 | 宁德时代新能源科技股份有限公司 | Defect detection method and apparatus |
| CN117437221A (en) * | 2023-12-18 | 2024-01-23 | 浙江祥晋汽车零部件股份有限公司 | Method and system for detecting bright decorative strip based on image detection |
| CN117437221B (en) * | 2023-12-18 | 2024-04-26 | 浙江祥晋汽车零部件股份有限公司 | Method and system for detecting bright decorative strip based on image detection |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460991A (en) * | 2020-03-31 | 2020-07-28 | 科大讯飞股份有限公司 | Anomaly detection method, related device and readable storage medium |
| US20200364905A1 (en) * | 2019-05-16 | 2020-11-19 | Keyence Corporation | Image Inspection Apparatus |
| JP2021064215A (en) * | 2019-10-15 | 2021-04-22 | 日本製鉄株式会社 | Surface property inspection device and surface property inspection method |
| CN112801298A (en) * | 2021-01-20 | 2021-05-14 | 北京百度网讯科技有限公司 | Abnormal sample detection method, device, equipment and storage medium |
| CN112991330A (en) * | 2021-04-19 | 2021-06-18 | 征图新视(江苏)科技股份有限公司 | Knowledge distillation-based positive sample industrial defect detection method |
| WO2021191908A1 (en) * | 2020-03-25 | 2021-09-30 | Yissum Research Development Company Of The Hebrew University Of Jerusalem Ltd. | Deep learning-based anomaly detection in images |
-
2021
- 2021-10-12 CN CN202111185034.9A patent/CN113902710A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200364905A1 (en) * | 2019-05-16 | 2020-11-19 | Keyence Corporation | Image Inspection Apparatus |
| JP2021064215A (en) * | 2019-10-15 | 2021-04-22 | 日本製鉄株式会社 | Surface property inspection device and surface property inspection method |
| WO2021191908A1 (en) * | 2020-03-25 | 2021-09-30 | Yissum Research Development Company Of The Hebrew University Of Jerusalem Ltd. | Deep learning-based anomaly detection in images |
| CN111460991A (en) * | 2020-03-31 | 2020-07-28 | 科大讯飞股份有限公司 | Anomaly detection method, related device and readable storage medium |
| CN112801298A (en) * | 2021-01-20 | 2021-05-14 | 北京百度网讯科技有限公司 | Abnormal sample detection method, device, equipment and storage medium |
| CN112991330A (en) * | 2021-04-19 | 2021-06-18 | 征图新视(江苏)科技股份有限公司 | Knowledge distillation-based positive sample industrial defect detection method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024000372A1 (en) * | 2022-06-30 | 2024-01-04 | 宁德时代新能源科技股份有限公司 | Defect detection method and apparatus |
| CN115471645A (en) * | 2022-11-15 | 2022-12-13 | 南京信息工程大学 | Knowledge distillation anomaly detection method based on U-shaped student network |
| CN117437221A (en) * | 2023-12-18 | 2024-01-23 | 浙江祥晋汽车零部件股份有限公司 | Method and system for detecting bright decorative strip based on image detection |
| CN117437221B (en) * | 2023-12-18 | 2024-04-26 | 浙江祥晋汽车零部件股份有限公司 | Method and system for detecting bright decorative strip based on image detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105118044B (en) | A kind of wheel shape cast article defect automatic testing method | |
| CN108346144B (en) | Automatic bridge crack monitoring and identifying method based on computer vision | |
| CN110992349A (en) | Underground pipeline abnormity automatic positioning and identification method based on deep learning | |
| CN108885700A (en) | Data set semi-automatic labelling | |
| CN106875373A (en) | Mobile phone screen MURA defect inspection methods based on convolutional neural networks pruning algorithms | |
| CN113902710A (en) | Method and system for detecting surface defects of industrial parts based on anomaly detection algorithm | |
| CN109060817B (en) | Artificial intelligence reinspection system and method thereof | |
| KR20190075707A (en) | Method for sorting products using deep learning | |
| CN110648305B (en) | Industrial image detection method, system and computer readable recording medium | |
| CN109324604A (en) | A kind of intelligent train resultant fault analysis method based on source signal | |
| CN112508105B (en) | Fault detection and retrieval method for oil extraction machine | |
| CN113920107A (en) | Insulator damage detection method based on improved yolov5 algorithm | |
| CN109886947A (en) | The high-tension bus-bar defect inspection method of convolutional neural networks based on region | |
| CN111400040B (en) | Industrial Internet system based on deep learning and edge calculation and working method | |
| CN109001211A (en) | Welds seam for long distance pipeline detection system and method based on convolutional neural networks | |
| CN111783616B (en) | Nondestructive testing method based on data-driven self-learning | |
| CN111402249B (en) | Image evolution analysis method based on deep learning | |
| US20220245402A1 (en) | Ai-based pre-training model determination system, and ai-based vision inspection management system using same for product production lines | |
| CN115797354A (en) | Method for detecting appearance defects of laser welding seam | |
| CN115070780A (en) | Industrial robot grabbing method and device based on digital twinning and storage medium | |
| CN113592813B (en) | New energy battery welding defect detection method based on deep learning semantic segmentation | |
| CN111967308A (en) | Online road surface unevenness identification method and system | |
| CN115294009A (en) | Method and equipment for detecting welding defects of battery tabs based on machine learning and storage medium | |
| CN117115147A (en) | Textile detection method and system based on machine vision | |
| CN117406027A (en) | Distribution network fault distance measurement method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220107 |
|
| RJ01 | Rejection of invention patent application after publication |