CN111179223A - Deep learning-based industrial automatic defect detection method - Google Patents
Deep learning-based industrial automatic defect detection method Download PDFInfo
- Publication number
- CN111179223A CN111179223A CN201911276520.4A CN201911276520A CN111179223A CN 111179223 A CN111179223 A CN 111179223A CN 201911276520 A CN201911276520 A CN 201911276520A CN 111179223 A CN111179223 A CN 111179223A
- Authority
- CN
- China
- Prior art keywords
- detection
- model
- image
- defect
- steps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
An industrial automatic defect detection method based on deep learning. The invention relates to the technical field of product defect detection, aiming at avoiding complex feature extraction work, having higher accuracy and better generalization capability, under the conditions of smaller defect proportion and complex detection background, quickly training an applicable detection model and obtaining a detection method with better detection effect than the traditional detection method, processing a shot photo by utilizing a trained target detection model YOLO-V3 and judging whether a detected object exists in the photo; and identifying the shot photos by using a neural network topology inclusion-V3 image identification model obtained by small sample data migration learning deployed on the server, and judging whether the target positions contain defects. The invention is mainly applied to the detection occasions of industrial production lines.
Description
Technical Field
The invention relates to the technical field of product defect detection, in particular to the field of defect detection of an industrial automatic defect detection method and system based on deep learning.
Background
Due to the influence of various external factors such as vibration, temperature, pressure and the like, product defects inevitably occur in industrial automatic production. The defect detection refers to the detection of defects such as spots, dents, scratches, color differences, defects and the like on the surface of a product, and is one of the most important tasks in industrial quality control. With the development of economy and the improvement of industrial production efficiency, based on the traditional artificial defect detection method, the detection speed is low, the efficiency is low, errors are easy to occur in the detection process, and the high-efficiency requirement of modern industrial automatic production is difficult to meet.
The detection technology of machine vision based on artificial feature extraction makes up the defect of artificial defect detection to a certain extent, realizes automatic defect detection, and improves the industrial production efficiency. However, the detection method has the following disadvantages that the actual requirements of the existing industrial production are difficult to meet: the manual feature extraction is complex and can not cover all defect features; the feature extraction process requires very specialized domain knowledge; the detection accuracy and generalization capability are weak under complex detection background and multi-target detection scene; when the detected object changes, all rules and algorithms need to be redesigned and developed.
In recent two years, the artificial intelligence-deep learning technology has been rapidly developed and widely applied to a plurality of fields such as energy, traffic, medical treatment, meteorology and the like. However, deep learning techniques are also relatively less applicable in defect detection. The industrial automatic defect detection method based on deep learning avoids the problems of complex manual feature selection process, need of prior knowledge in related fields and the like in a general machine learning method, and has high detection accuracy and good generalization capability in complex environments and multi-target scenes. By applying the method provided by the invention, an applicable detection model can be trained rapidly under the condition of insufficient sample data, rapid hot deployment is supported, and a better detection effect is obtained compared with the traditional detection method. The defect detection system is a modularized system based on gRPC (Remote Procedure Call), MQTT (Message teletransmitting Message queue telemetry transmission) and Redis (Remote Dictionary service), has simple required hardware, low cost and convenient deployment, and is suitable for high-concurrency industrial application scenes.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an industrial automatic defect detection method and system based on deep learning. Under the condition that sample data is insufficient, an applicable detection model can be trained quickly, quick hot deployment is supported, and a better detection effect is achieved compared with the traditional detection method. The detection system is a modular structure system based on gRPC, MQTT and Redis, is simple to deploy, and can be applied to high-concurrency industrial application scenes. Therefore, the invention adopts the technical scheme that the industrial automatic defect detection method based on deep learning comprises the following steps:
(a) the method comprises the following steps Photographing industrial products on the production line;
(b) the method comprises the following steps Converting the picture into three-dimensional matrix data, and sending the three-dimensional matrix data to a corresponding image detection message theme of a Redis database, wherein the process is based on a Redis publishing/subscribing mode;
(c) the method comprises the following steps After a program subscribing a corresponding image detection theme monitors the message, a target detection model YOLO-V3 trained in advance after receiving the image is used for processing the shot image and judging whether a detected object exists in the image;
(d) the method comprises the following steps If yes, intercepting one or more detected targets from the picture, then sending the image matrix to a corresponding image identification message theme of a Redis database, and if no target is detected, returning to the step (c) of continuously monitoring whether a message exists on the corresponding image detection message theme of the Redis database;
(e) the method comprises the following steps When a program deployed on a server monitors that a message exists on a corresponding image identification message theme of a database, the program sequentially receives the target picture matrix data, then, a neural network topology inclusion-V3 image identification model obtained by small sample data migration learning deployed on the server is called, the target data are identified, and whether the target positions contain defects is judged;
(f) the method comprises the following steps If the Incep-V3 model identification result is that the defect is contained and the confidence coefficient exceeds a set threshold value, calling an MQTT client program, and sending alarm information to an MQTT corresponding alarm theme;
(g) the method comprises the following steps If the Incep-V3 model identification result is that the image contains no defect or contains a defect, but the confidence coefficient is lower than a set threshold value, the image is not processed, only the image is backed up to a server hard disk, and then the next target data is processed continuously;
(h) the method comprises the following steps After receiving the alarm message sent from the server on the subscribed message topic, the MQTT client in the industrial field triggers a corresponding alarm device or triggers execution of a corresponding control program, so as to control the automatic production equipment and correspondingly process the defective product.
The YOLO-V3 target detection model training is to obtain the prior frame size through the iterative solved clustering analysis algorithm K-means clustering analysis and replace the prior frame size in the original YOLO-V3 configuration, and when the YOLO-V3 target detection model is trained, the difficulty of fine adjustment of the prior frame to the actual position by a network can be reduced, so that the model can be rapidly converged.
And (3) detecting the data set of the inclusion-V3 through a YOLO-V3 model to obtain a new data set, and then training a defect classification model of the inclusion-V3 based on transfer learning. The training parameters are set as: the batch size is 100, the number of iterations is 800 steps, and the initial learning rate is 0.01.
The invention has the characteristics and beneficial effects that:
(1): based on deep learning, the problems of manual feature extraction work in a general machine learning detection method and high requirement on field professional knowledge are solved.
(2): the method combining the YOLO-V3 target detection model and the inclusion-V3 image classification model can improve the image detection precision and stability. Especially, the method has obvious advantages in the detection of multiple defect targets with small defect image ratio. And the method is not influenced by complex environment, and has high detection accuracy.
(3): the detection method can rapidly train the required defect detection model under the condition of a small sample set, rapidly deploy and obtain better detection accuracy.
(4): the defect detection system disclosed by the invention is modularized in deployment, simple in hardware, low in deployment cost and suitable for high-concurrency industrial automatic production scenes.
(5): the message subscription/publishing mechanism based on MQTT and Redis is convenient and efficient, and is suitable for the industrial environment with limited network bandwidth.
(6): and a TensorFlow Serving (Google open source service framework) service is adopted to deploy a defect identification model, and a function is deployed in hot state, so that the management model is convenient to update and iterate.
Description of the drawings:
FIG. 1 is a diagram: the invention relates to a structure diagram of a defect detection system;
FIG. 2 is a diagram of: a sample picture collected by an image collection end;
FIG. 3 is a diagram of: a picture obtained after data enhancement is performed on sample0 in fig. 2;
FIG. 4 is a diagram of: training an average loss curve of a YOLO-V3 target detection model;
FIG. 5 is a diagram: training the average IOU curve of the YOLO-V3 target detection model on the 13 x 13 scale
FIG. 6 is a diagram of: detecting the processed picture by using a YOLO-V3 target detection model;
FIG. 7 is a diagram of: the image which is processed by the YOLO-V3 target detection model and is obtained after interception processing is carried out on the image 6;
FIG. 8 is a diagram of: comparing the model accuracy of a defect detection method combining a YOLO-V3 target detection model and an inclusion-V3 defect identification model with the accuracy of a defect identification model only using the inclusion-V3;
FIG. 9 is a diagram of: comparing a model loss curve of a defect detection method by combining a YOLO-V3 target detection model and an inclusion-V3 defect identification model with a loss curve of a defect identification model only using the inclusion-V3;
FIG. 10 is a diagram: updating an iteration schematic diagram of a defect identification model in TensorFlow Serving;
FIG. 11 is a diagram of: hardware diagram of image acquisition side: raspberry pi + USB camera;
FIG. 12 is a diagram: after the defect detection system detects the defective picture, sending a corresponding picture message to an effect picture in the WeChat equipment management group;
FIG. 13 is a monitoring diagram of a terminal and a server side, wherein the first identification is two image acquisition terminals, the identifier ③ is a server history log, and the fourth identification is a message sending log sent by an MQTT client;
FIG. 14 is an example diagram of an actual application of an industrial field defect detection client.
Detailed Description
In order to overcome the defects of the prior art, the invention provides the deep learning-based industrial automatic defect detection method and the deep learning-based industrial automatic defect detection system, the method avoids complex feature extraction work, has higher accuracy and better generalization capability, and embodies advantages under the conditions of small defect occupation ratio and complex detection background. Under the condition that sample data is insufficient, an applicable defect detection model can be trained quickly, quick hot deployment is supported, and a better detection effect is achieved compared with the traditional detection method. The detection system is a modular structure system based on gRPC and MQTT, is simple to deploy, and can be applied to high-concurrency industrial application scenes.
The technical scheme adopted by the invention is as follows: the deep learning-based industrial automatic defect detection method and system comprise the following steps:
(i) the method comprises the following steps A raspberry pi + a USB camera is used as terminal image acquisition equipment, and a picture acquisition program deployed in the raspberry pi automatically finishes photographing industrial products on a production line (frame difference method).
(j) The method comprises the following steps The acquisition process converts the picture into three-dimensional matrix data and sends the three-dimensional matrix data to a theme of a corresponding image defect detection message of the Redis database (based on a publishing/subscribing mode);
(k) the method comprises the following steps After a program subscribing a corresponding image defect detection theme monitors a message, a picture is received and then is processed by a YOLO-V3 target detection model (a small sample set, which is trained in advance) (a target detection algorithm you only lookone version V3), and whether a detected object (none, one or a plurality of objects) exists in the picture is judged.
(l) The method comprises the following steps If so, intercepting one or more detected targets from the picture, and then sending the image matrix to a corresponding image defect identification message topic (based on a publish/subscribe mode) of the database Redis. If no target is detected, returning to (c) continuously monitoring whether a message exists on the subject of the corresponding image defect detection message of Redis.
(m): when a program deployed on a server monitors that a message exists on a corresponding image defect identification message theme of a database Redis, the target picture matrix data are sequentially received, then an inclusion-V3 (a neural network topological structure of GoogLeNet) image defect identification model (small sample data obtained by transfer learning) deployed in a TensorFlow Serving service on the server is called by a gPC to identify the target data, and whether the target positions contain defects is judged.
(n): if the Incep-V3 model identification result is that the defect is contained and the confidence coefficient exceeds the set threshold value, an MQTT client program is called, and alarm information is sent to a corresponding alarm subject of the MQTT server (based on a publishing/subscribing mode). Meanwhile, the service program calls a WeChat API interface to send the captured pictures to a WeChat equipment management group, so that a manager can conveniently check information. And the pictures are backed up to the hard disk of the server, so that the later-stage research and the checking and use are facilitated.
(o): if the Incep-V3 model identification result is that no defect exists or the defect exists, but the confidence coefficient is lower than the set threshold value, no processing is performed, only pictures are backed up to a server hard disk, later-stage research and use are facilitated, and then the next target data is processed continuously.
(p): after receiving the alarm message sent by the MQTT server from the subscribed message topic, the industrial field MQTT client triggers a corresponding alarm device or triggers a PLC to execute a corresponding control program, so as to control the automatic production equipment and correspondingly process the defective products.
The invention is further described below with reference to the accompanying drawings.
System structure
FIG. 1 shows a system structure diagram of an industrial automatic defect detection method based on deep learning, which mainly comprises 5 parts of an image data acquisition service, an image classification service, a system alarm service, a data forwarding service, a data storage service and the like, and the defect detection work is completed together.
Image data acquisition service: and calling a USB camera by an acquisition program deployed on the raspberry Pi 3B to acquire an image, then identifying suspicious defect parts in the image by processing through a YOLO-V3 model, and finally intercepting the identified defect parts from the image and sending the intercepted defect parts to an image classification server.
Image classification service: the method comprises the steps of taking an inclusion-V3 model deployed on a server as an image identification service, detecting suspicious defect screenshots sent by an image data acquisition service, outputting a defect detection result, and determining whether to trigger an alarm system and whether to send an equipment control state instruction according to the detection result.
And (3) system alarm service: according to the control state instruction issued by the image classification service, a control unit PLC (programmable logic controller) of the related equipment is triggered to execute a corresponding program, the output state of the equipment is controlled, and whether an audible and visual alarm device is triggered or not is determined according to setting. Meanwhile, the service also pushes alarm information to a WeChat equipment management group through a WeChat API.
Data forwarding service: data forwarding mainly adopts two modes, one mode is that the image data forwarding process is based on a message publishing/subscribing mode of a database Redis. And secondly, the state control information of the PLC device and the alarm device adopts a message publishing/subscribing mechanism based on a lightweight MQTT protocol.
Data storage service: the image data and the alarm control data generated in the whole defect detection service are recorded and stored, on one hand, the image data is required for meeting the follow-up updating and upgrading training of the model, on the other hand, the key parameter data of the equipment is recorded in a historical mode, and the follow-up data analysis, research and use of the equipment are facilitated.
(II) data set creation
(a) Data acquisition
The program deployed in raspberry pi 3B uses a frame difference method to acquire picture data through a USB camera, the acquisition size of the model camera is 500 × 375, and the color of the wire coil is 6 types, namely red, green, blue, yellow, black and white. The package has transparent film and white plastic tape. Several representative pictures are shown in fig. 2, wherein sample0 is a case of simultaneously beating a plurality of wire trays, sample 3 is a case of beating a wire tray at only one corner, sample 4 is a case of coexistence of no tape package (package defect) and tape package (package pass), and sample 5 is a case of package defect with a package gap
(b) Data enhancement
The Incep-V3 model requires a large number of training samples for training, but in actual production, the acquisition of samples, particularly coil samples with defects, is relatively difficult in a short period, so that the capacity of the sample data is increased through image preprocessing. The data enhancement mainly comprises the following steps: rotation, translation, scaling, edge filling, brightness adjustment, saturation adjustment, clipping, blurring, graying, and the like. Fig. 3 includes horizontal flipping, vertical flipping, 20% brightness increase, 60% saturation decrease, 30% gray reduction, and 60% center clipping for sample0 in fig. 2. Through data enhancement, 1 picture is changed into 7 pictures, and the sample capacity is greatly increased.
(c) Data set composition
The collected raw data is classified into 3 types: with wrapping (packaging is acceptable), without wrapping or with wrapping gaps (packaging defects), and others. Where 'other' means a view similar to sample 3 in figure 2 without target coils or a view similar to sample 4 with and without both wrapped and unwrapped coils. According to the actual requirement, the defects do not need to be classified, and the model can be simplified to divide the samples into two types: qualified package and package defects. Wherein the data set of inclusion-V3 is: training set [1280, 1266 ], testing set [100, 100 ]. It should be noted that: because the qualified samples of the package are easy to obtain, the qualified samples in the training set and the testing set are original sample pictures, the defect samples of the package are relatively difficult to obtain, the defect samples in the training set and the testing set are 1400 pictures obtained by enhancing the data of original 200 defect pictures, 1366 enhanced pictures are obtained by removing some pictures which are displayed on a cut wireless disc or wire coil, and 10 percent of the training set is used as a verification set during training; the data set of the YOLO-V3 is characterized in that 120 pictures are randomly extracted from the data set of the inclusion-V3, the positions of wire coils are manually marked by using a LabelImg tool to generate an xml file in a VOC format, and then the script is used for producing a Txt file in the YOLO format.
(III) model training
(a) Yolo-V3 target detection model training
To accommodate the data set, the prior box size is obtained by K-means cluster analysis (an iterative clustering algorithm), and replaces the prior box size in the original YOLO-V3 configuration. The difficulty of fine-tuning the prior frame to the actual position of the network can be reduced when the YOLO-V3 prediction model is trained, so that the model can be converged quickly. The experimental data cluster k is 9, and the cluster points are (109,146), (157,73), (180,160), (204,218), (242,117), (254,170), (285,240), (292,215), (440,295). It can be seen that the size of the target on the data set is almost all above 100 × 100, and the prior box sizes (116,90), (156,192), (373,326) on the 13 × 13 scale, which are similar to the default parameters of the original configuration, belong to the large target. This is consistent with the situation that in practical experimental tests, the target can be detected only in the 13 × 13 scale when the original parameters are used, and the target is not detected in other scales. And the training effect of the model on multiple scales is better than that of multiple sizes obtained by clustering. This is because the data set used is smaller, and the sizes of the wire coil targets belong to large targets, and compared with the characteristic maps with the dimensions of 26 × 26, 52 × 52 and 13 × 13, the resolution information is less, the semantic information is large, and the targets and the background can be better distinguished. Therefore, the experiment adopts the original model parameters, and can meet the detection requirement.
The experiment machine is a ubuntu 16.04 system memory 16G, an Intel i7-4980HQ 2.8GHZ processor and an NVIDIAGeForce GTX 980M 4G display card, the batch size is set to be 16, the iteration number is 5000, the learning rate is 0.001 in 0-2000 rounds, 0.0001 in 2000-3000 rounds and 0.00001 in 3000-5000 rounds by using a step-by-step strategy. The results of training on the data set are shown in fig. 4 and 5, which are loss variation curves of model training and IOU (intersection ratio between the prediction box and the real box) curve variation on a 13 × 13 scale, respectively. The average loss curve has a large loss at the beginning of iteration, and the loss is rapidly reduced along with the increase of the number of iterations, and is basically in a stable state at about 2000 iterations, and the loss fluctuates about 0.0045. The IOU curve is also stable after 2000 iterations, fluctuating around 0.88.
(b) Inception-V3 defect recognition model training
Firstly, processing an inclusion-V3 data set by a trained YOLO-V3 model, and identifying the sample in the graph 2, wherein the result is shown in FIG. 6, and the wire coils in the graph are accurately identified and marked by yellow rectangular boxes. (detection 3 was not identified by the model because it had only a small corner) and then these identified coils were cut out of the picture to obtain the data shown in figure 7 (the lowest coil in detection 4 in figure 6 was filtered out of the program because the rectangular box was too small to reach the threshold 80 x 80).
Processing the data set of the inclusion-V3 through a YOLO-V3 model to obtain a new data set, and then training a defect classification model of the inclusion-V3 based on transfer learning. The training parameters are set as: the batch size is 100, the number of iteration steps is 800, and the learning rate is 0.01. FIGS. 8 and 9 show the accuracy curves and cross-entropy loss curves of the inclusion-V3 classification models obtained with and without the YOLO-V3 and the YOLO-V3 on the training set. It can be seen from the figure that the accuracy of the inclusion-V3 classification model (red curve) obtained by training after the YOLO-V3 recognition treatment is stabilized at 99.49% after 400 steps, the variance is 0.0000506, compared with the accuracy of the model obtained without treatment, the variance is 97.70%, the variance is 0.000251, the fluctuation amplitude of the accuracy curve and the loss curve is small, and the whole is more stable.
(IV) model deployment
(a) TensorFlow Serving service
The method includes the steps that TensorFlow Serving service is built on a server ubutu 16.04 of a machine room through a docker container technology, a trained inclusion-V3 defect recognition model is deployed on the server, a plurality of remote clients can access the model service through gRPC or RESTful API, meanwhile, the TensorFlow Serving supports online hot deployment, and later-stage updating of an iterative model is facilitated, the structure of the TensorFlow Serving service is shown in FIG. 10, model 1 to model 3 represent model iterative updating, and model 3 represents a model which is being served at present. The trained model is remotely called in a gRPC mode, and the detection result of the graph and the confidence coefficient of the result are obtained. And determining whether to call the MQTT client to send state messages such as alarm, control and the like to the MQTT server or not according to a threshold value designed by the service program.
(b) Acquisition end hardware and information receiving end
the left side of the image acquisition module in the figure 11 is provided with a USB camera, the right side of the image acquisition module in the raspberry pi 3B is provided with image acquisition equipment of the invention, the image acquisition equipment is deployed in an industrial production field and is responsible for acquiring and acquiring product images in the production process, the actual deployment situation in the industrial field is shown in the figure 14, the USB camera is arranged in the figure firstly, the raspberry pi machine body is arranged in the figure secondly, and the whole acquisition module is connected to a system network through workshop WIFI.
Fig. 12 shows an effect that when the detection result of the defect detection system is an unqualified product, the service program calls the wechat API interface to push the captured picture to the wechat device management group. It can be seen that the detection results are unpacked and give confidence levels of 0.98 and 0.97, as well as the time at which the defect picture was detected.
in fig. 13, ① is, a camera monitoring picture at an image data acquisition end and corresponding log information are provided, secondly, a log at a server side is provided, which records an identification result of an identification model and whether the log is pushed to a WeChat client, and ④ is, a message log at an MQTT client is provided, which records alarm log information.
Claims (3)
1. An industrial automatic defect detection method based on deep learning is characterized by comprising the following steps:
(a) the method comprises the following steps Photographing industrial products on the production line;
(b) the method comprises the following steps Converting the picture into three-dimensional matrix data, and sending the three-dimensional matrix data to a corresponding image detection message theme of a Redis database, wherein the process is based on a Redis publishing/subscribing mode;
(c) the method comprises the following steps After a program subscribing a corresponding image detection theme monitors the message, a target detection model YOLO-V3 trained in advance after receiving the image is used for processing the shot image and judging whether a detected object exists in the image;
(d) the method comprises the following steps If yes, intercepting one or more detected targets from the picture, then sending the image matrix to a corresponding image identification message theme of a Redis database, and if no target is detected, returning to the step (c) of continuously monitoring whether a message exists on the corresponding image detection message theme of the Redis database;
(e) the method comprises the following steps When a program deployed on a server monitors that a message exists on a corresponding image identification message theme of a database, the program sequentially receives the target picture matrix data, then, a neural network topology inclusion-V3 image identification model obtained by small sample data migration learning deployed on the server is called, the target data are identified, and whether the target positions contain defects is judged;
(f) the method comprises the following steps If the Incep-V3 model identification result is that the defect is contained and the confidence coefficient exceeds a set threshold value, calling an MQTT client program, and sending alarm information to an MQTT corresponding alarm theme;
(g) the method comprises the following steps If the Incep-V3 model identification result is that the image contains no defect or contains a defect, but the confidence coefficient is lower than a set threshold value, the image is not processed, only the image is backed up to a server hard disk, and then the next target data is processed continuously;
(h) the method comprises the following steps After receiving the alarm message sent from the server on the subscribed message topic, the MQTT client in the industrial field triggers a corresponding alarm device or triggers execution of a corresponding control program, so as to control the automatic production equipment and correspondingly process the defective product.
2. The deep learning-based industrial automatic defect detection method as claimed in claim 1, wherein the YOLO-V3 target detection model is trained by obtaining the prior frame size through a clustering analysis algorithm K-means clustering analysis of iterative solution and replacing the prior frame size in the original YOLO-V3 configuration, so that the difficulty of network fine-tuning the prior frame to the actual position can be reduced when the YOLO-V3 target detection model is trained, and the model can be rapidly converged.
3. The deep learning-based industrial automation defect detection method as claimed in claim 1, wherein the inclusion-V3 data set is subjected to a YOLO-V3 model detection process to obtain a new data set, and then a defect classification model of the inclusion-V3 based on transfer learning is trained. The training parameters are set as: the batch size is 100, the number of iterations is 800 steps, and the initial learning rate is 0.01.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911276520.4A CN111179223B (en) | 2019-12-12 | 2019-12-12 | Industrial automation defect detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911276520.4A CN111179223B (en) | 2019-12-12 | 2019-12-12 | Industrial automation defect detection method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111179223A true CN111179223A (en) | 2020-05-19 |
| CN111179223B CN111179223B (en) | 2023-05-09 |
Family
ID=70653877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911276520.4A Active CN111179223B (en) | 2019-12-12 | 2019-12-12 | Industrial automation defect detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111179223B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111445471A (en) * | 2020-06-16 | 2020-07-24 | 杭州百子尖科技股份有限公司 | Product surface defect detection method and device based on deep learning and machine vision |
| CN111612784A (en) * | 2020-06-01 | 2020-09-01 | 南通大学 | Steel plate surface defect detection method based on classification-first YOLO network |
| CN111881937A (en) * | 2020-06-22 | 2020-11-03 | 深圳金三立视频科技股份有限公司 | Transmission line hardware target detection and defect identification method and terminal |
| CN112017172A (en) * | 2020-08-31 | 2020-12-01 | 佛山科学技术学院 | System and method for detecting defects of deep learning product based on raspberry group |
| CN112435245A (en) * | 2020-11-27 | 2021-03-02 | 济宁鲁科检测器材有限公司 | Magnetic mark defect automatic identification method based on Internet of things |
| CN112581469A (en) * | 2020-12-31 | 2021-03-30 | 上汽通用五菱汽车股份有限公司 | Robot vision flexibility detection system and method |
| CN112700436A (en) * | 2021-01-13 | 2021-04-23 | 上海微亿智造科技有限公司 | Method, system and medium for improving iteration of industrial quality inspection model |
| CN113297910A (en) * | 2021-04-25 | 2021-08-24 | 云南电网有限责任公司信息中心 | Distribution network field operation safety belt identification method |
| CN113343891A (en) * | 2021-06-24 | 2021-09-03 | 深圳市起点人工智能科技有限公司 | Detection device and detection method for child kicking quilt |
| CN113569737A (en) * | 2021-07-28 | 2021-10-29 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Notebook screen defect detection method and medium based on autonomous learning network model |
| CN113743203A (en) * | 2021-07-28 | 2021-12-03 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Notebook screen defect detection method and equipment based on deep migration learning network |
| CN113838015A (en) * | 2021-09-15 | 2021-12-24 | 上海电器科学研究所（集团）有限公司 | Electric appliance product appearance defect detection method based on network cooperation |
| CN113916899A (en) * | 2021-10-11 | 2022-01-11 | 四川科伦药业股份有限公司 | Method, system and device for detecting large soft infusion bag product based on visual identification |
| CN114994046A (en) * | 2022-04-19 | 2022-09-02 | 深圳格芯集成电路装备有限公司 | Defect detection system based on deep learning model |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016201947A1 (en) * | 2015-06-16 | 2016-12-22 | 华南理工大学 | Method for automated detection of defects in cast wheel products |
| CN106546846A (en) * | 2016-10-18 | 2017-03-29 | 天津大学 | Electric energy quality signal detection means based on compressed sensing blind Source Separation Technology |
| CN109003271A (en) * | 2018-07-25 | 2018-12-14 | 江苏拙术智能制造有限公司 | A kind of Wiring harness connector winding displacement quality determining method based on deep learning YOLO algorithm |
| US20180370027A1 (en) * | 2017-06-27 | 2018-12-27 | Fanuc Corporation | Machine learning device, robot control system, and machine learning method |
| CN109978835A (en) * | 2019-03-04 | 2019-07-05 | 广东石油化工学院 | A kind of online assembly defect recognition system and method |
-
2019
- 2019-12-12 CN CN201911276520.4A patent/CN111179223B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016201947A1 (en) * | 2015-06-16 | 2016-12-22 | 华南理工大学 | Method for automated detection of defects in cast wheel products |
| CN106546846A (en) * | 2016-10-18 | 2017-03-29 | 天津大学 | Electric energy quality signal detection means based on compressed sensing blind Source Separation Technology |
| US20180370027A1 (en) * | 2017-06-27 | 2018-12-27 | Fanuc Corporation | Machine learning device, robot control system, and machine learning method |
| CN109003271A (en) * | 2018-07-25 | 2018-12-14 | 江苏拙术智能制造有限公司 | A kind of Wiring harness connector winding displacement quality determining method based on deep learning YOLO algorithm |
| CN109978835A (en) * | 2019-03-04 | 2019-07-05 | 广东石油化工学院 | A kind of online assembly defect recognition system and method |
Non-Patent Citations (3)
| Title |
|---|
| ACROS-GARCLA, A等: "Evaluation of deep nueral networks for traffic sign detetion systems" * |
| 徐巍等: "快速独立分量分析方法在感应电动机故障检测中的应用" * |
| 李庆忠等: "基于改进YOLO和迁移学习的水下鱼类目标实时检测" * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111612784A (en) * | 2020-06-01 | 2020-09-01 | 南通大学 | Steel plate surface defect detection method based on classification-first YOLO network |
| CN111612784B (en) * | 2020-06-01 | 2023-11-14 | 南通大学 | Steel plate surface defect detection method based on classification priority YOLO network |
| CN111445471A (en) * | 2020-06-16 | 2020-07-24 | 杭州百子尖科技股份有限公司 | Product surface defect detection method and device based on deep learning and machine vision |
| CN111881937A (en) * | 2020-06-22 | 2020-11-03 | 深圳金三立视频科技股份有限公司 | Transmission line hardware target detection and defect identification method and terminal |
| CN112017172A (en) * | 2020-08-31 | 2020-12-01 | 佛山科学技术学院 | System and method for detecting defects of deep learning product based on raspberry group |
| CN112435245A (en) * | 2020-11-27 | 2021-03-02 | 济宁鲁科检测器材有限公司 | Magnetic mark defect automatic identification method based on Internet of things |
| CN112581469A (en) * | 2020-12-31 | 2021-03-30 | 上汽通用五菱汽车股份有限公司 | Robot vision flexibility detection system and method |
| CN112700436A (en) * | 2021-01-13 | 2021-04-23 | 上海微亿智造科技有限公司 | Method, system and medium for improving iteration of industrial quality inspection model |
| CN113297910B (en) * | 2021-04-25 | 2023-04-18 | 云南电网有限责任公司信息中心 | Distribution network field operation safety belt identification method |
| CN113297910A (en) * | 2021-04-25 | 2021-08-24 | 云南电网有限责任公司信息中心 | Distribution network field operation safety belt identification method |
| CN113343891A (en) * | 2021-06-24 | 2021-09-03 | 深圳市起点人工智能科技有限公司 | Detection device and detection method for child kicking quilt |
| CN113743203A (en) * | 2021-07-28 | 2021-12-03 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Notebook screen defect detection method and equipment based on deep migration learning network |
| CN113569737A (en) * | 2021-07-28 | 2021-10-29 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Notebook screen defect detection method and medium based on autonomous learning network model |
| CN113838015A (en) * | 2021-09-15 | 2021-12-24 | 上海电器科学研究所（集团）有限公司 | Electric appliance product appearance defect detection method based on network cooperation |
| CN113838015B (en) * | 2021-09-15 | 2023-09-22 | 上海电器科学研究所（集团）有限公司 | Electrical product appearance defect detection method based on network cooperation |
| CN113916899A (en) * | 2021-10-11 | 2022-01-11 | 四川科伦药业股份有限公司 | Method, system and device for detecting large soft infusion bag product based on visual identification |
| CN113916899B (en) * | 2021-10-11 | 2024-04-19 | 四川科伦药业股份有限公司 | Method, system and device for detecting large transfusion soft bag product based on visual identification |
| CN114994046A (en) * | 2022-04-19 | 2022-09-02 | 深圳格芯集成电路装备有限公司 | Defect detection system based on deep learning model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111179223B (en) | 2023-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111179223B (en) | Industrial automation defect detection method based on deep learning | |
| CN111179249A (en) | Power equipment detection method and device based on deep convolutional neural network | |
| CN113436184B (en) | Power equipment image defect discriminating method and system based on improved twin network | |
| CN116188475B (en) | Intelligent control method, system and medium for automatic optical detection of appearance defects | |
| CN110705847A (en) | Intelligent substation inspection method and system based on image recognition technology | |
| CN115861210B (en) | Transformer substation equipment abnormality detection method and system based on twin network | |
| CN112974303B (en) | Hyperspectrum-based fruit quality detection method, device and medium | |
| CN113723157B (en) | Crop disease identification method and device, electronic equipment and storage medium | |
| CN114697212A (en) | Device parameter processing method, device, system and medium | |
| CN111401169A (en) | Power supply business hall service personnel behavior identification method based on monitoring video information | |
| CN110309843A (en) | The automatic identifying method of multiclass component in a kind of power equipment image | |
| CN114511820A (en) | Goods shelf commodity detection method and device, computer equipment and storage medium | |
| CN111507249A (en) | Transformer substation nest identification method based on target detection | |
| CN113439227A (en) | Capturing and storing magnified images | |
| Xiao et al. | Fast recognition method for citrus under complex environments based on improved YOLOv3 | |
| CN108960849A (en) | For the method for anti-counterfeit and device of paper products, anti-fake traceability system | |
| CN117351472A (en) | Tobacco leaf information detection method and device and electronic equipment | |
| CN107907546A (en) | A kind of defects of vision detection method and system based on intelligent mobile terminal | |
| CN110458231B (en) | Ceramic product detection method, device and equipment | |
| CN116778357A (en) | Power line unmanned aerial vehicle inspection method and system utilizing visible light defect identification | |
| CN115937492A (en) | Transformer equipment infrared image identification method based on feature identification | |
| CN115526859A (en) | Method for identifying production defects, distributed processing platform, equipment and storage medium | |
| CN114283319A (en) | Locomotive wheel set tread stripping identification method | |
| CN114241190A (en) | Intelligent control system and method for chip production | |
| CN112434851A (en) | Big data based egg and poultry sales management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |