CN112788110A - Product appearance detection method based on cloud edge collaborative model optimization and implementation system thereof - Google Patents
Product appearance detection method based on cloud edge collaborative model optimization and implementation system thereof Download PDFInfo
- Publication number
- CN112788110A CN112788110A CN202011602970.0A CN202011602970A CN112788110A CN 112788110 A CN112788110 A CN 112788110A CN 202011602970 A CN202011602970 A CN 202011602970A CN 112788110 A CN112788110 A CN 112788110A
- Authority
- CN
- China
- Prior art keywords
- yolov3
- model
- detection
- cloud
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 113
- 238000005457 optimization Methods 0.000 title claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 20
- 230000006854 communication Effects 0.000 claims description 23
- 238000004891 communication Methods 0.000 claims description 23
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 21
- 238000002372 labelling Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 230000007175 bidirectional communication Effects 0.000 claims description 3
- 238000000034 method Methods 0.000 description 17
- 238000004519 manufacturing process Methods 0.000 description 15
- 238000003860 storage Methods 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000004880 explosion Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 240000007651 Rubus glaucus Species 0.000 description 1
- 235000011034 Rubus glaucus Nutrition 0.000 description 1
- 235000009122 Rubus idaeus Nutrition 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Abstract
The invention relates to a product appearance detection method based on cloud edge collaborative model optimization and an implementation system thereof, wherein the detection method comprises the following steps: s1, establishing a basic data set; s2, training a YOLOv3-tiny model and a YOLOv3 model, and respectively deploying the models on an edge server and a cloud platform; s3 and a YOLOv3-tiny model detect and identify the picture, and when the picture is detected to be qualified, the picture and a detection result are sent to a cloud platform to carry out S5; when the detection is unqualified, the detection is judged to be suspected unqualified, and the detection is sent to a cloud platform for S4; s4, carrying out secondary detection on the suspected unqualified picture on the cloud platform; when the secondary detection is unqualified, outputting a result, and ending; if the secondary detection is qualified, performing S5; and S5, storing the qualified pictures in the cloud platform, outputting the result and ending. The invention solves the problems of accuracy, flexibility, time delay and data utilization rate by using the working mode of cloud edge cooperation.
Description
Technical Field
The invention relates to a product appearance detection method based on cloud edge collaborative model optimization and an implementation system thereof, and belongs to the technical field of edge computing architecture and artificial intelligence.
Background
The detection rate of industrial production lines directly relates to the production efficiency, most of the existing production lines use the identification method of the air conditioner outdoor unit with a fixed model, the identification success rate is not high, the model is inconvenient to update and deploy, and the collection and the processing of a large amount of industrial data have problems.
The working mode of the existing production lines of most factories does not relate to a cloud edge architecture, and the detection precision, the problem analysis and the overall optimization are still to be improved. The edge calculation has high requirements on scene individuation, and the reasonability and the practicability of the edge calculation need to be comprehensively considered. Therefore, the cloud-edge cooperative working mode is established by matching with the edge equipment, data can be effectively utilized, the cloud platform pressure is reduced, and the time delay is reduced, so that the production quality and the production efficiency are improved.
Discretely manufactured products are finally assembled from a plurality of parts through a series of discrete processes. An enterprise that processes such products may be referred to as a discrete manufacturing enterprise. For example, the intelligent detection of the outdoor unit of the air conditioner belongs to a key ring in discrete manufacturing.
In the prior appearance detection of the discrete manufacturing industry, template matching and manual detection methods are mostly adopted, and the problems of poor precision, slow detection and low efficiency exist respectively. Meanwhile, deep learning based on big data has very few applications in the field, the mining degree of data value is not enough, and the utilization rate of industrial data is also in urgent need of improvement. Under the background of big data, AI and intelligent manufacturing, adopt cloud limit cooperative detection framework, dispose the marginal intelligence based on deep learning and carry out discrete manufacturing intelligence outward appearance and detect, can reduce the human cost, effectively improve system detection precision, efficiency and stability, have the significance to the intelligent transformation of discrete manufacturing.
The existing industrial detection system adopts a relatively complex network mechanism, but has relatively low calculation speed, does not relate to multi-target identification detection in a single picture, and does not relate to data set and model closed-loop optimization.
The 5G communication technology is emerging and is also a necessity for future development. By adopting 5G communication, the problem of complex industrial field wiring can be reduced on the premise of meeting the time requirement, the flexibility of industrial equipment deployment is effectively improved, and the industrial transformation trend of a 5G + industrial Internet is better met.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a product appearance detection method based on cloud-edge collaborative model optimization, which starts from the aspects of optimizing the architecture, reducing the pressure of a cloud platform and improving the data utilization rate, and realizes a cloud-edge division system for carrying out large-scale computation and overall analysis and small-scale computation and data preprocessing at the edge side by using a cloud-edge collaborative working mode, thereby solving the problems in the aspects of accuracy, flexibility, time delay and data utilization rate.
The invention also provides an implementation system of the product appearance detection method based on cloud edge collaborative model optimization.
The technical scheme of the invention is as follows:
a product appearance detection method based on cloud edge collaborative model optimization comprises the following steps:
s1, collecting whether the picture of the product appearance is known to be qualified or not, and establishing a basic data set after marking the picture of the product appearance;
s2, respectively training a YOLOv3-tiny model and a YOLOv3 model by using a basic data set, deploying the trained YOLOv3-tiny model at an edge server end, and deploying the trained YOLOv3 model at a cloud platform;
s3, at the edge server end, detecting and identifying the picture of the product appearance needing to be detected by using the trained YOLOv3-tiny model;
when the detection confidence coefficient is greater than or equal to the set confidence coefficient, judging that the product is qualified, sending the picture of the product outdoor unit and the detection result to the cloud platform, and then performing step S5;
when the detection confidence is smaller than the set confidence, determining that the product is suspected to be unqualified, independently sending the picture of the product outdoor unit to a cloud platform, and then performing step S4;
s4, carrying out secondary detection and identification on the picture of the suspected unqualified product outdoor unit by using a trained YOLOv3 model on a cloud platform;
when the confidence coefficient of the secondary detection is smaller than the set confidence coefficient, the detection result is unqualified, the detection result is output, and the detection is finished;
when the confidence of the secondary detection is greater than or equal to the set confidence and the detection result is qualified, performing step S5;
s5, storing the qualified picture of the product outdoor unit in a cloud platform, outputting a detection result, and finishing detection; and after the pictures of the product external machine are marked, updating the basic data set.
Cloud edge collaboration refers to multi-directional interactive collaboration between a cloud center and network edge devices. Specifically, the central cloud is obtained by virtualizing a large number of high-performance servers, and has the capability of monitoring performance data and working conditions of the edge servers while performing large-scale high-speed computation, so as to interactively achieve resource coordination and work coordination. The cloud edge collaboration in the invention comprises computing resource collaboration, storage resource collaboration and application management collaboration.
The computing resource collaboration is embodied as: according to the difference of cloud and edge computing resources, deployment of different models and allocation of tasks with different computing quantities are carried out, so that the computing resources are fully utilized;
the storage resource is characterized in that: the edge side only needs to store a small amount of temporary data. A large amount of data is sent to the cloud for storage, and the large-capacity storage resources of the cloud are reasonably utilized;
the application management collaboration is embodied as: the edge server provides a deployment environment and an operation environment of the model, and the cloud is responsible for deploying and updating the model at the edge node and monitoring the operation condition.
Preferably, the method further comprises step S6: and using the basic data set obtained by updating in the step S5, periodically performing update training on the YOLOv3-tiny model and the YOLOv3 model, deploying the updated YOLOv3-tiny model to the edge device, and deploying the updated YOLOv3 model to the cloud.
The recognition accuracy can be improved by periodically updating and training the YOLOv3-tiny model and the YOLOv3 model, so that the self-optimization of the models is realized. In the setting of period, after the data set is updated last time, newly added data setThe number of pictures is n, and the update period is set to be n0. Updating the value of n when a new picture is added to the data set, and when n is equal to n0And the cloud platform starts to update and train the model, and meanwhile, n is reset to be 0, and the next period is waited.
Preferably, in step S3, the external product unit on the product pipeline is positioned and detected in real time by using the industrial camera to take a snapshot. Real-time video monitoring is not needed, so that the equipment cost is saved while the identification capability is ensured.
Preferably, in step S1, the labellimg image labeling tool is used to label the picture of the product external machine, the frame is used to label the part to be detected,
after the file output by the labelimg image labeling tool is processed by Python programming, a file for YOLOv3 or YOLOv3-tiny training is generated, and therefore the construction of an initial effective data set is completed.
Preferably, in step S2, the YOLOv3 model includes 52 convolutional layers, where the 52 convolutional layers include 1 convolutional layer connected to the input, 5 convolutional layers for down-sampling, and 23 residual blocks, each residual block includes two convolutional layers, and a shortcut is set between the two convolutional layers for cross-layer connection;
the YOLOv3 model outputs three output sizes of 52 × 52, 26 × 26 and 13 × 13, and the three output sizes are respectively led out from three residual blocks; the output result comprises a target identification box, a target name label and a detection confidence coefficient.
The YOLO method belongs to a one-stage target detection algorithm, is different from a two-stage method, cannot generate candidate regions (regions), and therefore the identification speed is higher; meanwhile, in the area selection stage of YOLO, the whole picture is input, so that the detection of the image is more accurate and wrong background information is not easy to generate. However, YOLO has a disadvantage in that a 7 × 7 grid is used when a picture is divided, and thus the positioning accuracy of an object may not be sufficient. Compared with a YOLO V1 method, the YOLO V2 method improves precision and detection speed, the YOLOv3 is greatly optimized on the basis of the YOLO V2, a network structure is adjusted, a Darknet-53 network structure is adopted, a multi-scale fusion method is adopted, the recognition capability of small targets is improved, the precision and the speed are improved, the YOLOv3 method is combined with double verification of a cloud edge framework, the precision and the usability of a new data set are improved, and convenience is provided for periodic updating of a model.
The YOLOv3 model is based on a Darknet-53 network structure, and the Darknet-53 network has 52 convolutional layers and 1 full connection layer; the Darknet-53 network introduces shortcut cross-layer connection between convolution layers, so that the network structure is divided into a plurality of residual block residual blocks; the output of the residual block is the superposition of the input of the residual block and the result obtained after the input is subjected to the operation of several convolution layers in the residual block, and the purpose is to reduce the risk of gradient explosion. YOLOv3 is a full convolutional layer structure, adopts the front 52-layer structure of Darknet-53, does not use the last full connection layer of the Darknet-53 network, abandons the pooling layer pooling and adopts the convolutional layer for down-sampling.
According to different detection receptive fields, YOLOv3 can extract features from feature scales with different sizes, and has three output scales of 52 × 52, 26 × 26 and 13 × 13; the feature maps of the three scales can be independently predicted, or can be subjected to upsampling by a convolutional layer to be identical in size and then further predicted after being spliced by a tensor splicing layer (Concat).
Preferably, in step S2, the YOLOv3-tiny model includes 10 convolutional layers and 8 maxpool max pooling layers, and the output size is 13 × 13 × 255; meanwhile, the 5 th convolutional layer and the 8 th convolutional layer of the yollov 3-tiny model respectively output intermediate values of 26 × 26 × 256 and 13 × 13 × 256, the 13 × 13 × 256 outputs a final result through one convolutional layer, the upsampling layer is spliced with the 26 × 26 × 256 tensor, and the two convolutional layers, and the final output size is 26 × 26 × 255.
The model of YOLOv3-tiny is much more simplified than YOLOv3, and no structure of residual block is used. The structure of YOLOv3-tiny is relatively lightweight and can be used for relatively simple tasks.
Preferably, in step S3, the picture of the product appearance is compressed at the edge server, and then uploaded from the edge server to the cloud platform. The data transmission quantity can be reduced, and the cost is saved.
According to the invention, the edge server and the cloud platform are preferably communicated by adopting a socket network socket, the edge server is used as a client, and the cloud platform is used as a server to carry out bidirectional communication.
And in the aspect of communication between the edge device and the cloud platform, a stable and feasible socket network socket is adopted for communication. socket communication firstly needs to establish sockets at a client and a server to obtain a host name and a port number; the server side is used for monitoring the request, after the client side sends the communication request, the client side and the server side establish connection through three-way handshake, and then data transmission is completed through encoding sending and decoding receiving; after the transmission is finished, the client and the server are disconnected by waving hands for four times.
The system for realizing the product appearance detection method based on the cloud edge collaborative model optimization comprises a cloud platform, an edge server and an edge sensing device,
the edge sensing equipment is connected with the edge server and used for acquiring a picture of the air conditioner external unit to be detected and transmitting the picture of the air conditioner external unit to the edge server;
the edge server is in communication connection with the cloud platform, carries out first detection on the picture of the air conditioner outdoor unit acquired by the edge sensing equipment by using a trained YOLOv3-tiny model, and uploads the picture and the recognition result of the air conditioner outdoor unit which are qualified in detection and the picture and the recognition result of the suspected unqualified air conditioner outdoor unit to the cloud platform respectively;
the cloud platform is used for carrying out secondary detection and identification on the images of the suspected unqualified air conditioner outdoor units, storing the images of the air conditioner outdoor units qualified by the secondary detection and identification in a basic data set, and updating the basic data set; and periodically carrying out update training on a YOLOv3-tiny model and a YOLOv3 model by using the updated basic data set, and putting the YOLOv3-tiny model which is subjected to update training into the edge server.
According to the invention, preferably, the edge server is accessed to the 5G network through the MH 5000-315G industrial module to communicate with the cloud platform, so that a product appearance detection communication environment under the 5G environment is constructed.
Adopt hua be 5G LGA industrial module MH5000-31, this module packs together hardware such as 5G baseband chip, radio frequency, storage, power management, provides standard software and hardware interface outward, covers 2G/3G/4G/5G, and down speed is up to 2Gbps, and upward speed is up to 230Mbps, satisfies the high bandwidth requirement of picture transmission and model download in the case. Taking the Ubuntu system as an example, the 5G module supplies power separately through a 5V power supply, first loads a 5G-capable SIM card, then turns on the power supply, and connects to the USB port of the Ubuntu host through a USB, so that the dial-up access to the 5G network can be manually operated.
The invention has the beneficial effects that:
1. according to the requirements of the air conditioner production industrial production line on time delay and identification accuracy, the cloud side division system starts from the aspects of optimizing the architecture, reducing the pressure of the cloud platform and improving the data utilization rate, realizes large-scale calculation and overall analysis of the cloud platform and small-scale calculation and data preprocessing of the edge side by using a cloud side cooperative working mode and by means of a YOLOv3 series algorithm and a socket network socket communication method, and solves the problems in the aspects of accuracy, flexibility, time delay and data utilization rate. And for the new data sets which are subjected to multiple inspection and classification, the reliability is high, the cloud platform periodically trains, updates the model weight and issues the edge device, and the accuracy can be improved to a higher stable value.
2. The method adopts a YOLOv3 series algorithm with mature technology and stable performance for industrial picture target detection, and the YOLOv3 is based on the Darknet-53 network, so that the identification precision is high, the identification speed is high, and the network structure is more simplified; the average accuracy of the Yolov3 model is more than 96%, the single-amplitude detection time is 0.62 seconds/amplitude, the accuracy of the Yolov3-tiny training is more than 90%, and the single-amplitude detection time is greatly shortened by only 0.19 seconds/amplitude compared with the Yolov 3.
3. Through the combined processing of the edge device and the cloud platform, the accuracy and the optimization capability of the identification task are improved, the problem that the model is difficult to update is solved, and the overall flexibility of the detection system is improved; according to the invention, the cloud edge cooperation is utilized, so that the effectiveness of data set construction and the utilization rate of data are effectively improved.
4. This patent combines 5G communication and discrete manufacturing through using the 5G module, inserts the wide area network with mill when satisfying communication quality, time delay demand, realizes cloud, digital and intelligent manufacturing in the mill. Meanwhile, the 5G wireless communication supports low time delay, high bandwidth and large connection, industrial equipment can be liberated from wired network connection, the independence of the equipment is improved, updating, replacement and maintenance are facilitated, and the problems of disorder wiring, difficult maintenance and difficult migration of the industrial equipment under wired communication are solved.
Drawings
FIG. 1 is an architecture diagram of an implementation system of a product appearance detection method based on cloud edge collaborative model optimization according to the present invention;
fig. 2 is a flowchart of a product appearance detection method based on cloud edge collaborative model optimization provided by the invention.
Detailed Description
The invention is further described below, but not limited thereto, with reference to the following examples and the accompanying drawings.
Example 1
In this embodiment, intelligent detection of an air conditioner external unit is taken as an example for explanation, as shown in fig. 2, the method includes the following steps:
s1, collecting pictures of the appearance of the air conditioner outdoor unit, which are known to be qualified or not, and establishing a basic data set after marking the pictures of the appearance of the air conditioner outdoor unit;
s2, respectively training a YOLOv3-tiny model and a YOLOv3 model by using a basic data set, deploying the trained YOLOv3-tiny model at an edge server end, and deploying the trained YOLOv3 model at a cloud platform; and taking the trained YOLOv3-tiny model as an A model and taking the trained YOLOv3 model as a B model.
S3, detecting and identifying the picture of the appearance of the air conditioner outdoor unit to be detected by using the trained YOLOv3-tiny model at the edge server end;
when the detection confidence coefficient is greater than or equal to the set confidence coefficient, judging that the detection is qualified, sending the picture and the detection result of the air conditioner external unit to the cloud platform, and then performing step S5;
when the detection confidence is smaller than the set confidence, determining that the detection is suspected to be unqualified, independently sending the picture of the outdoor unit of the air conditioner to the cloud platform, and then performing step S4;
s4, performing secondary detection and identification on the suspected unqualified picture of the outdoor unit of the air conditioner by using the trained YOLOv3 model on the cloud platform;
when the confidence coefficient of the secondary detection is smaller than the set confidence coefficient, the detection result is unqualified, the detection result is output, and the detection is finished;
when the confidence of the secondary detection is greater than or equal to the set confidence and the detection result is qualified, performing step S5;
s5, storing the qualified picture of the air conditioner external unit in a cloud platform, outputting a detection result, and finishing detection; and after the pictures of the air conditioner external unit are marked, updating the basic data set.
Cloud edge collaboration refers to multi-directional interactive collaboration between a cloud center and network edge devices. Specifically, the central cloud is obtained by virtualizing a large number of high-performance servers, and has the capability of monitoring performance data and working conditions of the edge servers while performing large-scale high-speed computation, so as to interactively achieve resource coordination and work coordination. The cloud edge collaboration in the invention comprises computing resource collaboration, storage resource collaboration and application management collaboration.
The computing resource collaboration is embodied as: according to the difference of cloud and edge computing resources, deployment of different models and allocation of tasks with different computing quantities are carried out, so that the computing resources are fully utilized;
the storage resource is characterized in that: the edge side only needs to store a small amount of temporary data. A large amount of data is sent to the cloud for storage, and the large-capacity storage resources of the cloud are reasonably utilized;
the application management collaboration is embodied as: the edge server provides a deployment environment and an operation environment of the model, and the cloud is responsible for deploying and updating the model at the edge node and monitoring the operation condition.
Example 2
According to the embodiment 1, the product appearance detection method based on cloud edge collaborative model optimization is characterized in that:
the detection method further includes step S6: and using the basic data set obtained by updating in the step S5, periodically performing update training on the YOLOv3-tiny model and the YOLOv3 model, deploying the updated YOLOv3-tiny model to the edge device, and deploying the updated YOLOv3 model to the cloud.
The recognition accuracy can be improved by periodically updating and training the YOLOv3-tiny model and the YOLOv3 model, so that the self-optimization of the models is realized. In the period setting, after the data set is updated last time, the number of newly added pictures in the data set is n, and the updating period is set to be n0. Updating the value of n when a new picture is added to the data set, and when n is equal to n0And the cloud platform starts to update and train the model, and meanwhile, n is reset to be 0, and the next period is waited.
In the step S1, labeling the picture of the air conditioner outdoor unit by using a labellimg image labeling tool, and labeling a part to be detected by using a picture frame, wherein the part to be detected comprises a trademark, an air conditioner cyclone net and an online pipe;
after the file output by the labelimg image labeling tool is processed by Python programming, a file for YOLOv3 or YOLOv3-tiny training is generated, and therefore the construction of an initial effective data set is completed.
In step S2, the yollov 3 model includes 52 convolutional layers, where the 52 convolutional layers include 1 convolutional layer connected to input, 5 convolutional layers for down-sampling, and 23 residual blocks, each residual block includes two convolutional layers, and a short layer is disposed between the two convolutional layers for cross-layer connection;
the YOLOv3 model outputs three output sizes of 52 × 52, 26 × 26 and 13 × 13, and the three output sizes are respectively led out from three residual blocks; the output result comprises a target identification box, a target name label and a detection confidence coefficient.
The YOLO method belongs to a one-stage target detection algorithm, is different from a two-stage method, cannot generate candidate regions (regions), and therefore the identification speed is higher; meanwhile, in the area selection stage of YOLO, the whole picture is input, so that the detection of the image is more accurate and wrong background information is not easy to generate. However, YOLO has a disadvantage in that a 7 × 7 grid is used when a picture is divided, and thus the positioning accuracy of an object may not be sufficient. Compared with a YOLO V1 method, the YOLO V2 method improves precision and detection speed, the YOLOv3 is greatly optimized on the basis of the YOLO V2, a network structure is adjusted, a Darknet-53 network structure is adopted, a multi-scale fusion method is adopted, the recognition capability of small targets is improved, the precision and the speed are improved, the YOLOv3 method is combined with double verification of a cloud edge framework, the precision and the usability of a new data set are improved, and convenience is provided for periodic updating of a model.
The YOLOv3 model is based on a Darknet-53 network structure, and the Darknet-53 network has 52 convolutional layers and 1 full connection layer; the Darknet-53 network introduces shortcut cross-layer connection between convolution layers, so that the network structure is divided into a plurality of residual block residual blocks; the output of the residual block is the superposition of the input of the residual block and the result obtained after the input is subjected to the operation of several convolution layers in the residual block, and the purpose is to reduce the risk of gradient explosion. YOLOv3 is a full convolutional layer structure, adopts the front 52-layer structure of Darknet-53, does not use the last full connection layer of the Darknet-53 network, abandons the pooling layer pooling and adopts the convolutional layer for down-sampling.
According to different detection receptive fields, YOLOv3 can extract features from feature scales with different sizes, and has three output scales of 52 × 52, 26 × 26 and 13 × 13; the feature maps of the three scales can be independently predicted, or can be subjected to upsampling by a convolutional layer to be identical in size and then further predicted after being spliced by a tensor splicing layer (Concat).
In step S2, the YOLOv3-tiny model includes 10 convolutional layers and 8 maxpool maximum pooling layers, and the output size is 13 × 13 × 255; meanwhile, the 5 th convolutional layer and the 8 th convolutional layer of the yollov 3-tiny model respectively output intermediate values of 26 × 26 × 256 and 13 × 13 × 256, the 13 × 13 × 256 outputs a final result through one convolutional layer, the upsampling layer is spliced with the 26 × 26 × 256 tensor, and the two convolutional layers, and the final output size is 26 × 26 × 255.
The model of YOLOv3-tiny is much more simplified than YOLOv3, and no structure of residual block is used. The structure of YOLOv3-tiny is relatively lightweight and can be used for relatively simple tasks.
And in step S3, positioning, detecting and snapshotting an air conditioner external unit on the air conditioner assembly line in real time by using the industrial camera. Real-time video monitoring is not needed, so that the equipment cost is saved while the identification capability is ensured.
In step S3, the picture of the outdoor unit of the air conditioner is compressed at the edge server, and then uploaded to the cloud platform from the edge server. The data transmission quantity can be reduced, and the cost is saved.
The average accuracy of the Yolov3 model is more than 96%, the single-amplitude detection time is 0.62 seconds/amplitude, the accuracy of the Yolov3-tiny training is more than 90%, and the single-amplitude detection time is greatly shortened by only 0.19 seconds/amplitude compared with the Yolov 3.
Example 3
According to the embodiment 1, the product appearance detection method based on cloud edge collaborative model optimization is characterized in that:
the edge server and the cloud platform are communicated by socket network sockets, the edge server serves as a client, and the cloud platform serves as a server to perform bidirectional communication.
And in the aspect of communication between the edge device and the cloud platform, a stable and feasible socket network socket is adopted for communication. socket communication firstly needs to establish sockets at a client and a server to obtain a host name and a port number; the server side is used for monitoring the request, after the client side sends the communication request, the client side and the server side establish connection through three-way handshake, and then data transmission is completed through encoding sending and decoding receiving; after the transmission is finished, the client and the server are disconnected by waving hands for four times.
Example 4
In any of embodiments 1 to 3, as shown in fig. 1, an edge sensing device is connected to an edge server, and is configured to obtain a picture of an air conditioner external unit to be detected, and transmit the picture of the air conditioner external unit to the edge server;
the edge server is in communication connection with the cloud platform, carries out first detection on the picture of the air conditioner outdoor unit acquired by the edge sensing equipment by using a trained YOLOv3-tiny model, and uploads the picture and the recognition result of the air conditioner outdoor unit which are qualified in detection and the picture and the recognition result of the suspected unqualified air conditioner outdoor unit to the cloud platform respectively;
the cloud platform is used for carrying out secondary detection and identification on the images of the suspected unqualified air conditioner outdoor units, storing the images of the air conditioner outdoor units qualified by the secondary detection and identification in a basic data set, and updating the basic data set; and periodically carrying out update training on a YOLOv3-tiny model and a YOLOv3 model by using the updated basic data set, and putting the YOLOv3-tiny model which is subjected to update training into the edge server.
The cloud platform has strong data processing capacity, a high-performance server is selected to serve as a cloud end, functions of data gathering storage, webpage real-time display and the like are achieved, a deep learning model can be trained and updated, and the cloud platform is a core link of a three-layer framework.
The edge server layer can perform the identification function of the simple AI model and can display the result in real time; the edge server layer can adopt equipment with basic deep learning computing power such as raspberry groups, and meanwhile has certain data storage capacity, and can be in data communication with the cloud platform layer after being connected to a network.
And the edge sensor layer is connected with the edge server layer and can finish the acquisition work of corresponding industrial data.
Example 5
The implementation system of the product appearance detection method based on cloud edge collaborative model optimization according to embodiment 4 is characterized in that:
the edge server is accessed to the 5G network through the MH 5000-315G industrial module to communicate with the cloud platform, and a discrete manufacturing industry intelligent detection communication environment under the 5G environment is constructed.
Adopt hua be 5G LGA industrial module MH5000-31, this module packs together hardware such as 5G baseband chip, radio frequency, storage, power management, provides standard software and hardware interface outward, covers 2G/3G/4G/5G, and down speed is up to 2Gbps, and upward speed is up to 230Mbps, satisfies the high bandwidth requirement of picture transmission and model download in the case. Taking the Ubuntu system as an example, the 5G module supplies power separately through a 5V power supply, first loads a 5G-capable SIM card, then turns on the power supply, and connects to the USB port of the Ubuntu host through a USB, so that the dial-up access to the 5G network can be manually operated.
Claims (10)
1. A product appearance detection method based on cloud edge collaborative model optimization is characterized by comprising the following steps:
s1, collecting whether the picture of the product appearance is known to be qualified or not, and establishing a basic data set after marking the picture of the product appearance;
s2, respectively training a YOLOv3-tiny model and a YOLOv3 model by using a basic data set, deploying the trained YOLOv3-tiny model at an edge server end, and deploying the trained YOLOv3 model at a cloud platform;
s3, at the edge server end, detecting and identifying the picture of the product appearance needing to be detected by using the trained YOLOv3-tiny model;
when the detection confidence coefficient is greater than or equal to the set confidence coefficient, judging that the product is qualified, sending the picture of the product outdoor unit and the detection result to the cloud platform, and then performing step S5;
when the detection confidence is smaller than the set confidence, determining that the product is suspected to be unqualified, independently sending the picture of the product outdoor unit to a cloud platform, and then performing step S4;
s4, carrying out secondary detection and identification on the picture of the suspected unqualified product outdoor unit by using a trained YOLOv3 model on a cloud platform;
when the confidence coefficient of the secondary detection is smaller than the set confidence coefficient, the detection result is unqualified, the detection result is output, and the detection is finished;
when the confidence of the secondary detection is greater than or equal to the set confidence and the detection result is qualified, performing step S5;
s5, storing the qualified picture of the product outdoor unit in a cloud platform, outputting a detection result, and finishing detection; and after the pictures of the product external machine are marked, updating the basic data set.
2. The product appearance detection method based on cloud edge collaborative model optimization according to claim 1, further comprising step S6: and using the basic data set obtained by updating in the step S5, periodically performing update training on the YOLOv3-tiny model and the YOLOv3 model, deploying the updated YOLOv3-tiny model to the edge device, and deploying the updated YOLOv3 model to the cloud.
3. The product appearance detection method based on cloud-edge collaborative model optimization according to claim 1, wherein in step S3, the external machine of the product on the product pipeline is positioned, detected and captured in real time by using an industrial camera.
4. The product appearance detection method based on cloud edge collaborative model optimization according to claim 1, wherein in step S1, a labellimg image labeling tool is used to label the picture of the product outdoor unit, the frame is used to label the part to be detected,
after the file output by the labelimg image labeling tool is processed by Python programming, a file for YOLOv3 or YOLOv3-tiny training is generated, and therefore the construction of an initial effective data set is completed.
5. The product appearance detection method based on cloud edge collaborative model optimization according to claim 1, wherein in step S2, the YOLOv3 model includes 52 convolutional layers, the 52 convolutional layers include 1 convolutional layer connected to input, 5 convolutional layers for down-sampling, and 23 residual blocks, each residual block includes two convolutional layers, and a shortcut is set between the two convolutional layers for cross-layer connection;
the YOLOv3 model outputs three output sizes of 52 × 52, 26 × 26 and 13 × 13, and the three output sizes are respectively led out from three residual blocks; the output result comprises a target identification box, a target name label and a detection confidence coefficient.
6. The product appearance detection method based on cloud edge collaborative model optimization according to claim 1, wherein in step S2, the YOLOv3-tiny model includes 10 convolutional layers and 8 maxpool maximum pooling layers, and the output size is 13 × 13 × 255; meanwhile, the 5 th convolutional layer and the 8 th convolutional layer of the yollov 3-tiny model respectively output intermediate values of 26 × 26 × 256 and 13 × 13 × 256, the 13 × 13 × 256 outputs a final result through one convolutional layer, the upsampling layer is spliced with the 26 × 26 × 256 tensor, and the two convolutional layers, and the final output size is 26 × 26 × 255.
7. The product appearance detection method based on cloud-edge collaborative model optimization according to claim 1, characterized in that a socket network socket is adopted between an edge server and a cloud platform for communication, the edge server serves as a client, and the cloud platform serves as a server for bidirectional communication.
8. The product appearance detection method based on cloud edge collaborative model optimization according to claim 1, wherein in step S3, the picture of the product appearance is compressed at the edge server side and then uploaded from the edge server side to the cloud platform.
9. The system for implementing the intelligent detection method for the outdoor unit of the air conditioner based on the cloud-edge collaborative model optimization according to any one of claims 1 to 8, comprising a cloud platform, an edge server and an edge sensing device,
the edge sensing equipment is connected with the edge server and used for acquiring a picture of the air conditioner external unit to be detected and transmitting the picture of the air conditioner external unit to the edge server;
the edge server is in communication connection with the cloud platform, carries out first detection on the picture of the air conditioner outdoor unit acquired by the edge sensing equipment by using a trained YOLOv3-tiny model, and uploads the picture and the recognition result of the air conditioner outdoor unit which are qualified in detection and the picture and the recognition result of the suspected unqualified air conditioner outdoor unit to the cloud platform respectively;
the cloud platform is used for carrying out secondary detection and identification on the images of the suspected unqualified air conditioner outdoor units, storing the images of the air conditioner outdoor units qualified by the secondary detection and identification in a basic data set, and updating the basic data set; and periodically carrying out update training on a YOLOv3-tiny model and a YOLOv3 model by using the updated basic data set, and putting the YOLOv3-tiny model which is subjected to update training into the edge server.
10. The system for implementing the intelligent detection method for the outdoor unit of the air conditioner based on the cloud-edge collaborative model optimization according to claim 9, wherein an edge server is connected to a 5G network through an MH 5000-315G industrial module to communicate with a cloud platform, so that a product appearance detection communication environment in a 5G environment is constructed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011602970.0A CN112788110A (en) | 2020-12-29 | 2020-12-29 | Product appearance detection method based on cloud edge collaborative model optimization and implementation system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011602970.0A CN112788110A (en) | 2020-12-29 | 2020-12-29 | Product appearance detection method based on cloud edge collaborative model optimization and implementation system thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112788110A true CN112788110A (en) | 2021-05-11 |
Family
ID=75751690
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011602970.0A Pending CN112788110A (en) | 2020-12-29 | 2020-12-29 | Product appearance detection method based on cloud edge collaborative model optimization and implementation system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112788110A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113657207A (en) * | 2021-07-29 | 2021-11-16 | 威胜信息技术股份有限公司 | Intelligent monitoring method and system for fire and light of cloud-edge cooperative power distribution station |
| CN114136374A (en) * | 2021-11-25 | 2022-03-04 | 江中药业股份有限公司 | Tablet online sampling detection control system and tablet quality detection system |
| CN115761684A (en) * | 2023-01-10 | 2023-03-07 | 常熟理工学院 | AGV target recognition and attitude angle resolving method and system based on machine vision |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200133254A1 (en) * | 2018-05-07 | 2020-04-30 | Strong Force Iot Portfolio 2016, Llc | Methods and systems for data collection, learning, and streaming of machine signals for part identification and operating characteristics determination using the industrial internet of things |
| US20200175334A1 (en) * | 2018-11-29 | 2020-06-04 | Qualcomm Incorporated | Edge computing |
| CN111431986A (en) * | 2020-03-18 | 2020-07-17 | 宁波智诚祥科技发展有限公司 | Industrial intelligent quality inspection system based on 5G and AI cloud edge cooperation |
| CN112101175A (en) * | 2020-09-09 | 2020-12-18 | 沈阳帝信人工智能产业研究院有限公司 | Expressway vehicle detection and multi-attribute feature extraction method based on local images |
-
2020
- 2020-12-29 CN CN202011602970.0A patent/CN112788110A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200133254A1 (en) * | 2018-05-07 | 2020-04-30 | Strong Force Iot Portfolio 2016, Llc | Methods and systems for data collection, learning, and streaming of machine signals for part identification and operating characteristics determination using the industrial internet of things |
| US20200175334A1 (en) * | 2018-11-29 | 2020-06-04 | Qualcomm Incorporated | Edge computing |
| CN111431986A (en) * | 2020-03-18 | 2020-07-17 | 宁波智诚祥科技发展有限公司 | Industrial intelligent quality inspection system based on 5G and AI cloud edge cooperation |
| CN112101175A (en) * | 2020-09-09 | 2020-12-18 | 沈阳帝信人工智能产业研究院有限公司 | Expressway vehicle detection and multi-attribute feature extraction method based on local images |
Non-Patent Citations (1)
| Title |
|---|
| 赵永强: "深度学习目标检测方法综述", 《中国图象图形学报》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113657207A (en) * | 2021-07-29 | 2021-11-16 | 威胜信息技术股份有限公司 | Intelligent monitoring method and system for fire and light of cloud-edge cooperative power distribution station |
| CN113657207B (en) * | 2021-07-29 | 2023-08-08 | 威胜信息技术股份有限公司 | Cloud-side cooperative power distribution station fire light intelligent monitoring method and system |
| CN114136374A (en) * | 2021-11-25 | 2022-03-04 | 江中药业股份有限公司 | Tablet online sampling detection control system and tablet quality detection system |
| CN114136374B (en) * | 2021-11-25 | 2023-11-21 | 江中药业股份有限公司 | Tablet on-line sampling detection control system and tablet quality detection system |
| CN115761684A (en) * | 2023-01-10 | 2023-03-07 | 常熟理工学院 | AGV target recognition and attitude angle resolving method and system based on machine vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112788110A (en) | Product appearance detection method based on cloud edge collaborative model optimization and implementation system thereof | |
| CN108171117B (en) | Electric power artificial intelligence visual analysis system based on multicore heterogeneous Computing | |
| CN110674861B (en) | Intelligent analysis method and device for power transmission and transformation inspection images | |
| US20220180600A1 (en) | 3d reconstruction method based on on-site edge-cloud collaboration for cultivated land | |
| CN108764456B (en) | Airborne target identification model construction platform, airborne target identification method and equipment | |
| CN112668237A (en) | Digital twin model based on industrial internet cloud platform and construction method thereof | |
| CN115457006B (en) | Unmanned aerial vehicle inspection defect classification method and device based on similarity consistency self-distillation | |
| CN108683877A (en) | Distributed massive video resolution system based on Spark | |
| CN110379036A (en) | Intelligent substation patrol recognition methods, system, device and storage medium | |
| CN105447647A (en) | Shield intelligent maintenance management system | |
| CN113129284B (en) | Appearance detection method based on 5G cloud edge cooperation and implementation system | |
| CN108093419A (en) | A kind of base station information acquiring system and its control method based on Internet of Things | |
| CN108197632A (en) | The identifying system and its method that distributed Freehandhand-drawing based on convolutional neural networks is drawn | |
| CN113312957A (en) | off-Shift identification method, device, equipment and storage medium based on video image | |
| CN113205039A (en) | Power equipment fault image identification and disaster investigation system and method based on multiple DCNNs | |
| CN113556768A (en) | Sensor data anomaly detection method and system | |
| CN110415267A (en) | A kind of online thermal infrared target identification device of low-power consumption and working method | |
| CN116092199A (en) | Employee working state identification method and identification system | |
| CN114332659A (en) | Power transmission line defect inspection method and device based on lightweight model issuing | |
| CN113408630A (en) | Transformer substation indicator lamp state identification method | |
| CN112884164A (en) | Federal machine learning migration method and system for intelligent mobile terminal | |
| CN108366229A (en) | A kind of pointing device intelligence method for inspecting | |
| CN115077488B (en) | Factory personnel real-time positioning and monitoring system and method based on digital twinning | |
| CN212459378U (en) | Ground object hyperspectral meter remote sensing land utilization sample acquisition instrument based on unmanned aerial vehicle | |
| CN109147962A (en) | A kind of geologic prospect remote medical consultation with specialists and supervisory systems based on Internet of Things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210511 |
|
| RJ01 | Rejection of invention patent application after publication |