CN116578323A - Deep learning algorithm iteration method based on Yun Bian cooperation - Google Patents
Deep learning algorithm iteration method based on Yun Bian cooperation Download PDFInfo
- Publication number
- CN116578323A CN116578323A CN202310864736.2A CN202310864736A CN116578323A CN 116578323 A CN116578323 A CN 116578323A CN 202310864736 A CN202310864736 A CN 202310864736A CN 116578323 A CN116578323 A CN 116578323A
- Authority
- CN
- China
- Prior art keywords
- data
- algorithm
- edge computing
- module
- cloud platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000013135 deep learning Methods 0.000 title claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 128
- 238000004891 communication Methods 0.000 claims description 37
- 230000008569 process Effects 0.000 claims description 23
- 238000001514 detection method Methods 0.000 claims description 18
- 238000011156 evaluation Methods 0.000 claims description 16
- 238000007781 pre-processing Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 12
- 238000002372 labelling Methods 0.000 claims description 11
- 230000011218 segmentation Effects 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 8
- 230000003993 interaction Effects 0.000 claims description 7
- 238000004140 cleaning Methods 0.000 claims description 6
- 230000007547 defect Effects 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000003672 processing method Methods 0.000 claims description 4
- 230000004807 localization Effects 0.000 claims description 3
- 238000004513 sizing Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 18
- 210000004027 cell Anatomy 0.000 description 11
- 238000010586 diagram Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 210000000265 leukocyte Anatomy 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000003651 basophil Anatomy 0.000 description 1
- 210000001772 blood platelet Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 210000003979 eosinophil Anatomy 0.000 description 1
- 210000003743 erythrocyte Anatomy 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 210000004698 lymphocyte Anatomy 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 210000005087 mononuclear cell Anatomy 0.000 description 1
- 210000000440 neutrophil Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a Yun Bian cooperation-based deep learning algorithm iteration method, which comprises the steps that S1, after edge computing equipment is deployed on site, a corresponding image processing algorithm configuration request is sent to a cloud platform according to site requirements; s2, after the cloud receives the request, issuing a corresponding image processing algorithm in the algorithm warehouse to corresponding edge computing equipment; s3, the edge computing equipment tries to run an image processing algorithm, and sends the running state, the running result and the preprocessed image data to the cloud; s4, the cloud end analyzes the data according to the data sent by the side end equipment, if the processing result meets the expected requirement, the S3 is returned, and otherwise, the S5 is entered; s5, the cloud platform completes online data marking and auditing according to the data which does not meet the expected requirement, and the algorithm model is trained to obtain an optimal model; s6: and the cloud platform transmits the iterated optimal model and related configuration to the corresponding edge computing equipment for automatic updating operation. The iteration efficiency is high.
Description
Technical Field
The invention relates to the technical field of equipment management and application of the Internet of things, in particular to a Yun Bian cooperation-based deep learning algorithm iteration method.
Background
The image processing system based on deep learning can detect objects and features appearing in different forms and can better identify objects in a variable environment, so that the image processing system based on deep learning is widely applied to industrial production. However, algorithm models used by the deep learning-based system when different detection tasks are executed are different, and manual deployment of a debugging algorithm is very labor-consuming if the debugging algorithm is required to be deployed according to different field requirements; meanwhile, in an industrial field of practical application, updating iteration is needed to be realized on an algorithm model according to the actual condition of the field, the algorithm iteration of an image processing system based on deep learning needs a large amount of field data support, data are required to be collected from the industrial field for marking, a high-performance server is used for loading a convolutional neural network for training, and the algorithm after iteration upgrading is manually updated to a corresponding field system.
Disclosure of Invention
Aiming at the technical problems, the invention provides a Yun Bian cooperation-based deep learning algorithm iteration method.
The technical scheme adopted for solving the technical problems is as follows:
an iterative method of a deep learning algorithm based on Yun Bian cooperation, the method comprises the following steps:
s100: the method comprises the steps of deploying edge computing equipment to an industrial site, connecting the edge computing equipment with a vision sensor, establishing communication with a cloud platform, and acquiring image data transmitted by the vision sensor and task requirements of the site; the cloud platform defines a method for processing the image data, and a calling method corresponding to the image data processing method is defined on the visual sensor; obtaining a corresponding image processing algorithm configuration request according to the image data transmitted by the vision sensor and the task requirements of the site, and sending the image processing algorithm configuration request to the cloud platform;
s200: after receiving the image processing algorithm configuration request, the cloud platform acquires a corresponding image processing algorithm package from an algorithm warehouse and transmits the corresponding image processing algorithm package to corresponding edge computing equipment; the image processing algorithm package comprises an algorithm model;
s300: the edge computing equipment pre-processes the image data to obtain pre-processed image data, and runs an image processing algorithm package to obtain running conditions and processing results, and the running conditions, the processing results and the pre-processed image data are sent to the cloud platform;
s400: the cloud platform analyzes the running condition and the processing result, judges whether the processing result of the corresponding edge computing device meets the expected requirement, if so, returns to the step S300 to process the next image data, and if not, enters the next step;
s500: the cloud platform takes out the data with the running result not reaching the expected requirement from the preprocessed image data, completes online data marking and auditing, automatically carries out algorithm model training according to new marking auditing data and historical data, generates a new algorithm model, carries out test verification to obtain an optimal model, and updates an image processing algorithm package;
s600: and the cloud platform transmits the updated image processing algorithm package to the corresponding edge computing equipment, and the edge computing equipment automatically updates and operates.
Preferably, establishing communication between the edge computing device and the cloud platform in step S100 includes enabling a single edge computing device to communicate with the cloud platform, or enabling multiple edge computing devices to establish an edge cluster to communicate with the cloud platform.
Preferably, in step S100, the edge computing device connects with the vision sensor and establishes communication with the cloud platform, including:
the edge computing device can establish data connection with the visual sensor through an RJ-45 Ethernet interface and USB3.2gen1;
the edge computing device can independently communicate with the cloud platform through any one of WI-FI, ethernet and 4G, and upload running conditions, processing results and preprocessed image data.
Preferably, the method further comprises:
the edge computing equipment on the same site can establish an edge cluster through the gateway of the Internet of things by utilizing an RS232 or RS484 serial port, and the edge computing equipment communicates with the cloud platform to upload data of the edge computing equipment in the edge cluster.
Preferably, the edge computing device comprises a first communication service module, an algorithm configuration and updating module, an algorithm running module and a system management module, wherein the first communication service module is connected with the algorithm configuration and updating module, the algorithm configuration and updating module is connected with the algorithm running module, the algorithm running module is connected with the system management module, and the system management module is connected with the first communication service module;
the first communication service module is used for receiving and transmitting data interacted by the cloud platform and serving data interaction with other Internet entities;
the algorithm configuration and updating module is used for downloading a corresponding image processing algorithm package from the cloud platform according to the field use requirement, configuring according to the field actual condition, and automatically updating the algorithm package after iteration of the cloud platform after the configuration is completed;
the algorithm running module is used for carrying out image processing running in real time according to the algorithm model in the algorithm package and interacting with the terminal equipment;
the system management module is used for monitoring and managing the running condition, the processing process and the result, selectively storing the running data generated in the running process in a local place according to the preset requirement, encrypting and storing the global data generated in the processing process and uploading the global data to the cloud platform.
Preferably, the image processing algorithm package includes a defect detection image processing algorithm, a sizing image processing algorithm, an object localization image processing algorithm, and an OCR recognition image processing algorithm.
Preferably, the cloud platform comprises a second communication service module, a device management module, a data warehouse module, an algorithm warehouse module, a data labeling and auditing module and a model iteration and evaluation module which are all connected with the second communication service module, wherein the data warehouse module is connected with the data labeling and auditing module which is connected with the model iteration and evaluation module, and the model iteration and evaluation module is connected with the algorithm warehouse module;
the device management module is used for analyzing the operation data uploaded by the edge computing device, judging whether the image processing result reaches an expected target or not, and displaying the operation condition of the online device in a centralized manner in an instrument panel;
the data warehouse module is used for storing all historical software versions and running logs uploaded by the edge computing equipment, backing up backtracking, and classifying and storing the preprocessed image data uploaded by the edge computing equipment according to the difference of the image processing algorithm packages operated by the edge computing equipment;
the algorithm warehouse module is used for storing algorithm packages of different application types and historical versions thereof, the algorithm packages comprise all necessary files for running corresponding applications, when the edge computing equipment initiates an image processing algorithm configuration request, the corresponding algorithm packages are extracted according to the image processing algorithm configuration request and sent to the corresponding edge computing equipment, meanwhile, an algorithm model after each training iteration is also stored in the algorithm warehouse, and the algorithm warehouse carries out iteration model issuing;
the data marking and auditing function module is used for providing an online data standard platform, when the processing result does not reach the expected value, taking out the data with the non-ideal operation result from the data warehouse, and pushing the data marking and auditing task at the data marking and auditing platform so as to enable a back-end maintainer to complete the marking and auditing of the data online;
the model iteration and evaluation module is used for processing an algorithm task according to the image to be optimized, automatically training and evaluating the algorithm model based on new annotation auditing data and historical data, and submitting the algorithm model after iteration to an algorithm warehouse;
the second communication service module is responsible for data transceiving of the edge computing device and data interaction services with other internet entities.
Preferably, in step S300, the edge computing device performs a preprocessing operation on the image data to obtain a preprocessing rule of the data preset for different image processing tasks, where the preprocessing rule of the data may be updated remotely through the cloud platform.
Preferably, the preset data preprocessing rule comprises image pre-segmentation, image data de-duplication and data cleaning, wherein the image pre-segmentation is to segment the image data uploaded by the visual sensor in advance, cut out partial background and areas which do not need to be detected, and reserve the areas which need to be detected by the image algorithm; the image data de-duplication is the image data repeated by automatic shielding, and the uploading processing is not carried out on the image data; the data cleaning is to clean repeated operation conditions, non-operation image processing schemes, processing results output by invalid equipment and invalid image data in the data which needs to be uploaded by the edge computing equipment.
Preferably, in step S300, the operation status and the processing result uploaded by the edge computing device are data uploaded in real time, and the preprocessed image data need to be uploaded when the image detection system is not operated; the image detection system is a program system for calling an algorithm package issued by the cloud platform to process the image.
According to the deep learning algorithm iteration method based on Yun Bian cooperation, different image processing algorithm packages can be rapidly deployed to the edge computing equipment by utilizing the convenience of cloud edge cooperation, a large amount of manpower and material resources are saved, abundant computing and storage resources of a cloud platform are utilized, a convenient and rapid online data labeling and auditing mode is adopted, and efficient iteration upgrading of the deep learning algorithm model is realized on the basis of an image algorithm iteration model with stable increment.
Drawings
FIG. 1 is a flowchart of an iterative method of a deep learning algorithm based on Yun Bian cooperation according to an embodiment of the present invention;
FIG. 2 is a hardware architecture diagram of an iterative method of a deep learning algorithm based on Yun Bian cooperation according to an embodiment of the present invention;
FIG. 3 is a diagram of an edge computing device service entity according to an embodiment of the present invention;
fig. 4 is a diagram illustrating a cloud end platform service entity according to an embodiment of the present invention.
Detailed Description
In order to make the technical scheme of the present invention better understood by those skilled in the art, the present invention will be further described in detail with reference to the accompanying drawings.
In one embodiment, as shown in fig. 1, a method for iterating a deep learning algorithm based on Yun Bian cooperation, the method comprises the following steps:
s100: the method comprises the steps of deploying edge computing equipment to an industrial site, connecting the edge computing equipment with a vision sensor, establishing communication with a cloud platform, and acquiring image data transmitted by the vision sensor and task requirements of the site; the cloud platform defines a method for processing the image data, and a calling method corresponding to the image data processing method is defined on the visual sensor; and obtaining a corresponding image processing algorithm configuration request according to the image data transmitted by the vision sensor and the task requirement of the site, and sending the image processing algorithm configuration request to the cloud platform.
In one embodiment, as shown in fig. 2, the step of establishing communication between the edge computing device and the cloud platform in step S100 includes the step of establishing communication between the edge computing device and the cloud platform by a single edge computing device, or establishing communication between the edge cluster and the cloud platform by a plurality of edge computing devices.
In one embodiment, the edge computing device in step S100 connects with the vision sensor and establishes communication with the cloud platform, including:
the edge computing device can establish data connection with the visual sensor through an RJ-45 Ethernet interface and USB3.2gen1;
the edge computing device can independently communicate with the cloud platform through any one of WI-FI, ethernet and 4G, and upload running conditions, processing results and preprocessed image data.
In one embodiment, further comprising:
the edge computing equipment on the same site can establish an edge cluster through the gateway of the Internet of things by utilizing an RS232 or RS484 serial port, and the edge computing equipment communicates with the cloud platform to upload data of the edge computing equipment in the edge cluster.
Specifically, the edge computing device can establish data connection with the vision sensor through an RJ-45 Ethernet interface and the highest USB3.2gen1, and acquire image data of the image sensor in real time; the edge computing equipment can independently communicate with the cloud through any one of WI-FI, ethernet and 4G, and upload running conditions, processing results and cache image data; and the edge computing equipment on the same site establishes an edge cluster through the gateway of the Internet of things by utilizing an RS232 or RS484 serial port, and the data of the edge computing equipment in the whole cluster is uploaded by one edge computing equipment.
In one embodiment, as shown in fig. 3, the edge computing device includes a first communication service module, an algorithm configuration and update module, an algorithm running module and a system management module, where the first communication service module is connected with the algorithm configuration and update module, the algorithm configuration and update module is connected with the algorithm running module, the algorithm running module is connected with the system management module, and the system management module is connected with the first communication service module;
the first communication service module is used for receiving and transmitting data interacted by the cloud platform and serving data interaction with other Internet entities;
the algorithm configuration and updating module is used for downloading a corresponding image processing algorithm package from the cloud platform according to the field use requirement, configuring according to the field actual condition, and automatically updating the algorithm package after iteration of the cloud platform after the configuration is completed;
the algorithm running module is used for carrying out image processing running in real time according to the algorithm model in the algorithm package and interacting with the terminal equipment;
the system management module is used for monitoring and managing the running condition, the processing process and the result, selectively storing the running data generated in the running process in a local place according to the preset requirement, encrypting and storing the global data generated in the processing process and uploading the global data to the cloud platform.
In one embodiment, the image processing algorithm package includes a defect detection image processing algorithm, a sizing image processing algorithm, an object localization image processing algorithm, and an OCR recognition image processing algorithm.
Specifically, the edge computing device mainly comprises four functional modules, namely a first communication service, algorithm configuration and updating, algorithm running and system management, wherein the four functional modules endow the edge computing device with the capabilities of carrying out real-time image processing and data storage uploading; the algorithm configuration and updating module can download different image processing algorithm packages from the cloud according to the field use requirement, including but not limited to image processing algorithms such as defect detection, dimension measurement, target positioning, OCR recognition and the like, automatically configure and deploy an algorithm model according to the field actual situation, and automatically update the algorithm package after cloud iteration after configuration is completed; the algorithm running module endows the edge computing equipment with actual applications such as measurement, detection, positioning, identification and the like in real time according to the task requirements of the site, and interacts with the terminal equipment; the system management module is responsible for monitoring and managing the running condition, the processing process and the result, selectively stores data generated in the running process, including but not limited to equipment running log and image data, in a local place and uploads the data to the cloud end according to requirements.
Further, field use requirements refer to different use requirements for different processing requirements. The algorithmic model framework used is also different, for example, for different types or sample images of objects. And calling corresponding processing methods for different use requirements such as detection, identification, classification, segmentation and the like to obtain different algorithm packages. The terminal equipment is different equipment according to actual application scenes; for example, the edge computer terminal detects an image target acquired by the camera by calling an algorithm package issued by the cloud to obtain coordinate position information of the target. The preset requirements are for example to detect and identify the cell images, and detect and identify, classify and divide the cells in the images according to the requirements; the parameter data in the algorithm processing process is stored according to the requirement: 1. the number of cells in the image detected according to the detection model; 2. counting the number of cells in different categories according to the classification model; 3. and calculating the average area of the cells of the image according to the area of the cells in the image calculated by the segmentation model.
The global data is output data generated in the process of processing the image by utilizing an algorithm, and the main data of the detection task comprises information such as coordinate position information, confidence and the like of an image target; the classification task mainly comprises the category information of the image and the confidence of the category; for the segmentation task, contour information of an image and the like are included. Encryption is implemented by providing a secure channel between a user and a data center node using an encrypted network communication channel, such as using a virtual private network, thereby ensuring that the transmitted data is secure.
S200: after receiving the image processing algorithm configuration request, the cloud platform acquires a corresponding image processing algorithm package from an algorithm warehouse and transmits the corresponding image processing algorithm package to corresponding edge computing equipment; wherein the image processing algorithm package comprises an algorithm model.
In one embodiment, as shown in fig. 4, the cloud platform includes a second communication service module, a device management module, a data warehouse module, an algorithm warehouse module, a data labeling and auditing module and a model iteration and evaluation module all connected with the second communication service module, the data warehouse module is connected with the data labeling and auditing module, the data labeling and auditing module is connected with the model iteration and evaluation module, and the model iteration and evaluation module is connected with the algorithm warehouse module;
the device management module is used for analyzing the operation data uploaded by the edge computing device, judging whether the image processing result reaches an expected target or not, and displaying the operation condition of the online device in a centralized manner in an instrument panel;
the data warehouse module is used for storing all historical software versions and running logs uploaded by the edge computing equipment, backing up backtracking, and classifying and storing the preprocessed image data uploaded by the edge computing equipment according to the difference of the image processing algorithm packages operated by the edge computing equipment;
the algorithm warehouse module is used for storing algorithm packages of different application types and historical versions thereof, the algorithm packages comprise all necessary files for running corresponding applications, when the edge computing equipment initiates an image processing algorithm configuration request, the corresponding algorithm packages are extracted according to the image processing algorithm configuration request and sent to the corresponding edge computing equipment, meanwhile, an algorithm model after each training iteration is also stored in the algorithm warehouse, and the algorithm warehouse carries out iteration model issuing;
the data marking and auditing function module is used for providing an online data standard platform, when the processing result does not reach the expected value, taking out the data with the non-ideal operation result from the data warehouse, and pushing the data marking and auditing task at the data marking and auditing platform so as to enable a back-end maintainer to complete the marking and auditing of the data online;
the model iteration and evaluation module is used for processing an algorithm task according to the image to be optimized, automatically training and evaluating the algorithm model based on new annotation auditing data and historical data, and submitting the algorithm model after iteration to an algorithm warehouse;
the second communication service module is responsible for data transceiving of the edge computing device and data interaction services with other internet entities.
Specifically, the cloud platform mainly comprises five functional modules, namely a second communication service, equipment management, a data warehouse, an algorithm warehouse, data labeling and auditing, model iteration and evaluation, wherein the five functional modules endow the cloud with management capability, data and algorithm storage capability and algorithm iteration optimization capability for the terminal; the device management module is responsible for analyzing the operation data uploaded by the edge computing device according to the operation data, judging whether an image processing result of the device reaches an expected target or not, and displaying the operation condition of the online device in a centralized manner in an instrument panel; the data warehouse module is responsible for storing all historical operation logs uploaded by the edge computing equipment for backups and backups, and meanwhile, the image data can be classified and stored according to the corresponding algorithm model; the algorithm warehouse functional module stores algorithm packages of different application types (such as character recognition, defect detection, size measurement, target positioning and the like) and historical versions thereof, the algorithm packages comprise all necessary files for running the application, including but not limited to algorithm model files, other dependent files for running the algorithm model and related configuration files, and when the edge computing equipment initiates an application, the cloud end sends the corresponding algorithm packages to the terminal or the terminal cluster; meanwhile, the model after each training iteration is also stored in an algorithm warehouse, and the algorithm warehouse performs iterative model issuing; the data marking and auditing function module provides an online data standard platform, when the operation result of the side end does not reach the expected value, the cloud takes out the data with the non-ideal operation result from the data warehouse, the data marking and auditing task is pushed in the data marking and auditing platform, and the rear maintenance personnel completes the marking and auditing of the data online; the model iteration and evaluation module is responsible for optimizing an image processing algorithm task according to requirements, automatically carries out model training and evaluation based on new annotation auditing data and historical data, and submits the iterative algorithm model to an algorithm warehouse; the second communication service module is responsible for data transceiving of the edge computing device and data interaction services with other internet entities.
Further, the operation data includes: (1) operating conditions: whether the edge equipment successfully calls the algorithm model or not, and if the edge equipment fails to call, returning an error code for analyzing the error reason; (2) processing results: if the algorithm package is successfully called, the recognition result of the image calling algorithm model is returned, and the recognition result possibly contains the position information, the confidence level and the like of the target. Software version control and logging data: for storing the successfully running and released software version and the log record of the program during the running process. And (3) screening, processing and uploading the collected image data to the cloud according to the related processing in the algorithm package.
S300: the edge computing equipment pre-processes the image data to obtain pre-processed image data, and the image processing algorithm package is operated to obtain operation conditions and processing results, and the operation conditions, the processing results and the pre-processed image data are sent to the cloud platform.
In one embodiment, the edge computing device performs a preprocessing operation on the image data in step S300 to preset data preprocessing rules for different image processing tasks, where the data preprocessing rules may be updated remotely through the cloud platform.
Specifically, the image processing task generally includes object detection of an image, classification of an image, segmentation of an image, and the like. Specific items are for example: detection and identification of cells in a cell image, classification of cells such as platelets, erythrocytes, leukocytes (where leukocytes can also be classified as eosinophils/basophils/neutrophils/lymphocytes/mononuclear cells), division of cells, calculation of cell area, and the like.
In one embodiment, the preset data preprocessing rule comprises image pre-segmentation, image data de-duplication and data cleaning, wherein the image pre-segmentation is performed on the image data uploaded by the video sensor in advance, a part of background and an area which does not need to be detected are cut off, and an area which needs to be detected by an image algorithm is reserved; the image data de-duplication is the image data repeated by automatic shielding, and the uploading processing is not carried out on the image data; the data cleaning is to clean repeated operation conditions, non-operation image processing schemes, processing results output by invalid equipment and invalid image data in the data which needs to be uploaded by the edge computing equipment.
In one embodiment, the operation status and the processing result uploaded by the edge computing device in step S300 are data uploaded in real time, and the preprocessed image data needs to be uploaded when the image detection system is not operated; the image detection system is a program system for calling an algorithm package issued by the cloud platform to process the image.
S400: the cloud platform analyzes the running condition and the processing result, judges whether the processing result of the corresponding edge computing device meets the expected requirement, if so, returns to the step S300 to process the next image data, and if not, enters the next step;
s500: the cloud platform takes out the data with the running result not reaching the expected requirement from the preprocessed image data, completes online data marking and auditing, automatically carries out algorithm model training according to new marking auditing data and historical data, generates a new algorithm model, carries out test verification to obtain an optimal model, and updates an image processing algorithm package;
s600: and the cloud platform transmits the updated image processing algorithm package to the corresponding edge computing equipment, and the edge computing equipment automatically updates and operates.
The invention has the advantages that:
1. according to the deep learning algorithm iteration method based on Yun Bian cooperation, different image processing algorithms can be rapidly deployed to the terminal or the terminal cluster by utilizing the convenience of cloud-edge cooperation, and a large amount of manpower and material resources are saved.
2. A Yun Bian collaboration-based deep learning algorithm iteration method utilizes abundant computing and storage resources of a cloud platform, adopts a convenient and quick online data labeling and auditing mode, and realizes efficient iteration upgrading of a deep learning algorithm model on the basis of an incremental stable image algorithm iteration model.
3. According to the deep learning algorithm iteration method based on Yun Bian cooperation, all edge computing devices only store partial data collected locally, global data are encrypted and stored in a cloud computing platform, and a good basic environment is provided for data safety protection.
The invention provides a deep learning algorithm iteration method based on Yun Bian cooperation. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the core concepts of the invention. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
Claims (10)
1. A Yun Bian cooperation-based deep learning algorithm iteration method, which is characterized by comprising the following steps of:
s100: the method comprises the steps that edge computing equipment is deployed to an industrial site, connected with a vision sensor and communicated with a cloud platform, and image data transmitted by the vision sensor and task requirements of the site are obtained; the cloud platform defines a method for processing image data, and the vision sensor defines a calling method of a corresponding image data processing method; obtaining a corresponding image processing algorithm configuration request according to the image data transmitted by the visual sensor and the task requirements of the site, and sending the image processing algorithm configuration request to a cloud platform;
s200: after receiving the image processing algorithm configuration request, the cloud platform acquires a corresponding image processing algorithm package from an algorithm warehouse and transmits the corresponding image processing algorithm package to corresponding edge computing equipment; wherein the image processing algorithm package comprises an algorithm model;
s300: the edge computing equipment pre-processes the image data to obtain pre-processed image data, runs the image processing algorithm package to obtain running conditions and processing results, and sends the running conditions, the processing results and the pre-processed image data to the cloud platform;
s400: the cloud platform analyzes the running condition and the processing result, judges whether the processing result of the corresponding edge computing device meets the expected requirement, if so, returns to the step S300 to process the next image data, and if not, enters the next step;
s500: the cloud platform takes out data with operation results not reaching expected requirements from the preprocessed image data, completes online data marking and auditing, automatically carries out algorithm model training according to new marking auditing data and historical data, generates a new algorithm model, carries out test verification to obtain an optimal model, and updates the image processing algorithm package;
s600: and the cloud platform transmits the updated image processing algorithm package to corresponding edge computing equipment, and the edge computing equipment automatically updates and operates.
2. The method of claim 1, wherein establishing communication between the edge computing device and the cloud platform in step S100 includes establishing communication between a single edge computing device and the cloud platform, and establishing communication between an edge cluster and the cloud platform by a plurality of edge computing devices.
3. The method of claim 2, wherein the edge computing device connecting with the vision sensor and establishing communication with the cloud platform in step S100 comprises:
the edge computing device can establish data connection with the visual sensor through an RJ-45 Ethernet interface and USB3.2gen1;
the edge computing device can independently communicate with the cloud platform through any one of WI-FI, ethernet and 4G, and upload running conditions, processing results and preprocessed image data.
4. The method as recited in claim 2, further comprising:
the edge computing equipment on the same site can establish an edge cluster through an Internet of things gateway by utilizing an RS232 or RS484 serial port, and the edge computing equipment communicates with the cloud platform to upload data of the edge computing equipment in the edge cluster.
5. The method of any one of claims 3 or 4, wherein the edge computing device comprises a first communication service module, an algorithm configuration and update module, an algorithm run module, and a system management module, the first communication service module connecting the algorithm configuration and update module, the algorithm configuration and update module connecting the algorithm run module, the algorithm run module connecting the system management module, the system management module connecting the first communication service module;
the first communication service module is used for receiving and transmitting data interacted by the cloud platform and data interaction service of other internet entities;
the algorithm configuration and updating module is used for downloading a corresponding image processing algorithm package from the cloud platform according to the field use requirement, configuring according to the field actual condition, and automatically updating the algorithm package after iteration of the cloud platform after the configuration is completed;
the algorithm running module is used for carrying out image processing running in real time according to the algorithm model in the algorithm package and interacting with the terminal equipment;
the system management module is used for monitoring and managing the running condition, the processing process and the result, selectively storing the running data generated in the running process in a local place according to the preset requirement, encrypting and storing the global data generated in the processing process and uploading the global data to the cloud platform.
6. The method of claim 5, wherein the image processing algorithm package comprises a defect detection image processing algorithm, a sizing image processing algorithm, an object localization image processing algorithm, and an OCR recognition image processing algorithm.
7. The method of claim 6, wherein the cloud platform comprises a second communication service module, a device management module, a data warehouse module, an algorithm warehouse module, a data labeling and auditing module, and a model iteration and evaluation module all connected to the second communication service module, the data warehouse module connected to the data labeling and auditing module, the data labeling and auditing module connected to the model iteration and evaluation module, the model iteration and evaluation module connected to the algorithm warehouse module;
the device management module is used for analyzing the operation data uploaded by the edge computing device, judging whether the image processing result reaches an expected target or not, and displaying the operation condition of the online device in a centralized manner in an instrument panel;
the data warehouse module is used for storing all historical software versions and running logs uploaded by the edge computing equipment, backing up backtracking, and storing the preprocessed image data uploaded by the edge computing equipment in a classified manner according to the difference of the running image processing algorithm packages of the edge computing equipment;
the algorithm warehouse module is used for storing algorithm packages of different application types and historical versions thereof, wherein the algorithm packages comprise all necessary files for running corresponding applications, when the edge computing equipment initiates an image processing algorithm configuration request, the corresponding algorithm packages are extracted according to the image processing algorithm configuration request and sent to the corresponding edge computing equipment, meanwhile, an algorithm model after each training iteration is stored in the algorithm warehouse, and the algorithm warehouse is used for issuing an iteration model;
the data marking and auditing function module is used for providing an online data standard platform, when the processing result does not reach the expectation, taking out the data with the non-ideal operation result from the data warehouse, and pushing the data marking and auditing task at the data marking and auditing platform so as to enable a back-end maintainer to finish marking and auditing of the data online;
the model iteration and evaluation module is used for optimizing an image processing algorithm task according to requirements, automatically carrying out algorithm model training and evaluation based on new annotation auditing data and historical data, and submitting the iterative algorithm model to the algorithm warehouse;
the second communication service module is responsible for data transceiving of the edge computing device and data interaction services with other internet entities.
8. The method according to claim 7, wherein the preprocessing operation performed on the image data by the edge computing device in step S300 is a preprocessing rule of data preset for different image processing tasks, wherein the preprocessing rule of data can be updated remotely through the cloud platform.
9. The method according to claim 8, wherein the preset data preprocessing rules comprise image pre-segmentation, image data de-duplication and data cleaning, wherein the image pre-segmentation is performed on image data uploaded by a visual sensor in advance, a part of background and areas which do not need to be detected are cut off, and areas which need to be detected by an image algorithm are reserved; the image data de-duplication is the image data repeated by automatic shielding, and the uploading processing is not carried out on the image data; and the data cleaning is to clean repeated operation conditions, non-operation image processing schemes, processing results output by invalid equipment and invalid image data in the data which is required to be uploaded by the edge computing equipment.
10. The method according to claim 9, wherein the operation status and the processing result uploaded by the edge computing device in step S300 are data uploaded in real time, and the preprocessed image data is required to be uploaded when the image detection system is not operated; the image detection system is a program system for calling an algorithm package issued by the cloud platform to process images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310864736.2A CN116578323A (en) | 2023-07-14 | 2023-07-14 | Deep learning algorithm iteration method based on Yun Bian cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310864736.2A CN116578323A (en) | 2023-07-14 | 2023-07-14 | Deep learning algorithm iteration method based on Yun Bian cooperation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116578323A true CN116578323A (en) | 2023-08-11 |
Family
ID=87541729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310864736.2A Pending CN116578323A (en) | 2023-07-14 | 2023-07-14 | Deep learning algorithm iteration method based on Yun Bian cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116578323A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784026A (en) * | 2020-05-28 | 2020-10-16 | 国网信通亿力科技有限责任公司 | Cloud-side cooperative sensing-based all-dimensional physical examination system for electrical equipment of transformer substation |
| CN113392760A (en) * | 2021-06-15 | 2021-09-14 | 中国民航机场建设集团有限公司 | Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction |
| CN113596158A (en) * | 2021-07-29 | 2021-11-02 | 杭州海康威视系统技术有限公司 | Scene-based algorithm configuration method and device |
| CN113918423A (en) * | 2021-10-21 | 2022-01-11 | 城云科技（中国）有限公司 | Cloud platform monitoring method and device and application thereof |
| CN113962660A (en) * | 2021-10-26 | 2022-01-21 | 中国民航机场建设集团有限公司 | AI technology-based construction worker behavior safety management correction method and system |
| CN114205141A (en) * | 2021-12-09 | 2022-03-18 | 重庆紫光华山智安科技有限公司 | IPC algorithm deployment admission method, system, medium and electronic terminal |
| CN114677624A (en) * | 2022-03-18 | 2022-06-28 | 南京农业大学 | Sow parturition intelligent monitoring system based on cloud edge synergy |
-
2023
- 2023-07-14 CN CN202310864736.2A patent/CN116578323A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784026A (en) * | 2020-05-28 | 2020-10-16 | 国网信通亿力科技有限责任公司 | Cloud-side cooperative sensing-based all-dimensional physical examination system for electrical equipment of transformer substation |
| CN113392760A (en) * | 2021-06-15 | 2021-09-14 | 中国民航机场建设集团有限公司 | Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction |
| CN113596158A (en) * | 2021-07-29 | 2021-11-02 | 杭州海康威视系统技术有限公司 | Scene-based algorithm configuration method and device |
| CN113918423A (en) * | 2021-10-21 | 2022-01-11 | 城云科技（中国）有限公司 | Cloud platform monitoring method and device and application thereof |
| CN113962660A (en) * | 2021-10-26 | 2022-01-21 | 中国民航机场建设集团有限公司 | AI technology-based construction worker behavior safety management correction method and system |
| CN114205141A (en) * | 2021-12-09 | 2022-03-18 | 重庆紫光华山智安科技有限公司 | IPC algorithm deployment admission method, system, medium and electronic terminal |
| CN114677624A (en) * | 2022-03-18 | 2022-06-28 | 南京农业大学 | Sow parturition intelligent monitoring system based on cloud edge synergy |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107179324B (en) | The methods, devices and systems of testing product packaging | |
| US20230013544A1 (en) | Method, Apparatus and System for Detecting Abnormal Operating States of a Device | |
| EP3106869A1 (en) | Systems and methods for non-destructive testing involving remotely located expert | |
| CN107817787A (en) | A kind of intelligent producing line robotic failure diagnostic method based on machine learning | |
| CN111601361B (en) | Method and device for detecting key nodes of Ad hoc network in real time | |
| EP1958034B1 (en) | Use of sequential clustering for instance selection in machine condition monitoring | |
| CN103746829A (en) | Cluster-based fault perception system and method thereof | |
| CN112416369B (en) | Intelligent deployment method oriented to heterogeneous mixed environment | |
| CN114297935A (en) | Airport terminal building departure optimization operation simulation system and method based on digital twin | |
| CN113312957A (en) | off-Shift identification method, device, equipment and storage medium based on video image | |
| CN116168164B (en) | Digital twin visualization method based on robot industrial chain | |
| CN116940956A (en) | Industrial Internet of things, method and medium for equipment function decline type fault early warning | |
| CN113568900A (en) | Big data cleaning method based on artificial intelligence and cloud server | |
| CN113168528A (en) | Continuous learning image stream processing system | |
| CN115169602A (en) | Maintenance method and device for power equipment, storage medium and computer equipment | |
| CN115409992A (en) | Remote driving patrol car system | |
| CN116578323A (en) | Deep learning algorithm iteration method based on Yun Bian cooperation | |
| CN117132890A (en) | Remote sensing image target detection method and system based on Kubernetes edge computing cluster | |
| CN115563069B (en) | Data sharing processing method and system based on artificial intelligence and cloud platform | |
| CN114550107B (en) | Bridge linkage intelligent inspection method and system based on unmanned aerial vehicle cluster and cloud platform | |
| CN100395759C (en) | Computer assistant checking system based on CAD platform and method thereof | |
| CN113420646B (en) | Lock station connection lock detection system and method based on deep learning | |
| Maloof et al. | Learning to detect rooftops in aerial images | |
| CN114359670A (en) | Unstructured data labeling method and device, computer equipment and storage medium | |
| CN113703394A (en) | Cutter monitoring and managing method and system based on edge calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230811 |