CN110705684A - Environment self-adaptive learning method and system based on end cloud cooperation - Google Patents
Environment self-adaptive learning method and system based on end cloud cooperation Download PDFInfo
- Publication number
- CN110705684A CN110705684A CN201910777631.7A CN201910777631A CN110705684A CN 110705684 A CN110705684 A CN 110705684A CN 201910777631 A CN201910777631 A CN 201910777631A CN 110705684 A CN110705684 A CN 110705684A
- Authority
- CN
- China
- Prior art keywords
- model
- cloud
- environment
- data
- adaptive learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012549 training Methods 0.000 claims abstract description 21
- 230000003044 adaptive effect Effects 0.000 claims abstract description 18
- 230000007613 environmental effect Effects 0.000 claims abstract description 18
- 230000006835 compression Effects 0.000 claims description 16
- 238000007906 compression Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 12
- 238000013138 pruning Methods 0.000 claims description 10
- 238000012512 characterization method Methods 0.000 claims description 5
- 238000003745 diagnosis Methods 0.000 claims description 5
- 238000011158 quantitative evaluation Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 description 10
- 230000004044 response Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 7
- 238000005265 energy consumption Methods 0.000 description 7
- 238000007667 floating Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000005520 cutting process Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 239000002699 waste material Substances 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000009133 cooperative interaction Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an environment self-adaptive learning method and system based on end cloud cooperation, which comprises the following steps: step 1, an object end collects environmental data, and judges whether the environmental data can be analyzed or not by using a local identification model of the object end, if so, an identification result of the environmental data is obtained and output, otherwise, step 2 is executed; and 2, uploading the environment data to a cloud end, identifying the environment data by using a cloud end high-precision model, selecting an object identification model of a specific environment according to an identification result, assisting the local identification model to perform self-adaptive learning by using the identification result and original training data of the specific object identification model to update the local identification model to obtain an updated model, replacing the local identification model of the object end, and executing the step 1 again. According to the cloud-side model self-adaptive learning method, the self-adaptive learning of the cloud-side model is realized, the resource consumption of the model in operation at the object side is reduced in a terminal cloud cooperation mode, and the identification precision and the adaptive capacity of the model to a specific computing environment are improved.
Description
Technical Field
The invention relates to the field of end cloud cooperative computing, intelligent object end perception and environment self-adaptive learning frameworks, in particular to an environment self-adaptive learning framework based on end cloud cooperation and a system thereof.
Background
With the increasing number of intelligent terminals accessing the internet, the data scale also increases explosively. The traditional perception computing system mostly adopts a cloud computing mode, and mass monitoring data are transmitted to a cloud computing to increase computing loads of network transmission and the cloud, so that the serious problems of energy consumption waste, response delay, privacy leakage and the like are caused. The computing mode cannot meet the requirements of mass data large-scale transmission and computation and application outbreak in the coming artificial intelligence era. Meanwhile, the existing cloud-based trained model has certain universality, but is usually low in precision and poor in adaptability in a specific complex computing environment. Edge computing aims to migrate the computing tasks in the cloud to the edge or device as much as possible, allowing the computation to occur in the closest proximity to the data source and the user. However, the software and hardware of the device and the communication system are abnormal and heterogeneous, and a unified and effective computing platform cannot be provided. Meanwhile, a large number of resource-limited devices exist at the edge, and are not suitable for bearing all the computing load from the cloud. Therefore, future AI systems require some end-cloud coordinated architecture and system to address the above challenges.
Current AI application functions, such as voice recognition, online translation, recommendation systems, etc., are deployed in the cloud. The current trend is to try to migrate some functions to the edge or device side, thereby improving security and privacy safeguards and reducing response latency. On the other hand, AI systems deployed on the edge or end, such as autopilots, drones, and home robots, are also trying to share data and update models with cloud-rich computing resources. Therefore, the next development of AI systems will be to span devices and clouds to accomplish some challenging tasks more efficiently and accurately in a coordinated fashion.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an environment self-adaption method and system of end-cloud cooperation based on an edge computing concept, and the problems of application delay and network bandwidth pressure caused by the existing cloud learning and prediction technology and the problems that a prediction model on an edge end cannot evolve and be updated are overcome.
Aiming at the defects of the prior art, the invention provides an environment self-adaptive learning method based on end cloud cooperation, which comprises the following steps:
step 1, an object end collects environmental data, and judges whether the environmental data can be analyzed or not by using a local identification model of the object end, if so, an identification result of the environmental data is obtained and output, otherwise, step 2 is executed;
and 2, uploading the environment data to a cloud end, identifying the environment data by using a cloud end high-precision model, selecting an object identification model of a specific environment according to an identification result, assisting the local identification model to perform self-adaptive learning by using the identification result and original training data of the specific object identification model to update the local identification model to obtain an updated model, replacing the local identification model of the object end, and executing the step 1 again.
The environment self-adaptive learning method based on end cloud cooperation, wherein the step 2 includes compressing the update model and then relocating the update model on the object end, wherein the compressing specifically includes:
and step 21, scoring the convolution kernels of each output channel of the updated model convolution layer according to CNN network characterization diagnosis and quantitative evaluation, setting the weight parameters and bias parameters of the convolution kernels lower than a preset score to be 0, obtaining a pruning model by only storing the structure and weight parameter values of the convolution kernels with values other than 0 in the updated model, retraining the pruning model by using training data to obtain a compression model, and replacing the local identification model of the object end with the compression model.
The environment self-adaptive learning method based on the cooperation of the end cloud comprises the following steps of 1: the local identification model adopts 8-bit fixed point number during operation, adopts nonlinear coding during operation, reduces the distance between digits in a digit set place to increase precision, and reduces the memory overhead by making the digit distance larger in a digit sparse place.
The environment self-adaptive learning method based on the cooperation of the end cloud comprises the following steps: the system comprises a target detection model, a target tracking model, an object recognition model and a face recognition model.
The environment self-adaptive learning method based on the terminal cloud cooperation is characterized in that the object terminal comprises intelligent wearable equipment, Internet of things equipment and a gateway.
The invention also provides an environment self-adaptive learning system based on end cloud cooperation, which comprises the following steps:
the module 1, the object terminal collects the environmental data, and judges whether the environmental data can be analyzed by using the local recognition model of the object terminal, if yes, the recognition result of the environmental data is obtained and output, otherwise, the module 2 is executed;
the module 2 uploads the environment data to a cloud end, the environment data is identified by using a cloud end high-precision model, an object identification model of a specific environment is selected according to an identification result, the local identification model is assisted to perform self-adaptive learning by using the identification result and original training data of the specific object identification model so as to update the local identification model, an updated model is obtained, the local identification model of the object end is replaced, and the module 1 is executed again.
The environment self-adaptive learning system based on the end cloud cooperation, wherein the module 2 compresses the update model and then deploys the update model on the object end, and the compression specifically comprises:
the module 21 is used for scoring the convolution kernels of all output channels of the updated model convolution layer according to CNN network characterization diagnosis and quantitative evaluation, setting the weight parameters and bias parameters of the convolution kernels lower than the preset score to be 0, obtaining a pruning model by only storing the structure and the weight parameter values of the convolution kernels with values other than 0 in the updated model, retraining the pruning model by using training data to obtain a compression model, and replacing the local identification model of the object end with the compression model.
The environment self-adaptive learning system based on end cloud cooperation, wherein the module 1 comprises: the local identification model adopts 8-bit fixed point number during operation, adopts nonlinear coding during operation, reduces the distance between digits in a digit set place to increase precision, and reduces the memory overhead by making the digit distance larger in a digit sparse place.
The environment self-adaptive learning system based on the end cloud cooperation comprises the following local identification models: the system comprises a target detection model, a target tracking model, an object recognition model and a face recognition model.
The environment self-adaptive learning system based on terminal cloud cooperation is characterized in that the object terminal comprises intelligent wearable equipment, Internet of things equipment and a gateway.
According to the scheme, the invention has the advantages that: the cloud model self-adaptive learning method and the cloud model self-adaptive learning system realize the self-adaptive learning of the cloud model, reduce the memory, the calculated amount and the power consumption required by the model when the object terminal runs in a terminal cloud cooperation mode, and greatly improve the identification precision and the adaptive capacity of the model aiming at a specific computing environment.
Drawings
FIG. 1 is a diagram of an environment adaptive learning technology architecture with end cloud collaboration;
FIG. 2 is a schematic diagram of a neural network-based adaptive learning principle at the cloud end;
FIG. 3 is a schematic diagram of a depth model parser optimization based on Caffe;
FIG. 4 is a diagram of an environment adaptive learning framework software architecture in coordination with cloud at the end;
FIG. 5 is a system architecture diagram of an adaptive learning technique employing end cloud coordination.
Detailed Description
The invention mainly comprises 3 parts: 1) the cloud-end on-line environment self-adaptive learning module uses a machine learning algorithm based on a neural network, so that a cloud-end model can perform self-adaptive learning, and the environment adaptability and the target identification accuracy of the model are improved; 2) the lightweight depth model analysis technology adopts cutting to support floating point numbers, designs a new data structure and other methods to optimize the analyzer, and provides stable and efficient operation support for the operation of the model. 3) An environment self-adaptive learning framework with cooperation of a cloud and a terminal solves the problems of application delay and network bandwidth pressure of the existing cloud learning and prediction technology and the problem that a prediction model on an edge terminal cannot evolve and be updated by providing a universal model deployment and processing interface and supporting incremental learning and compression APIs (application programming interfaces) of various deep learning algorithms at the cloud terminal. The method mainly solves the problems of energy consumption waste, response delay, transmission load, privacy disclosure and the like in the traditional cloud learning, and the problems of limited resources of edge equipment and the like. Through the end cloud cooperation mode, the identification precision and the adaptive capacity of the model aiming at a specific computing environment are greatly improved, and the incremental adaptive learning effect is realized on the framework.
In order to make the aforementioned features and effects of the present invention more comprehensible, embodiments accompanying figures are described in detail below, wherein "on" means "on" unless otherwise noted.
The invention provides an environment self-adaptive technology of end cloud cooperation based on an edge computing concept aiming at complex sensing and computing environments, and solves the problems of energy consumption waste, response delay, privacy disclosure, limited object end resources and the like of the traditional sensing and computing system. According to the method, the identification precision and the adaptability of the model to a specific computing environment are greatly improved in a terminal cloud cooperation mode; and response delay and accuracy of the model in an object-side resource-limited environment are improved by researching an intelligent application operation technology with high efficiency and low power consumption.
As shown in fig. 1, the invention adopts the idea of end-cloud cooperation, fully exerts the respective advantages of the cloud end and the edge end, and realizes an environment self-adaptive technology of end-cloud cooperation based on the edge computing concept. The method utilizes the strong computing power of the cloud to train, update and compress the model; by using the characteristic that an intelligent terminal (edge end) is close to a data source, prediction and analysis are carried out on the terminal, and system time delay and energy consumption are reduced. Meanwhile, the terminal collects and preprocesses field data and transmits the field data to the cloud end to perform online incremental updating on the model, and the precision and the adaptability of the model are continuously improved. On the basic theory level, an online incremental learning model based on a neural network and a corresponding model compression cutting method are researched; on the intelligent terminal, performing coarse grain analysis on field data on a research end; on the system architecture level, the online learning mechanism and framework of the light-weight model runtime environment and the end cloud cooperation are researched. And finally, constructing an online incremental environment self-adaptive learning system based on the key theory, method and framework.
1) Cloud online 'incremental' learning algorithm module (cloud online environment adaptive learning module)
Due to the diversity and complexity of the real environment, the environment data (such as acceleration data, temperature and humidity data, distance data, picture data and the like) acquired at the object end are huge and complex, and the machine learning model obtained by training according to the initial labeled sample can not meet the real requirement along with the passage of time. If all the object end data are uploaded to the cloud end, manual marking is carried out, and then the model is learned again, so that a large amount of time, space and data transmission cost are needed. New methods are therefore needed to enable adaptive learning of models.
According to the invention, aiming at huge data volume in the object end environment, a machine learning method is adopted, so that massive movement of data between end clouds is avoided, the data transmission cost is effectively reduced, and the acquisition efficiency of effective data is improved. Aiming at the diversity and complexity of data in the object-side environment, a teacher-student network is adopted, so that the manual intervention cost is reduced, and the model is assisted to learn a new class in an incremental manner.
When the data volume of the picture which is not identified on the detection site reaches a set threshold, the object-side equipment automatically uploads the picture to the cloud, the cloud high-precision model (such as ResNet, GoogleNet, SENet and the like) identifies an unknown target, and an object identification model of a specific environment is selected according to the identification result. The specific environment refers to an application environment of the system. If the system can be applied to animal identification and detection in the area with rare human smoke, the environment and the animal category in different areas are obviously different. Aiming at different environments, the targeted training recognition model can realize high-precision recognition by using a simpler model, reduces the complexity and training time of the model, and is favorable for the deployment and model updating of the model on different edge devices. For such an application, the "specific environment" refers to an application environment for animal recognition and detection, such as a desert, a sea, and the like. The selection of the specific environment is based on the project application environment of the user. The specific meaning here is that the model trained and compressed by the invention is only for the specific application environment to which the model is to be directed, and the model has the characteristics of high precision and low memory occupation. In the case of changing one environment, the incremental training is required to be performed again according to the new specific environment data.
And according to the output of the high-precision model and the original training data of the specific object recognition model, a teacher-student network and other machine learning methods are adopted to assist the object recognition model in self-adaptive learning. And the updated object recognition model is compressed and then is redeployed on the object-side equipment. The module can continuously update the detection model (object recognition model) on the edge intelligent terminal, and improves the adaptability of the model to the environment and the accuracy of target recognition. The specific flow is shown in fig. 2. Assuming that the system provided by the invention is applied to animal identification detection in a sparsely populated area, the identification model deployed at the terminal is trained by data of an initial specific environment (such as the ocean, etc.), and the animal type identified by the model is limited. With the lapse of time, some objects which cannot be identified by the initial identification model can appear in the images collected by the monitoring camera, and when the number of the pictures which cannot be identified and collected by the terminal reaches a set threshold value, the pictures are uploaded to the cloud. After the cloud receives the pictures, the high-precision model identifies the same picture and the pictures after conversion by adopting a data distillation method, and the identification results are summarized into a label as the label of the picture. And inputting the image data obtained by the method and the initial image data into an animal recognition model for retraining, and adaptively learning new object recognition by the animal recognition model.
2) Lightweight depth model analysis technique
The edge intelligent terminal equipment has limited computing capacity, and the larger the computation amount required by the depth model is, the longer the time required by running at the mobile terminal is. In addition, each operation of the depth model needs to consume electric energy, and the larger the model operation amount is, the larger the electric energy is needed for completing the inference once. In the marginal intelligent terminal equipment with limited electric quantity, the electric energy consumed by the model in one-time reference must be finely calculated, so the calculation amount of the deep learning model cannot be too large.
The floating-point number is represented in a binary form in a computer system, the traditional full-precision 32-bit floating-point number can cover a very large number range, but occupies a large amount of memory, and meanwhile, the hardware resource overhead is large during operation. In fact, the high precision can not be used in the deep learning operation, so the simplest and direct method is to reduce the precision and convert the original 32-bit floating point number into a 16-bit floating point number or even an 8-bit fixed point number. On one hand, the storage space required by the model can be greatly reduced by reducing the bit length of the data (the 1KB can store 256 32-bit floating point numbers but can store 1024 8-bit floating point numbers), and on the other hand, the hardware implementation of the low-precision arithmetic unit is simpler and can run faster. Along with the reduction of data precision, the model accuracy rate also reduces, and a plurality of optimization strategies are generated, such as optimization coding (the original fixed point number is equal to the distance between linear coding numbers, but nonlinear coding can be used for reducing the distance between the numbers in a place with a digital set and increasing the precision, and the nonlinear coding can be used for increasing the distance between the numbers in a place with a sparse number.
On the premise of not influencing the execution progress of the model, the invention optimizes the analyzer by cutting methods such as supporting floating point numbers, designing a new data structure and the like, optimizes the calculated amount of the model and the execution efficiency thereof, and enables the model to run on intelligent terminal equipment with limited resources, thereby providing a unified and cloud model training and execution environment for cooperative environment adaptive amount learning.
3) Environment self-adaptive learning framework with cooperation of end cloud
The transmission and centralized processing of a large amount of data bring serious problems such as network bottleneck and instantaneous computing load pressure to the learning and prediction technology which mainly adopts the cloud, and the response delay and the energy consumption of the system are increased. The end-based prediction technology has weak self-adaptive capability of the model, and the cloud-trained pervasive model cannot adapt to a specific complex sensing and computing environment, so that the precision of the model is greatly reduced, and the problem can be effectively solved by the distributed software architecture of end-cloud intelligent collaborative interaction, as shown in fig. 4. The invention discloses an edge computing-based end cloud collaborative environment self-adaptive learning system, which is based on the core idea of an edge computing-based end cloud collaborative architecture, and is characterized in that a cloud end 'incremental' learning module, an end data preprocessing module and an end lightweight runtime environment module are researched and developed, an HTTP-based end cloud unidentified sample uploading and model distribution module is designed, an edge intelligent terminal prediction model is constructed to be lightweight, the cloud end learning model has a distributed software architecture with 'incremental' learning capability, and the end cloud collaborative environment self-adaptive learning system is constructed:
(1) providing a universal model deployment and processing interface;
(2) support various deep learning algorithms and can 'increment formula' study and compression API in the cloud.
The method can be used for constructing a real-time and efficient target detection and identification system.
As shown in fig. 3, the distributed software architecture of the end cloud intelligent collaborative interaction is based on an open source deep learning framework Caffe/TensorFlow, and a Python collaborative module is used for end cloud data interaction. The method comprises the steps that a local edge server deploys a universal model for cloud training to participating intelligent terminals, the intelligent terminals receive the model and upload unknown data to the cloud to generate training samples to perform classification, and then incremental training is performed. The cloud updates the original pervasive model through category matching, and then deploys the model to the client again for repeated circulation so as to improve the environment self-adaption degree of the AI model.
According to the method, the field self-adaptive updating of the end cloud learning model is dynamically carried out by adopting an end cloud collaborative interaction mode, the repeated training of similar domain image data is avoided, the data storage space is greatly reduced, the calculation overhead is reduced, and the efficiency is improved. Meanwhile, the whole system architecture with the cooperation of the end cloud can effectively reduce the data leakage risk caused by uploading a large amount of key data to the cloud, and effectively ensures the storage and transmission safety of the data.
As shown in fig. 5, the invention can utilize a server carrying a depth model accelerator card, combine with an open source depth learning framework Caffe or tensrflow, train a generalized model at the cloud, and deploy the model after cutting and compressing the model to an edge intelligent terminal. And carrying out preprocessing on the complex and changeable scene data acquired by the intelligent terminal, and then carrying out object identification on the edge intelligent terminal. And when the unrecognizable picture data volume of the edge intelligent terminal reaches a set threshold value, the edge intelligent terminal uploads the picture data volume to the cloud end, and the object recognition model is updated according to the received real scene data. And the cloud end cuts and compresses the updated model and then sends the model to the edge intelligent terminal again, and the next round of identification is carried out in a circulating and reciprocating manner, so that the dynamic adjustment of object identification and the intelligent improvement of identification accuracy are realized.
Due to the problems of response delay, energy consumption waste and the like in the traditional cloud learning, the problem of limited resources and the like in the edge equipment exists. Therefore, the invention provides an environment self-adaptive technology based on end cloud cooperation, the method mainly comprises 4 parts which respectively perform their own functions and integrate the advantages of the cloud and the end, thereby realizing the identification precision and the adaptive environment of the model to the specific computing environment. The method provides a universal model deployment and processing interface, supports incremental learning and compression API of various deep learning algorithms at the cloud, can be applied to complex high-precision models such as target detection and target identification, has a wide application range, can effectively reduce energy consumption of model operation, improves the response speed of the model, and improves the identification precision and adaptability of the model to a specific computing environment.
The following are system examples corresponding to the above method examples, and this embodiment can be implemented in cooperation with the above embodiments. The related technical details mentioned in the above embodiments are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the above-described embodiments.
The invention also provides an environment self-adaptive learning system based on end cloud cooperation, which comprises the following steps:
the module 1, the object terminal collects the environmental data, and judges whether the environmental data can be analyzed by using the local recognition model of the object terminal, if yes, the recognition result of the environmental data is obtained and output, otherwise, the module 2 is executed;
the module 2 uploads the environment data to a cloud end, the environment data is identified by using a cloud end high-precision model, an object identification model of a specific environment is selected according to an identification result, the local identification model is assisted to perform self-adaptive learning by using the identification result and original training data of the specific object identification model so as to update the local identification model, an updated model is obtained, the local identification model of the object end is replaced, and the module 1 is executed again.
The environment self-adaptive learning system based on the end cloud cooperation, wherein the module 2 compresses the update model and then deploys the update model on the object end, and the compression specifically comprises:
the module 21 is used for scoring the convolution kernels of all output channels of the updated model convolution layer according to CNN network characterization diagnosis and quantitative evaluation, setting the weight parameters and bias parameters of the convolution kernels lower than the preset score to be 0, obtaining a pruning model by only storing the structure and the weight parameter values of the convolution kernels with values other than 0 in the updated model, retraining the pruning model by using training data to obtain a compression model, and replacing the local identification model of the object end with the compression model.
The environment self-adaptive learning system based on end cloud cooperation, wherein the module 1 comprises: the local identification model adopts 8-bit fixed point number during operation, adopts nonlinear coding during operation, reduces the distance between digits in a digit set place to increase precision, and reduces the memory overhead by making the digit distance larger in a digit sparse place.
The environment self-adaptive learning system based on the end cloud cooperation comprises the following local identification models: the system comprises a target detection model, a target tracking model, an object recognition model and a face recognition model.
The environment self-adaptive learning system based on terminal cloud cooperation is characterized in that the object terminal comprises intelligent wearable equipment, Internet of things equipment and a gateway.
Claims (10)
1. An environment self-adaptive learning method based on end cloud cooperation is characterized by comprising the following steps:
step 1, an object end collects environmental data, and judges whether the environmental data can be analyzed or not by using a local identification model of the object end, if so, an identification result of the environmental data is obtained and output, otherwise, step 2 is executed;
and 2, uploading the environment data to a cloud end, identifying the environment data by using a cloud end high-precision model, selecting an object identification model of a specific environment according to an identification result, assisting the local identification model to perform self-adaptive learning by using the identification result and original training data of the specific object identification model to update the local identification model to obtain an updated model, replacing the local identification model of the object end, and executing the step 1 again.
2. The method as claimed in claim 1, wherein the step 2 includes compressing the updated model and then relocating the model to the object, wherein the compressing specifically includes:
and step 21, scoring the convolution kernels of each output channel of the updated model convolution layer according to CNN network characterization diagnosis and quantitative evaluation, setting the weight parameters and bias parameters of the convolution kernels lower than a preset score to be 0, obtaining a pruning model by only storing the structure and weight parameter values of the convolution kernels with values other than 0 in the updated model, retraining the pruning model by using training data to obtain a compression model, and replacing the local identification model of the object end with the compression model.
3. The method for adaptive learning based on environment of end cloud coordination according to claim 1 or 2, wherein the step 1 comprises: the local identification model adopts 8-bit fixed point number during operation, adopts nonlinear coding during operation, reduces the distance between digits in a digit set place to increase precision, and reduces the memory overhead by making the digit distance larger in a digit sparse place.
4. The method as claimed in claim 3, wherein the local recognition model comprises: the system comprises a target detection model, a target tracking model, an object recognition model and a face recognition model.
5. The environment adaptive learning method based on peer-cloud collaboration as claimed in claim 3, wherein the object peer comprises a smart wearable device, an internet of things device and a gateway.
6. An environment self-adaptive learning system based on end cloud cooperation is characterized by comprising:
the module 1, the object terminal collects the environmental data, and judges whether the environmental data can be analyzed by using the local recognition model of the object terminal, if yes, the recognition result of the environmental data is obtained and output, otherwise, the module 2 is executed;
the module 2 uploads the environment data to a cloud end, the environment data is identified by using a cloud end high-precision model, an object identification model of a specific environment is selected according to an identification result, the local identification model is assisted to perform self-adaptive learning by using the identification result and original training data of the specific object identification model so as to update the local identification model, an updated model is obtained, the local identification model of the object end is replaced, and the module 1 is executed again.
7. The adaptive environment learning system according to claim 6, wherein the module 2 comprises compressing the updated model and then re-deploying the compressed updated model on the object, wherein the compressing specifically comprises:
the module 21 is used for scoring the convolution kernels of all output channels of the updated model convolution layer according to CNN network characterization diagnosis and quantitative evaluation, setting the weight parameters and bias parameters of the convolution kernels lower than the preset score to be 0, obtaining a pruning model by only storing the structure and the weight parameter values of the convolution kernels with values other than 0 in the updated model, retraining the pruning model by using training data to obtain a compression model, and replacing the local identification model of the object end with the compression model.
8. The environment adaptive learning system based on peer-cloud collaboration as claimed in claim 6 or 7, wherein the module 1 comprises: the local identification model adopts 8-bit fixed point number during operation, adopts nonlinear coding during operation, reduces the distance between digits in a digit set place to increase precision, and reduces the memory overhead by making the digit distance larger in a digit sparse place.
9. The environment adaptive learning system based on peer-cloud collaboration as claimed in claim 8, wherein the local recognition model comprises: the system comprises a target detection model, a target tracking model, an object recognition model and a face recognition model.
10. The peer-cloud collaboration based environment adaptive learning system as claimed in claim 8, wherein the object side comprises a smart wearable device, an internet of things device and a gateway.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910777631.7A CN110705684A (en) | 2019-08-22 | 2019-08-22 | Environment self-adaptive learning method and system based on end cloud cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910777631.7A CN110705684A (en) | 2019-08-22 | 2019-08-22 | Environment self-adaptive learning method and system based on end cloud cooperation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110705684A true CN110705684A (en) | 2020-01-17 |
Family
ID=69193694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910777631.7A Pending CN110705684A (en) | 2019-08-22 | 2019-08-22 | Environment self-adaptive learning method and system based on end cloud cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110705684A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461694A (en) * | 2020-03-17 | 2020-07-28 | 上海大学 | Dish identification and pricing system and method based on multi-level learning model |
| CN111625361A (en) * | 2020-05-26 | 2020-09-04 | 华东师范大学 | Joint learning framework based on cooperation of cloud server and IoT (Internet of things) equipment |
| CN111738435A (en) * | 2020-06-22 | 2020-10-02 | 上海交通大学 | Online sparse training method and system based on mobile equipment |
| CN111783674A (en) * | 2020-07-02 | 2020-10-16 | 厦门市美亚柏科信息股份有限公司 | Face recognition method and system based on AR glasses |
| CN111985650A (en) * | 2020-07-10 | 2020-11-24 | 华中科技大学 | Activity recognition model and system considering both universality and individuation |
| CN112115975A (en) * | 2020-08-18 | 2020-12-22 | 山东信通电子股份有限公司 | Deep learning network model fast iterative training method and equipment suitable for monitoring device |
| CN112561097A (en) * | 2020-12-23 | 2021-03-26 | 济南浪潮高新科技投资发展有限公司 | Bearing monitoring method and system based on cloud and fog edge cooperation |
| CN112966601A (en) * | 2021-03-05 | 2021-06-15 | 上海深硅信息科技有限公司 | Method for artificial intelligence teachers and apprentices to learn by semi-supervision |
| CN113115072A (en) * | 2021-04-09 | 2021-07-13 | 中山大学 | Video target detection tracking scheduling method and system based on end cloud cooperation |
| CN113467771A (en) * | 2020-03-30 | 2021-10-01 | 中国科学院沈阳自动化研究所 | Model-based industrial edge cloud cooperation system and method |
| CN114168446A (en) * | 2022-02-10 | 2022-03-11 | 浙江大学 | Simulation evaluation method and device for mobile terminal operation algorithm model |
| CN114430366A (en) * | 2022-01-25 | 2022-05-03 | 北京百度网讯科技有限公司 | Information acquisition application issuing method, related device and computer program product |
| WO2022139683A1 (en) * | 2020-12-23 | 2022-06-30 | National University Of Singapore | Edge computing based face and gesture recognition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107481295A (en) * | 2017-08-14 | 2017-12-15 | 哈尔滨工业大学 | The image compression system of convolutional neural networks based on dynamic byte length distribution |
| CN107508866A (en) * | 2017-08-08 | 2017-12-22 | 重庆大学 | Reduce the method for the transmission consumption of mobile device end neural network model renewal |
| CN108133266A (en) * | 2017-12-12 | 2018-06-08 | 北京信息科技大学 | A kind of neural network weight compression method and application method based on non-uniform quantizing |
| CN108243216A (en) * | 2016-12-26 | 2018-07-03 | 华为技术有限公司 | Method, end side equipment, cloud side apparatus and the end cloud cooperative system of data processing |
| CN109978142A (en) * | 2019-03-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | The compression method and device of neural network model |
-
2019
- 2019-08-22 CN CN201910777631.7A patent/CN110705684A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108243216A (en) * | 2016-12-26 | 2018-07-03 | 华为技术有限公司 | Method, end side equipment, cloud side apparatus and the end cloud cooperative system of data processing |
| CN107508866A (en) * | 2017-08-08 | 2017-12-22 | 重庆大学 | Reduce the method for the transmission consumption of mobile device end neural network model renewal |
| CN107481295A (en) * | 2017-08-14 | 2017-12-15 | 哈尔滨工业大学 | The image compression system of convolutional neural networks based on dynamic byte length distribution |
| CN108133266A (en) * | 2017-12-12 | 2018-06-08 | 北京信息科技大学 | A kind of neural network weight compression method and application method based on non-uniform quantizing |
| CN109978142A (en) * | 2019-03-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | The compression method and device of neural network model |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461694A (en) * | 2020-03-17 | 2020-07-28 | 上海大学 | Dish identification and pricing system and method based on multi-level learning model |
| CN111461694B (en) * | 2020-03-17 | 2023-07-18 | 上海大学 | Dish identification and pricing system and method based on multi-level learning model |
| CN113467771B (en) * | 2020-03-30 | 2024-04-16 | 中国科学院沈阳自动化研究所 | Model-based industrial edge cloud collaboration system and method |
| CN113467771A (en) * | 2020-03-30 | 2021-10-01 | 中国科学院沈阳自动化研究所 | Model-based industrial edge cloud cooperation system and method |
| CN111625361B (en) * | 2020-05-26 | 2022-11-01 | 华东师范大学 | Joint learning framework based on cooperation of cloud server and IoT (Internet of things) equipment |
| CN111625361A (en) * | 2020-05-26 | 2020-09-04 | 华东师范大学 | Joint learning framework based on cooperation of cloud server and IoT (Internet of things) equipment |
| CN111738435A (en) * | 2020-06-22 | 2020-10-02 | 上海交通大学 | Online sparse training method and system based on mobile equipment |
| CN111738435B (en) * | 2020-06-22 | 2024-03-29 | 上海交通大学 | Online sparse training method and system based on mobile equipment |
| CN111783674A (en) * | 2020-07-02 | 2020-10-16 | 厦门市美亚柏科信息股份有限公司 | Face recognition method and system based on AR glasses |
| CN111985650A (en) * | 2020-07-10 | 2020-11-24 | 华中科技大学 | Activity recognition model and system considering both universality and individuation |
| CN112115975A (en) * | 2020-08-18 | 2020-12-22 | 山东信通电子股份有限公司 | Deep learning network model fast iterative training method and equipment suitable for monitoring device |
| CN112115975B (en) * | 2020-08-18 | 2024-04-12 | 山东信通电子股份有限公司 | Deep learning network model rapid iterative training method and equipment suitable for monitoring device |
| WO2022139683A1 (en) * | 2020-12-23 | 2022-06-30 | National University Of Singapore | Edge computing based face and gesture recognition |
| CN112561097B (en) * | 2020-12-23 | 2023-04-21 | 山东浪潮科学研究院有限公司 | Bearing monitoring method and system based on cloud and mist edge cooperation |
| CN112561097A (en) * | 2020-12-23 | 2021-03-26 | 济南浪潮高新科技投资发展有限公司 | Bearing monitoring method and system based on cloud and fog edge cooperation |
| CN112966601A (en) * | 2021-03-05 | 2021-06-15 | 上海深硅信息科技有限公司 | Method for artificial intelligence teachers and apprentices to learn by semi-supervision |
| CN113115072A (en) * | 2021-04-09 | 2021-07-13 | 中山大学 | Video target detection tracking scheduling method and system based on end cloud cooperation |
| CN114430366A (en) * | 2022-01-25 | 2022-05-03 | 北京百度网讯科技有限公司 | Information acquisition application issuing method, related device and computer program product |
| CN114430366B (en) * | 2022-01-25 | 2024-05-14 | 北京百度网讯科技有限公司 | Information acquisition application issuing method, related device and computer program product |
| CN114168446A (en) * | 2022-02-10 | 2022-03-11 | 浙江大学 | Simulation evaluation method and device for mobile terminal operation algorithm model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110705684A (en) | Environment self-adaptive learning method and system based on end cloud cooperation | |
| CN114362367A (en) | Cloud edge cooperation-oriented power transmission line monitoring system and method, and cloud edge cooperation-oriented power transmission line identification system and method | |
| CN112529149B (en) | Data processing method and related device | |
| CN111797992A (en) | Machine learning optimization method and device | |
| CN113505883A (en) | Neural network training method and device | |
| CN113516227B (en) | Neural network training method and device based on federal learning | |
| CN116368796A (en) | Data processing method and device | |
| CN117131974A (en) | Model training method, power prediction method and equipment | |
| CN112446462B (en) | Method and device for generating target neural network model | |
| CN113379045A (en) | Data enhancement method and device | |
| CN112862021B (en) | Content labeling method and related device | |
| CN114169506A (en) | Deep learning edge computing system framework based on industrial Internet of things platform | |
| Lou et al. | Towards efficient front-end visual sensing for digital retina: A model-centric paradigm | |
| CN113762503A (en) | Data processing method, device, equipment and computer readable storage medium | |
| CN116629375A (en) | Model processing method and system | |
| CN115983362A (en) | Quantization method, recommendation method and device | |
| Zhai et al. | Optimization of agricultural production control based on data processing technology of agricultural internet of things | |
| Liu et al. | Satellite internet of things for smart agriculture applications: A case study of computer vision | |
| Zhang et al. | The Effect of High Performance Computer on Deep Learning: A Face Expression Recognition Case | |
| Mitra et al. | ToEFL: A Novel Approach for Training on Edge in Smart Agriculture | |
| CN113011555B (en) | Data processing method, device, equipment and storage medium | |
| CN113011568B (en) | Model training method, data processing method and equipment | |
| Vignesh et al. | Deep Reinforcement Learning Based Weather Monitoring Systemusing Arduino for Smart Environment | |
| CN115482419B (en) | Data acquisition and analysis method and system for marine fishery products | |
| CN115129861B (en) | Text classification method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200117 |