CN113095506A - Machine learning method, system and medium based on end, edge and cloud cooperation - Google Patents
Machine learning method, system and medium based on end, edge and cloud cooperation Download PDFInfo
- Publication number
- CN113095506A CN113095506A CN202110322362.2A CN202110322362A CN113095506A CN 113095506 A CN113095506 A CN 113095506A CN 202110322362 A CN202110322362 A CN 202110322362A CN 113095506 A CN113095506 A CN 113095506A
- Authority
- CN
- China
- Prior art keywords
- machine learning
- feature
- features
- general
- universal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 162
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 15
- 239000013598 vector Substances 0.000 claims description 16
- 230000005540 biological transmission Effects 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 230000003044 adaptive effect Effects 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 6
- 238000003709 image segmentation Methods 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 22
- 238000013528 artificial neural network Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 8
- 230000007786 learning performance Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 210000001525 retina Anatomy 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
- G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application provides a machine learning method and a machine learning system based on an end, edge and cloud collaborative framework, which are used for extracting a plurality of general features of an input image; transmitting a plurality of generic features; receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task; training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image. According to the method, the universal features which can be used for various visual tasks are extracted from the end framework, the universal features are only transmitted on the side framework without transmitting original images collected by the end, and finally the universal features are utilized to train machine learning tasks at the cloud framework.
Description
Technical Field
The application belongs to the technical field of artificial intelligence, and particularly relates to a machine learning method, system and medium based on an end, edge and cloud collaborative framework.
Background
The promotion of artificial intelligence to the technology promotes social productivity and promotes the revolution of production relation. With the rapid development of machine learning techniques, video analysis techniques and image acquisition techniques,
in machine learning, after an image is collected at an end framework, due to the fact that the transmission speed of an edge framework for transmission is limited or the storage space of a cloud server is limited, and the like, the cost is high when the edge framework directly transmits the image collected at the end framework, and the problems that the machine learning performance of the cloud framework is poor or the preset effect cannot be achieved in the later stage are caused. Based on this, a novel machine learning method for image acquisition and transmission based on an end architecture, a side architecture and a cloud architecture is needed.
Disclosure of Invention
The invention provides a machine learning method, a machine learning system and a machine learning medium based on an end, edge and cloud collaborative framework, and aims to solve the problems of high cost and poor learning performance when an image acquired at an end framework is directly transmitted to an edge framework in the existing machine learning.
According to a first aspect of the embodiments of the present application, a machine learning method based on a terminal, edge and cloud collaborative architecture is provided, which specifically includes the following steps:
extracting a plurality of general features of an input image; the general features comprise a general feature map and/or a general feature vector of the input image;
transmitting a plurality of generic features;
receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task;
training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
In some embodiments of the present application, the step of extracting a plurality of common features of the input image is done at the end architecture; completing the step of transmitting a plurality of generic features at the edge structure; and finishing the steps of receiving a plurality of universal features, adaptively selecting at least one universal feature corresponding to the machine learning task and training the machine learning task corresponding to the at least one universal feature to obtain a trained machine learning model at the cloud architecture.
In some embodiments of the present application, extracting general features of an input image specifically includes:
and extracting the image features of the input image through a plurality of feature extraction networks of multiple levels, and respectively extracting a plurality of universal feature maps or universal feature vectors.
In some embodiments of the present application, before extracting a plurality of common features of the input image, image preprocessing is further included for the input image; image pre-processing includes image translation, image segmentation, image resizing, and/or image color space conversion.
In some embodiments of the present application, the adaptively selecting at least one general feature corresponding to the machine learning task specifically includes:
at least one generic feature is adaptively selected from the incoming plurality of generic features by an adaptive algorithm, the at least one generic feature being used to train a corresponding machine learning task.
In some embodiments of the present application, the machine learning task includes a pedestrian attribute identification task, a vehicle attribute identification task, a pedestrian identification task, and/or a vehicle identification task.
According to a second aspect of the embodiments of the present application, there is provided a machine learning system based on a peer-to-peer, edge-to-peer and cloud collaborative architecture, specifically including:
the universal feature generation module is used for extracting a plurality of universal features of the input image; the general features comprise a general feature map and/or a general feature vector of the input image;
the universal characteristic transmission module is used for transmitting a plurality of universal characteristics;
the self-adaptive universal feature selection module is used for receiving a plurality of universal features and self-adaptively selecting at least one universal feature corresponding to the machine learning task;
the retraining module is used for training the corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
In some embodiments of the present application, the generic feature generation module is disposed in the end architecture; the universal characteristic transmission module is arranged on the side framework; the adaptive universal feature selection module and the retraining module are arranged in the cloud architecture.
According to a third aspect of the embodiments of the present application, there is provided a machine learning device based on a peer-to-peer and cloud collaborative architecture, including:
a memory: for storing executable instructions; and
the processor is used for being connected with the memory to execute the executable instructions so as to complete the machine learning method based on the end, edge and cloud cooperative architecture.
According to a fourth aspect of embodiments of the present application, there is provided a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement a method of machine learning based on an end, edge, and cloud collaborative architecture.
By adopting the machine learning method and system based on the end, edge and cloud collaborative architecture in the embodiment of the application, a plurality of general features of an input image are extracted; the general features comprise a general feature map and/or a general feature vector of the input image; transmitting a plurality of generic features; receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task; training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image. According to the method, the universal features which can be used for various visual tasks are extracted from the end framework, the universal features are only transmitted on the side framework without transmitting original images collected by the end, and finally the universal features are utilized to train machine learning tasks at the cloud framework. The application greatly reduces the operation amount of the whole machine learning task by utilizing the image, and can greatly improve the analysis performance of utilizing the general characteristics at the cloud end. The problems of high cost and poor learning performance when images acquired at the end framework are directly transmitted to the side framework in the prior art are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic diagram illustrating steps of a machine learning method based on an end, edge and cloud collaborative architecture according to an embodiment of the present application;
fig. 2 is a schematic flowchart illustrating a machine learning method based on an end, edge, and cloud collaborative architecture according to an embodiment of the present application;
a flow diagram of an end architecture according to an embodiment of the present application is shown in fig. 3;
a flow diagram of an edge architecture according to an embodiment of the present application is shown in fig. 4;
a flow diagram of a cloud architecture according to an embodiment of the present application is shown in fig. 5;
fig. 6 is a schematic structural diagram illustrating a machine learning system based on a peer-to-peer, edge-to-cloud collaborative architecture according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a machine learning device based on a peer-to-peer, edge-to-cloud collaborative architecture according to an embodiment of the present application.
Detailed Description
In the process of implementing the application, the inventor finds that, when machine learning is performed, after an image is acquired based on an end architecture, due to the fact that the transmission speed of an edge architecture for transmission is limited or the storage space of a cloud server is limited, the cost is high when the edge architecture directly transmits the image acquired by the end architecture, and the problem that the machine learning performance of the cloud architecture is poor or a predetermined effect cannot be achieved in the later stage is caused.
Under such conditions, digital retinal techniques have emerged. The digital retina technology deploys a 'digital retina' at an end, extracts features of an acquired image like a retina, and then only needs to transmit the features to a cloud server at the edge to complete a machine learning task. And the characteristic is universal, and various machine learning tasks on the cloud server can use the characteristic.
The method and the system adopt a collaborative mode of an end architecture, a side architecture and a cloud architecture, and can be developed under common machine learning frameworks such as PyTorch and Tensorflow. The importance of adopting the cooperation of the end architecture, the side architecture and the cloud architecture is gradually highlighted. The application adopts the framework to collect images at the end and preprocess the images, and the cloud server completes various machine vision tasks through edge transmission.
In particular, the method comprises the following steps of,
according to the machine learning method and system based on the end, edge and cloud collaborative framework, a plurality of general features of an input image are extracted; the general features comprise a general feature map and/or a general feature vector of the input image; transmitting a plurality of generic features; receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task; training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
According to the method, the universal features which can be used for various visual tasks are extracted from the end framework, the universal features are only transmitted on the side framework without transmitting original images collected by the end, and finally the universal features are utilized to train machine learning tasks at the cloud framework.
The application greatly reduces the operation amount of the whole machine learning task by utilizing the image, and can greatly improve the analysis performance of utilizing the general characteristics at the cloud end. The problems of high cost and poor learning performance when images acquired at the end framework are directly transmitted to the side framework in the prior art are solved.
In order to make the technical solutions and advantages of the embodiments of the present application more apparent, the following further detailed description of the exemplary embodiments of the present application with reference to the accompanying drawings makes it clear that the described embodiments are only a part of the embodiments of the present application, and are not exhaustive of all embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
Example 1
Fig. 1 is a schematic diagram illustrating steps of a machine learning method based on an end, edge, and cloud collaborative architecture according to an embodiment of the present application. Fig. 2 is a flowchart illustrating a machine learning method based on a peer-to-peer, edge-to-cloud collaborative architecture according to an embodiment of the present application.
As shown in fig. 1, the machine learning method based on the end, edge and cloud collaborative architecture in the embodiment of the present application specifically includes the following steps:
s101: extracting a plurality of general features of an input image; the generic features include a generic feature map and/or a generic feature vector of the input image.
Specifically, image features of an input image are extracted through a plurality of feature extraction networks in multiple levels, and a plurality of general feature maps or general feature vectors are respectively extracted.
In other embodiments, before extracting the plurality of common features of the input image, image preprocessing is further included for the input image; image pre-processing includes image translation, image segmentation, image resizing, and/or image color space conversion.
S102: transmitting a plurality of generic features;
s103: and receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task.
Specifically, at least one generic feature is adaptively selected from the incoming plurality of generic features through an adaptive algorithm, the at least one generic feature being used to train a corresponding machine learning task.
S104: training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
In some embodiments of the present application, the machine learning task includes a pedestrian attribute identification task, a vehicle attribute identification task, a pedestrian identification task, and/or a vehicle identification task.
Specifically, the step of S101 extracting a plurality of common features of the input image is completed at the end architecture; completing S102 the step of transmitting a plurality of generic features at the edge structure; and completing the steps of S103 receiving a plurality of general features, adaptively selecting at least one general feature corresponding to the machine learning task, and S1014 training the corresponding machine learning task by the at least one general feature to obtain a trained machine learning model at the cloud architecture.
The method is based on the neural network, and the method is used for training at the end, extracting general features, transmitting the features on the edge, and finally retraining at the cloud to complete the machine learning task.
In the machine vision task of end-edge-cloud cooperative computing, in order to reduce the amount of transmitted data, universal characteristics of an image are generated by utilizing various neural networks at an end; at the edge, passing the generic feature; at the cloud, various machine vision tasks adaptively select common features; at the cloud, further retraining is performed with generic features.
Wherein the image is pre-processed on the end device; then, the preprocessed image is input into various leading-edge deep neural networks, and then a part of layers of the neural networks are adopted for feature extraction, so that the features of the image are obtained. The multiple image features obtained by multiple networks and multiple hierarchies are also called general features because the multiple image features can be applied to various machine vision tasks.
To further illustrate the embodiment, fig. 3 shows a schematic flow chart of the peer architecture according to the embodiment of the present application; a flow diagram of an edge architecture according to an embodiment of the present application is shown in fig. 4; a flow diagram of a cloud architecture according to an embodiment of the present application is shown in fig. 5.
As shown in fig. 3, for an incoming image xoriginalImage preprocessing is carried out by methods including but not limited to image translation, cutting, size scaling, color space conversion and the like to obtain a preprocessed image xpre。
The preprocessed image x is then applied to multiple levels of the various networkspreAnd (5) carrying out feature extraction. The network employed to extract the generic features can be mathematically represented as: f. ofFeature_Extract_1_1，fFeature_Extract_1_2，...，fFeature_Extract_m_nWherein m is the number of different networks, and n is the number of layers adopted by each network.
Thus, for image xpreThe extracted multiple common features can be expressed mathematically as: x is the number offeature_1_1＝fFeature_Extract_1_1(xpre)，xfeature_1_2＝fFeature_Extract_1_2(xpre)，...，xfeature_m_n＝fFeature_Extract_m_n(xpre). X obtained in this stepfeature_1_1，xfeature_1_2，...，xfeature_m_nNamely, the data is transmitted to the cloud end through the edge, and the cloud end can continue to use the universal characteristic.
As shown in fig. 4, x is derived from the end architecture using appropriate network devicesfeature_1_1，xfeature_1_2，...，xfeature_m_nAnd transmitting a series of common characteristics from the end to the cloud through the edge. The data volume of these generic features is much smaller than the original image x captured at the endpre。
As shown in FIG. 5, a machine vision task at the cloud receives x transmitted from the edgefeature_1_1，xfeature_1_2，...，xfeature_m_nA series of general characteristics are obtained, and x which is most suitable for self algorithm is selected through an adaptive algorithmfeature_selectedGeneral features. x is the number offeature_selectedThis generic feature will be applied to the retraining of the machine vision task.
Universal feature x selected at step on machine vision task at cloudfeature_selectedAnd (5) performing retraining. For heavy trainingCan be expressed mathematically as fTraining_Network. For general feature xfeature_selectedY ═ f obtained after retrainingTraining_Network(xfeature_selected) Namely the prediction result of the machine vision task.
Wherein, each machine learning task in the cloud includes but is not limited to: and selecting proper general characteristics from the general characteristics transmitted to the cloud for further relearning by adopting a self-adaptive algorithm. And each machine learning task continues machine learning on the machine learning model configured at the cloud end by utilizing the general characteristics so as to complete the machine vision task, which is called as heavy learning.
In the machine learning method based on the end, edge and cloud collaborative architecture in the embodiment of the application, a plurality of general features of an input image are extracted; the general features comprise a general feature map and/or a general feature vector of the input image; transmitting a plurality of generic features; receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task; training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
According to the method, the universal features which can be used for various visual tasks are extracted from the end framework, the universal features are only transmitted on the side framework without transmitting original images collected by the end, and finally the universal features are utilized to train machine learning tasks at the cloud framework. For different machine vision tasks and corresponding neural networks, common features are extracted uniformly at the end.
The application greatly reduces the operation amount of the whole machine learning task by utilizing the image, and can greatly improve the analysis performance of utilizing the general characteristics at the cloud end. The problems of high cost and poor learning performance when images acquired at the end framework are directly transmitted to the side framework in the prior art are solved.
Example 2
For details not disclosed in the machine learning system based on the peer-to-peer, peer-to-peer and cloud collaborative architecture of this embodiment, please refer to specific implementation contents of the machine learning method based on the peer-to-peer, peer-to-cloud collaborative architecture in other embodiments.
Fig. 6 is a schematic structural diagram of a machine learning system based on a peer-to-peer, edge-to-cloud collaborative architecture according to an embodiment of the present application.
As shown in fig. 6, the machine learning system based on the end, edge, and cloud collaborative architecture according to the embodiment of the present application specifically includes a general feature generation module 10, a general feature transmission module 20, an adaptive general feature selection module 30, and a retraining module 40.
In particular, the method comprises the following steps of,
a general feature generation module 10, configured to extract a plurality of general features of an input image; the generic features include a generic feature map and/or a generic feature vector of the input image.
Specifically, image features of an input image are extracted through a plurality of feature extraction networks in multiple levels, and a plurality of general feature maps or general feature vectors are respectively extracted.
In other embodiments, before extracting the plurality of common features of the input image, image preprocessing is further included for the input image; image pre-processing includes image translation, image segmentation, image resizing, and/or image color space conversion.
A generic feature transmission module 20 for transmitting a plurality of generic features.
And the adaptive universal feature selection module 30 is configured to receive a plurality of universal features and adaptively select at least one universal feature corresponding to the machine learning task.
Specifically, at least one generic feature is adaptively selected from the incoming plurality of generic features through an adaptive algorithm, the at least one generic feature being used to train a corresponding machine learning task.
The retraining module 40 is configured to train a corresponding machine learning task according to at least one general feature to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
Specifically, the generic feature generation module 10 is disposed in the end framework; the universal feature transmission module 20 is disposed on the edge architecture; the adaptive universal feature selection module 30 and the retraining module 40 are disposed in the cloud architecture.
The method is based on the neural network, and the method is used for training at the end, extracting general features, transmitting the features on the edge, and finally retraining at the cloud to complete the machine learning task.
In the machine vision task of end-edge-cloud cooperative computing, in order to reduce the amount of transmitted data, universal characteristics of an image are generated by utilizing various neural networks at an end; at the edge, passing the generic feature; at the cloud, various machine vision tasks adaptively select common features; at the cloud, further retraining is performed with generic features.
Wherein the image is pre-processed on the end device; then, the preprocessed image is input into various leading-edge deep neural networks, and then a part of layers of the neural networks are adopted for feature extraction, so that the features of the image are obtained. The multiple image features obtained by multiple networks and multiple hierarchies are also called general features because the multiple image features can be applied to various machine vision tasks.
To further illustrate the embodiment, fig. 3 shows a schematic flow chart of the peer architecture according to the embodiment of the present application; a flow diagram of an edge architecture according to an embodiment of the present application is shown in fig. 4; a flow diagram of a cloud architecture according to an embodiment of the present application is shown in fig. 5.
As shown in fig. 3, for an incoming image xoriginalImage preprocessing is carried out by methods including but not limited to image translation, cutting, size scaling, color space conversion and the like to obtain a preprocessed image xpre。
The preprocessed image x is then applied to multiple levels of the various networkspreAnd (5) carrying out feature extraction. The network employed to extract the generic features can be mathematically represented as: f. ofFeature_Extract_1_1，fFeature_Extract_1_2，...，fFeature_Extract_m_nWherein m is the number of different networks, and n is the number of layers adopted by each network.
Thus, for image xpreThe extracted multiple common features can be expressed mathematically as: x is the number offeature_1_1＝fFeature_Extract_1_1(xpre)，xfeature_1_2＝fFeature_Extract_1_2(xpre)，...，xfeature_m_n＝fFeature_Extract_m_n(xpre). X obtained in this stepfeature_1_1，xfeature_1_2，...，xfeature_m_nNamely, the data is transmitted to the cloud end through the edge, and the cloud end can continue to use the universal characteristic.
As shown in fig. 4, x is derived from the end architecture using appropriate network devicesfeature_1_1，xfeature_1_2，...，xfeature_m_nAnd transmitting a series of common characteristics from the end to the cloud through the edge. The data volume of these generic features is much smaller than the original image x captured at the endpre。
As shown in FIG. 5, a machine vision task at the cloud receives x transmitted from the edgefeature_1_1，xfeature_1_2，...，xfeature_m_nA series of general characteristics are obtained, and x which is most suitable for self algorithm is selected through an adaptive algorithmfeature_selectedGeneral features. x is the number offeature_selectedThis generic feature will be applied to the retraining of the machine vision task.
Universal feature x selected at step on machine vision task at cloudfeature_selectedAnd (5) performing retraining. The neural network used for retraining can be mathematically expressed as fTraining_Network. For general feature xfeature_selectedY ═ f obtained after retrainingTraining_Network(xfeature_selected) Namely the prediction result of the machine vision task.
Wherein, each machine learning task in the cloud includes but is not limited to: and selecting proper general characteristics from the general characteristics transmitted to the cloud for further relearning by adopting a self-adaptive algorithm. And each machine learning task continues machine learning on the machine learning model configured at the cloud end by utilizing the general characteristics so as to complete the machine vision task, which is called as heavy learning.
In the machine learning system based on the end, edge and cloud collaborative architecture in the embodiment of the application, the general feature generation module 10 extracts a plurality of general features of an input image; the general features comprise a general feature map and/or a general feature vector of the input image; the universal feature transmission module 20 transmits a plurality of universal features; the adaptive general feature selection module 30 receives a plurality of general features and adaptively selects at least one general feature corresponding to the machine learning task; the retraining module 40 trains the corresponding machine learning task according to at least one general feature to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
According to the method, the universal features which can be used for various visual tasks are extracted from the end framework, the universal features are only transmitted on the side framework without transmitting original images collected by the end, and finally the universal features are utilized to train machine learning tasks at the cloud framework. For different machine vision tasks and corresponding neural networks, common features are extracted uniformly at the end.
The application greatly reduces the operation amount of the whole machine learning task by utilizing the image, and can greatly improve the analysis performance of utilizing the general characteristics at the cloud end. The problems of high cost and poor learning performance when images acquired at the end framework are directly transmitted to the side framework in the prior art are solved.
Example 3
For details that are not disclosed in the machine learning device based on the peer-to-peer, peer-to-peer and cloud coordination architecture of this embodiment, please refer to specific implementation contents of the machine learning method or system based on the peer-to-peer, peer-to-cloud coordination architecture in other embodiments.
Fig. 7 is a schematic structural diagram of a machine learning device 400 based on a peer-to-peer, edge-to-cloud collaborative architecture according to an embodiment of the present application.
As shown in fig. 7, the machine learning apparatus 400 includes:
the memory 402: for storing executable instructions; and
a processor 401 is coupled to the memory 402 to execute executable instructions to perform the motion vector prediction method.
Those skilled in the art will appreciate that the schematic diagram 7 is merely an example of the machine learning device 400 and does not constitute a limitation of the machine learning device 400 and may include more or fewer components than shown, or combine certain components, or different components, e.g., the machine learning device 400 may also include input-output devices, network access devices, buses, etc.
The Processor 401 (CPU) may be other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. The general purpose processor may be a microprocessor or the processor 401 may be any conventional processor or the like, and the processor 401 is the control center of the machine learning device 400 and connects the various parts of the entire machine learning device 400 using various interfaces and lines.
The memory 402 may be used to store computer readable instructions and the processor 401 may implement the various functions of the machine learning device 400 by executing or executing computer readable instructions or modules stored in the memory 402 and invoking data stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the machine learning apparatus 400, and the like. In addition, the Memory 402 may include a hard disk, a Memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Memory Card (Flash Card), at least one disk storage device, a Flash Memory device, a Read-Only Memory (ROM), a Random Access Memory (RAM), or other non-volatile/volatile storage devices.
The modules integrated by the machine learning device 400, if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer-readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by hardware related to computer readable instructions, which may be stored in a computer readable storage medium, and when the computer readable instructions are executed by a processor, the steps of the method embodiments may be implemented.
Example 4
The present embodiment provides a computer-readable storage medium having stored thereon a computer program; the computer program is executed by a processor to implement the end, edge, and cloud collaborative architecture based machine learning methods in other embodiments.
The machine learning equipment and the computer storage medium based on the end, edge and cloud collaborative architecture in the embodiment of the application extract a plurality of general features of an input image; the general features comprise a general feature map and/or a general feature vector of the input image; transmitting a plurality of generic features; receiving a plurality of general features, and adaptively selecting at least one general feature corresponding to the machine learning task; training a corresponding machine learning task according to at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
According to the method, the universal features which can be used for various visual tasks are extracted from the end framework, the universal features are only transmitted on the side framework without transmitting original images collected by the end, and finally the universal features are utilized to train machine learning tasks at the cloud framework. For different machine vision tasks and corresponding neural networks, common features are extracted uniformly at the end.
The application greatly reduces the operation amount of the whole machine learning task by utilizing the image, and can greatly improve the analysis performance of utilizing the general characteristics at the cloud end. The problems of high cost and poor learning performance when images acquired at the end framework are directly transmitted to the side framework in the prior art are solved.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A machine learning method based on end, edge and cloud cooperative architecture specifically comprises the following steps:
extracting a plurality of general features of an input image; the general feature comprises a general feature map and/or a general feature vector of the input image;
transmitting the plurality of generic features;
receiving the universal features, and adaptively selecting at least one universal feature corresponding to the machine learning task;
training a corresponding machine learning task according to the at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
2. The machine learning method of claim 1, wherein the step of extracting a plurality of common features of the input image is done at an end architecture; completing said step of transmitting said plurality of generic features at the edge structure; and finishing the steps of receiving the plurality of universal features, adaptively selecting at least one universal feature corresponding to the machine learning task and training the machine learning task corresponding to the at least one universal feature to obtain a trained machine learning model at a cloud architecture.
3. The machine learning method according to claim 1, wherein the extracting the general features of the input image specifically comprises:
and extracting the image features of the input image through a plurality of feature extraction networks of multiple levels, and respectively extracting a plurality of general feature maps or general feature vectors.
4. The machine learning method of claim 1, wherein prior to extracting the plurality of generic features of the input image, further comprising image preprocessing the input image; the image pre-processing includes image translation, image segmentation, image resizing and/or image color space conversion.
5. The machine learning method according to claim 1, wherein the adaptively selecting at least one generic feature corresponding to the machine learning task specifically comprises:
adaptively selecting, by an adaptive algorithm, at least one generic feature from an incoming plurality of generic features, the at least one generic feature being used to train a corresponding machine learning task.
6. The machine learning method of any of claims 1-5, wherein the machine learning task comprises a pedestrian attribute identification task, a vehicle attribute identification task, a pedestrian identification task, and/or a vehicle identification task.
7. A machine learning system based on end, edge and cloud collaborative architecture is characterized by specifically comprising:
the universal feature generation module is used for extracting a plurality of universal features of the input image; the general feature comprises a general feature map and/or a general feature vector of the input image;
a universal feature transmission module for transmitting the plurality of universal features;
the self-adaptive universal feature selection module is used for receiving the universal features and self-adaptively selecting at least one universal feature corresponding to the machine learning task;
the retraining module is used for training the corresponding machine learning task according to the at least one general characteristic to obtain a trained machine learning model; and the trained machine learning model obtains the prediction result of the machine learning task according to the input image.
8. The machine learning system of claim 7, wherein the generic feature generation module is disposed in an end architecture; the universal characteristic transmission module is arranged on the side framework; the adaptive universal feature selection module and the retraining module are arranged in the cloud architecture.
9. A machine learning device based on end, edge and cloud collaborative architecture, comprising:
a memory: for storing executable instructions; and
a processor for interfacing with the memory to execute the executable instructions to perform the method of machine learning based on end, edge and cloud collaborative architecture of any of claims 1-6.
10. A computer-readable storage medium, having stored thereon a computer program; the computer program is executed by a processor to implement the method of end, edge and cloud collaborative architecture based machine learning according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322362.2A CN113095506A (en) | 2021-03-25 | 2021-03-25 | Machine learning method, system and medium based on end, edge and cloud cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322362.2A CN113095506A (en) | 2021-03-25 | 2021-03-25 | Machine learning method, system and medium based on end, edge and cloud cooperation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113095506A true CN113095506A (en) | 2021-07-09 |
Family
ID=76669796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110322362.2A Pending CN113095506A (en) | 2021-03-25 | 2021-03-25 | Machine learning method, system and medium based on end, edge and cloud cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095506A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113592003A (en) * | 2021-08-04 | 2021-11-02 | 智道网联科技(北京)有限公司 | Picture transmission method, device, equipment and storage medium |
| CN114422805A (en) * | 2022-03-30 | 2022-04-29 | 浙江智慧视频安防创新中心有限公司 | Video coding and decoding method, device and equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109543829A (en) * | 2018-10-15 | 2019-03-29 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Method and system for hybrid deployment of deep learning neural network on terminal and cloud |
| CN110414373A (en) * | 2019-07-08 | 2019-11-05 | 武汉大学 | A kind of deep learning palm vein identification system and method based on cloud side end cooperated computing |
| CN110440396A (en) * | 2019-07-11 | 2019-11-12 | 雄安达实智慧科技有限公司 | The central air-conditioning global optimization energy-saving control method and system of cloud side end collaboration |
| CN110942012A (en) * | 2019-11-22 | 2020-03-31 | 上海眼控科技股份有限公司 | Image feature extraction method, pedestrian re-identification method, device and computer equipment |
| CN111242282A (en) * | 2020-01-09 | 2020-06-05 | 中山大学 | Deep learning model training acceleration method based on end edge cloud cooperation |
| CN111553213A (en) * | 2020-04-17 | 2020-08-18 | 大连理工大学 | Real-time distributed identity-aware pedestrian attribute identification method in mobile edge cloud |
| CN112367400A (en) * | 2020-11-12 | 2021-02-12 | 广东电网有限责任公司 | Intelligent inspection method and system for power internet of things with edge cloud coordination |
-
2021
- 2021-03-25 CN CN202110322362.2A patent/CN113095506A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109543829A (en) * | 2018-10-15 | 2019-03-29 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Method and system for hybrid deployment of deep learning neural network on terminal and cloud |
| CN110414373A (en) * | 2019-07-08 | 2019-11-05 | 武汉大学 | A kind of deep learning palm vein identification system and method based on cloud side end cooperated computing |
| CN110440396A (en) * | 2019-07-11 | 2019-11-12 | 雄安达实智慧科技有限公司 | The central air-conditioning global optimization energy-saving control method and system of cloud side end collaboration |
| CN110942012A (en) * | 2019-11-22 | 2020-03-31 | 上海眼控科技股份有限公司 | Image feature extraction method, pedestrian re-identification method, device and computer equipment |
| CN111242282A (en) * | 2020-01-09 | 2020-06-05 | 中山大学 | Deep learning model training acceleration method based on end edge cloud cooperation |
| CN111553213A (en) * | 2020-04-17 | 2020-08-18 | 大连理工大学 | Real-time distributed identity-aware pedestrian attribute identification method in mobile edge cloud |
| CN112367400A (en) * | 2020-11-12 | 2021-02-12 | 广东电网有限责任公司 | Intelligent inspection method and system for power internet of things with edge cloud coordination |
Non-Patent Citations (3)
| Title |
|---|
| AMIR ERFAN ESHRATIFAR等: "Towards Collaborative Intelligence Friendly", 《ARXIV》 * |
| PEIYIN XING等: "End-Edge-Cloud Collaborative System: A Video Big Data Processing and Analysis Architecture", 《2020 IEEE CONFERENCE ON MULTIMEDIA INFORMATION PROCESSING AND RETRIEVAL (MIPR)》 * |
| 郭斌等: "深度学习模型终端环境自适应方法研究", 《中国科学：信息科学》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113592003A (en) * | 2021-08-04 | 2021-11-02 | 智道网联科技(北京)有限公司 | Picture transmission method, device, equipment and storage medium |
| CN113592003B (en) * | 2021-08-04 | 2023-12-26 | 智道网联科技(北京)有限公司 | Picture transmission method, device, equipment and storage medium |
| CN114422805A (en) * | 2022-03-30 | 2022-04-29 | 浙江智慧视频安防创新中心有限公司 | Video coding and decoding method, device and equipment |
| CN114422805B (en) * | 2022-03-30 | 2022-09-02 | 浙江智慧视频安防创新中心有限公司 | Video coding and decoding method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108416327B (en) | Target detection method and device, computer equipment and readable storage medium | |
| US20230089380A1 (en) | Neural network construction method and apparatus | |
| KR20170140214A (en) | Filter specificity as training criterion for neural networks | |
| US20210056388A1 (en) | Knowledge Transfer Between Different Deep Learning Architectures | |
| CN111696110B (en) | Scene segmentation method and system | |
| CN110222718B (en) | Image processing method and device | |
| CN113095506A (en) | Machine learning method, system and medium based on end, edge and cloud cooperation | |
| CN112184585B (en) | Image completion method and system based on semantic edge fusion | |
| EP3809364A1 (en) | Computing apparatus and method for constructing a mosaic image | |
| CN113592041B (en) | Image processing method, apparatus, device, storage medium, and computer program product | |
| WO2023207778A1 (en) | Data recovery method and device, computer, and storage medium | |
| US20230153965A1 (en) | Image processing method and related device | |
| US20220207913A1 (en) | Method and device for training multi-task recognition model and computer-readable storage medium | |
| CN111833360A (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN110688897A (en) | Pedestrian re-identification method and device based on joint judgment and generation learning | |
| CN113724128A (en) | Method for expanding training sample | |
| CN114445651A (en) | Training set construction method and device of semantic segmentation model and electronic equipment | |
| CN112598110B (en) | Neural network construction method, device, equipment and medium | |
| CN117252890A (en) | Carotid plaque segmentation method, device, equipment and medium | |
| CN107729885A (en) | A kind of face Enhancement Method based on the study of multiple residual error | |
| CN110490876B (en) | Image segmentation method based on lightweight neural network | |
| CN115758245A (en) | Multi-mode data classification method, device, equipment and storage medium | |
| CN111815658B (en) | Image recognition method and device | |
| CN113840169A (en) | Video processing method and device, computing equipment and storage medium | |
| CN113705600A (en) | Feature map determination method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210709 |
|
| RJ01 | Rejection of invention patent application after publication |