CN114020469A - Edge node-based multi-task learning method, device, medium and equipment - Google Patents
Edge node-based multi-task learning method, device, medium and equipment Download PDFInfo
- Publication number
- CN114020469A CN114020469A CN202111321824.5A CN202111321824A CN114020469A CN 114020469 A CN114020469 A CN 114020469A CN 202111321824 A CN202111321824 A CN 202111321824A CN 114020469 A CN114020469 A CN 114020469A
- Authority
- CN
- China
- Prior art keywords
- node
- edge node
- sampling
- model
- edge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The disclosure provides a multitask learning method, a multitask learning device, a multitask learning medium and multitask learning equipment based on edge nodes, and relates to the technical field of computers. Wherein the method comprises the following steps: carrying out sample balance processing on each edge node according to the sample number set of each edge node, and sending the sample balance processing result of each edge node to the corresponding edge node so that each edge node carries out up-sampling or down-sampling according to the sample balance processing result; acquiring learning model parameters corresponding to each node scheduler, wherein the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to samples of target edge nodes, and the target edge nodes are nodes distributed for the node schedulers from the edge nodes; and performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to each node scheduler. By balancing the samples, the redundant computation is reduced, and unbalanced node load and waste of computational resources can be avoided.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a multitask learning method based on edge nodes, a multitask learning apparatus based on edge nodes, a computer-readable storage medium, and an electronic device.
Background
Federal learning is a framework for training a learning model by multiple parties, and gradually becomes a key technology for breaking a data barrier due to the characteristics of privacy protection and the capability of cross-subject machine learning. However, the problem of uneven distribution of node samples may occur in the edge nodes, which not only causes the accuracy of the learning model to decrease, but also causes unbalanced load of the edge nodes, thereby causing resource waste.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure provides an edge node-based multitask learning method, an edge node-based multitask learning apparatus, a computer-readable storage medium, and an electronic device, thereby solving, at least to some extent, the problems of unbalanced load of edge nodes and resource waste in the related art.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to a first aspect of the present disclosure, there is provided an edge node-based multitask learning method, including: carrying out sample balance processing on each edge node according to the sample number set of each edge node, and sending the sample balance processing result of each edge node to the corresponding edge node so that each edge node carries out up-sampling or down-sampling according to the sample balance processing result; acquiring learning model parameters corresponding to each node scheduler, wherein the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to samples of target edge nodes, and the target edge nodes are nodes distributed to the node schedulers from the edge nodes; and performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to the node schedulers.
In an exemplary embodiment of the present disclosure, the performing a sample balancing process on each edge node according to the sample number set of each edge node includes: determining a mean and a variance of the sample number set according to the sample number set of each edge node; determining a Z value corresponding to the sample number of each edge node according to the mean value and the variance; and carrying out sample balance processing on each edge node based on the Z value corresponding to the sample number of each edge node.
In an exemplary embodiment of the present disclosure, the performing, on the basis of the Z value corresponding to the sample number of each edge node, sample balancing processing on each edge node includes: when a Z value corresponding to the number of samples of an edge node exceeds a first preset threshold value, taking the edge node as a down-sampling edge node so as to perform sample down-sampling on the down-sampling edge node; when a Z value corresponding to the number of samples of an edge node is lower than a second preset threshold value, taking the edge node as an up-sampling edge node so as to enable the up-sampling edge node to perform sample up-sampling; wherein the first preset threshold is greater than the second preset threshold.
In an exemplary embodiment of the present disclosure, the method further comprises: determining a down-sampling scale of the down-sampling edge node according to the number of samples of the down-sampling edge node, a Z value corresponding to the number of samples of the down-sampling edge node, the first preset threshold, the mean value and the variance, so that the down-sampling edge node performs sample down-sampling according to the down-sampling scale; and determining the up-sampling scale of the up-sampling edge node according to the number of samples of the up-sampling edge node, the Z value corresponding to the number of samples of the up-sampling edge node, the second preset threshold, the mean value and the variance, so that the up-sampling edge node performs sample up-sampling according to the up-sampling scale.
In an exemplary embodiment of the present disclosure, the performing multi-task learning on the preset learning model by using the learning model parameters corresponding to the node schedulers includes: and establishing a model matrix and a model relation matrix corresponding to the model matrix by adopting the learning model parameters corresponding to each node scheduler, and updating the model matrix based on the model relation matrix to obtain a target model matrix.
In an exemplary embodiment of the present disclosure, the updating the model matrix based on the model relationship matrix to obtain a target model matrix includes: constructing a target optimization function of the multi-task learning by adopting the model matrix and the model relation matrix; and updating the model matrix, and taking the model matrix when the target function is optimal as the target model matrix.
In an exemplary embodiment of the present disclosure, the updating the model matrix includes: and performing one round of updating on the model matrix by taking a single node scheduler as a unit to obtain a new model matrix, so that the target edge node corresponding to each node scheduler performs a new round of iterative training on the preset learning model based on the new model matrix.
In an exemplary embodiment of the disclosure, the performing a round of update on the model matrix in units of a single node scheduler includes: and sequentially updating the sub-problem parameters corresponding to the multi-task learning by taking a single node scheduler as a unit, updating the model relation matrix based on the updated sub-problem parameters, and updating the model matrix based on the updated model relation matrix.
In an exemplary embodiment of the present disclosure, the target edge node is a node determined from the edge nodes according to a KL divergence of the node scheduler.
According to a second aspect of the present disclosure, there is provided an edge node-based multitask learning apparatus including: the sample balance processing module is used for carrying out sample balance processing on each edge node according to the sample number set of each edge node and sending the sample balance processing result of each edge node to the corresponding edge node so as to enable each edge node to carry out up-sampling or down-sampling according to the sample balance processing result; a model parameter obtaining module, configured to obtain learning model parameters corresponding to each node scheduler, where the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to a sample of a target edge node, and the target edge node is a node allocated to the node scheduler from among the edge nodes; and the multi-task learning module is used for performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to the node schedulers.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described edge node-based multitask learning method.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described edge node-based multi-task learning method via execution of the executable instructions.
The technical scheme of the disclosure has the following beneficial effects:
in the multitask learning process based on the edge nodes, sample balance processing is carried out on each edge node according to the sample number set of each edge node, and the sample balance processing result of each edge node is sent to the corresponding edge node, so that each edge node carries out up-sampling or down-sampling according to the sample balance processing result; acquiring learning model parameters corresponding to each node scheduler, wherein the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to samples of target edge nodes, and the target edge nodes are nodes distributed for the node schedulers from the edge nodes; and performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to each node scheduler. By carrying out sample balance processing on the edge nodes in advance, the influence caused by the unbalanced global scale and the unbalanced class of the samples is reduced, and further the unbalanced load of the edge nodes is avoided. In addition, the edge nodes are distributed to the node schedulers in a node scheduling mode, and participate in the multi-task learning by taking the node schedulers as units, so that the redundant calculation of the multi-task learning can be greatly reduced, the federal learning efficiency is improved, the resource consumption is reduced, and the cost is reduced. In addition, the node scheduler is applied to multi-task federal learning, and the influence caused by the non-independent and same distribution of sample heights of edge nodes can be avoided, so that the federal learning precision is improved, and the applicable application scene is favorably expanded.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is apparent that the drawings in the following description are only some embodiments of the present disclosure, and that other drawings can be obtained from those drawings without inventive effort for a person skilled in the art.
FIG. 1 illustrates a multitasking learning model building system architecture diagram in accordance with the present exemplary embodiment;
FIG. 2 illustrates a flow chart of a method of edge node based multitask learning in the exemplary embodiment;
FIG. 3 is a flowchart illustrating a sample balancing process performed on the edge nodes in the exemplary embodiment;
FIG. 4 illustrates a flowchart of a Z-value based multi-task learning in the present exemplary embodiment;
FIG. 5 is a block diagram of a multi-task learning system with assigned edge nodes for a node scheduler in the exemplary embodiment;
FIG. 6 is a block diagram showing the structure of an edge node-based multitask learning apparatus according to the present exemplary embodiment;
fig. 7 illustrates an electronic device for implementing an edge node-based multitask learning method in the present exemplary embodiment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
Herein, "first", "second", etc. are labels for specific objects, and do not limit the number or order of the objects.
In the related art, the statistical heterogeneity characteristic of federal learning may exist due to the unbalanced distribution of edge node samples, and the statistical heterogeneity characteristic is mainly classified into three types: unbalanced distribution in scale, unbalanced distribution in category, non-independent co-distribution on edge nodes. The existing multi-task learning mode needs to iterate sample data of each edge node to solve the problem of sample distribution imbalance, so that calculation redundancy can be caused, and the distribution of the sample data cannot be fundamentally changed by the mode.
In addition, the problem of sample maldistribution is also solved by the node scheduler at present. However, in the existing node scheduling mode, after the edge node distributed by the node scheduler is trained, the scheduler is used as a unit to participate in federal averaging, and the influence caused by the non-independent same distribution characteristic of the edge node sample data cannot be avoided.
In view of one or more of the above problems, exemplary embodiments of the present disclosure provide an edge node-based multitask learning method, which may be applied in scenarios including, but not limited to: message and task delay prediction based on federal learning; a smart city and smart community system based on the Internet of things; 5G intelligent medical treatment; a distribution system based on mobile phone behavior information.
For the message and task delay prediction scene based on the federal learning: in the task scheduling process of edge computing, a phenomenon that part of tasks of edge nodes are lost due to problems of communication delay, untimely task unloading and the like may occur, a central server for data such as message logs of the edge nodes is usually inaccessible, and delay data of different types of edge nodes and different task types are different in statistical characteristics. The multi-task learning method based on the edge nodes is applied to a message and task delay prediction scene based on federated learning, and a machine learning model for predicting node messages and task delays is trained through message log data of the edge nodes, so that the task scheduling efficiency of an edge platform can be improved to a certain extent, and tasks are prevented from being lost.
For smart city and smart community system scene based on internet of things: the building of wisdom city system often needs to collect the data of multidimension degree, high complexity, when the wisdom community system is carrying out the community monitoring network, often needs to carry out the model training through the third party based on a large amount of resident personal information. The multi-task learning method based on the edge nodes is applied to the smart city and smart community system based on the Internet of things, personalized model training can be completed on the premise that the edge node data are not disclosed to the central server under the conditions of large data scale and high data statistics heterogeneity, and privacy of personal information is guaranteed.
For a 5G smart medical scenario: when the machine learning is used for intelligent medical treatment, the problems of large sample data difference, high privacy, data barrier of combined diagnosis data samples and the like are often faced, and the data barrier problem can be well solved by the federal learning. According to the multi-task learning method based on the edge nodes, a model with higher precision can be trained for data samples with large differences, so that more accurate and personalized diagnosis can be realized.
For a distribution system scene based on mobile phone behavior information: sometimes, a model is constructed through behavior information reflected by private domain traffic of some mobile phone users to distribute recommendation strategies. The model training is difficult due to the huge data volume of private domain flow and the complex data types, and the multi-task learning method based on the edge nodes can well avoid privacy problems and eliminate the influence of data statistics heterogeneity on the model training.
Referring to fig. 1, fig. 1 is a diagram of a multitask learning model building system architecture provided by an embodiment of the present invention, including: edge nodes 110, a node dispatcher 120, and a central server 130. The edge node 110 is a service platform constructed on the network edge side close to the user, provides resources such as storage, computation, network and the like, sinks part of key service applications to the access network edge to reduce the width and delay loss caused by network transmission and multistage forwarding, and may be a certain machine room or a certain physical device, an edge gateway, a home gateway, an IoT gateway, a mobile phone, a computer and the like. The node scheduler 120 is primarily responsible for allocating its edge nodes, and may allocate one or more edge nodes for a node scheduler. The number of edge nodes 110 and node schedulers 120 in fig. 1 is merely exemplary and any number may be provided depending on implementation needs. The central server 130 may be a server cluster composed of a plurality of computing devices, may provide computing or application services, and has high-speed CPU computing capability, long-time reliable operation, strong I/O external data throughput capability, and better extensibility. The communication network in the system architecture may be a wide area network or a local area network, or a combination of both.
Fig. 2 shows a schematic flow of an edge node-based multitask learning method in the present exemplary embodiment, which is executed by a central server and includes the following steps S210 to S230:
step S210, carrying out sample balance processing on each edge node according to the sample number set of each edge node, and sending the sample balance processing result of each edge node to the corresponding edge node so that each edge node carries out up-sampling or down-sampling according to the sample balance processing result;
step S220, acquiring learning model parameters corresponding to each node scheduler, wherein the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to samples of target edge nodes, and the target edge nodes are nodes distributed for the node schedulers from each edge node;
and step S230, performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to each node scheduler.
In the multi-task learning process based on the edge nodes, the sample balance processing is performed on the edge nodes in advance, so that the influence caused by the global scale imbalance and the class imbalance of the samples is reduced, and further the load imbalance of the edge nodes is avoided. In addition, the edge nodes are distributed to the node schedulers in a node scheduling mode, and participate in the multi-task learning by taking the node schedulers as units, so that the redundant calculation of the multi-task learning can be greatly reduced, the federal learning efficiency is improved, the resource consumption is reduced, and the cost is reduced. In addition, the node scheduler is applied to multi-task federal learning, and the influence caused by the non-independent and same distribution of sample heights of edge nodes can be avoided, so that the federal learning precision is improved, and the applicable application scene is favorably expanded.
Each step in fig. 2 will be described in detail below.
Step S210, performing sample balancing processing on each edge node according to the sample number set of each edge node, and sending the sample balancing processing result of each edge node to the corresponding edge node, so that each edge node performs up-sampling or down-sampling according to the sample balancing processing result.
The number of samples of the edge node refers to the number of sample elements in the sample data set of the edge node, and may represent the sample size of the edge node.
The number of samples for each edge node may be collected by the central server and a set of numbers of samples for each edge node may be constructed, e.g., C ═ C1,C2,…,CN]Where C is a set of sample numbers for N edge nodes, C1,C2,…,CNRespectively representing the number of samples of each edge node.
In an optional implementation manner, the performing the sample balancing process on each edge node according to the sample number set of each edge node may be implemented by the steps shown in fig. 3, and specifically includes the following steps S310 to S330:
step S310, determining the mean value and the variance of the sample number set according to the sample number set of each edge node;
step S320, determining a Z value corresponding to the sample number of each edge node according to the mean value and the variance;
step S330, sample balance processing is carried out on each edge node based on the Z value corresponding to the sample number of each edge node.
Taking the sample number set C of each edge node as an example, a Z value set corresponding to the sample number of each edge node can be obtained by calculating Z ═ C- μ)/σ. Wherein Z represents a Z value set corresponding to the sample number of each edge node, mu represents a mean value of the sample number set, and sigma represents a variance of the sample number set.
In the process, the Z value of the number of the samples is calculated, the sample balance processing is carried out on the edge nodes in advance, and the problems of overall scale imbalance and class imbalance caused by various data types of the edge nodes can be solved.
In an optional implementation manner, in step S330, the sample balancing processing is performed on each edge node based on the Z value corresponding to the sample number of each edge node, and the following manner may be further adopted to implement: when the Z value corresponding to the number of the samples of the edge node exceeds a first preset threshold value, taking the edge node as a down-sampling edge node so as to perform sample down-sampling on the down-sampling edge node; when the Z value corresponding to the number of the samples of the edge node is lower than a second preset threshold value, taking the edge node as an up-sampling edge node so as to enable the up-sampling edge node to perform sample up-sampling; the first preset threshold is larger than the second preset threshold.
The following relationship may exist between the first preset threshold and the second preset threshold: tau isa＝-(1/τd) Wherein, τaRepresenting a second preset threshold, τdRepresenting a first preset threshold.
It should be noted that, when the edge node does not satisfy the comparison condition between the first preset threshold and the second preset threshold, the edge node is neither an up-sampling node nor a down-sampling node, and then the sample of the edge node is not balanced.
The edge node determines whether the edge node is an up-sampling node or a down-sampling node so as to clarify the type of the edge node to be processed, and the edge node performs corresponding down-sampling or up-sampling processing according to the type of the edge node to be processed, so that the problem of unbalance of the edge node sample is solved to a certain extent.
In an alternative embodiment, the sampling scales of the up-sampling edge node and the down-sampling edge node can also be determined by: determining the down-sampling scale of the down-sampling edge node according to the number of samples of the down-sampling edge node, the Z value corresponding to the number of samples of the down-sampling edge node, a first preset threshold, a mean value and a variance, so that the down-sampling edge node performs sample down-sampling according to the down-sampling scale; and determining the up-sampling scale of the up-sampling edge node according to the number of samples of the up-sampling edge node, the Z value corresponding to the number of samples of the up-sampling edge node, a second preset threshold, a mean value and a variance, so that the up-sampling edge node performs sample up-sampling according to the up-sampling scale.
The central server may be pre-initializedA 0 vector of length N represents the sampling scale R of each edge nodead＝[0]N, where N represents the number of edge nodes. When the y-th edge node is a down-sampled edge node, the down-sampling scale of the edge node may be
The edge node may scale its samples to Rad[y]Down-sampling; when the y-th edge node is an upsampled edge node, the upsampled scale of the upsampled edge node may be
The edge node may scale its samples to Rad[y]The up-sampling of (2) is to expand the samples of the edge node, and the expansion can be carried out by adopting a random displacement, rotation and shearing mode. Wherein, CyIs the number of samples, Z, of the edge nodeyFor the Z value, tau, corresponding to the number of samples of the edge nodedIs a first preset threshold, τaIs a second preset threshold, mu is the mean and sigma is the variance.
In the process, the scale parameters of up-sampling or down-sampling are provided for the sample balance processing operation of the edge nodes, so that the relatively accurate regulation and control of the sample-saving samples are realized.
It should be noted that, after the synchronous balance processing is completed, the sample data of each edge node may be subjected to the scrambling processing, so as to implement further balance processing on the sample data.
Step S220, obtaining learning model parameters corresponding to each node scheduler, where the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to a sample of a target edge node, and the target edge node is a node allocated to the node scheduler from each edge node.
After sample data balance is carried out on all edge nodes, the sample data updated by the edge nodes can participate in the subsequent process. The method is characterized in that each edge node is distributed to a plurality of node schedulers, and the distribution principle can be that the internal node of each node scheduler does not exceed the upper limit of the volume and the sample data distribution of the internal node of the node scheduler is nearly uniformly distributed to the maximum extent.
When assigning each edge node to several node schedulers, one node scheduler may be selected first and then all edge nodes are traversed until the node scheduler assignment is complete. An edge node assigned to a node scheduler, i.e. a target edge node, may be an internal node of the node scheduler.
The target edge node may be a node determined from the edge nodes according to the KL divergence of the node scheduler. The KL divergence, also known as the Kullback-Leibler divergence, is an asymmetric measure of the difference between two probability distributions (probability distributions). Specifically, the target edge node may be allocated to the node scheduler in the following manner.
First, a node scheduler S may be initializedmediatorAnd a set S of edge nodes to be distributedclient. If the node scheduler Smediato％Number of edge nodes | m! currently allocated<Gamma, where gamma is the maximum capacity of the node scheduler, then for SclientIs traversed. For each edge node i, calculating KL divergence and solving an edge node k which enables KL divergence to be minimum, namely k ← miniDKL((Pm+Pi)||Pu) And assigning the edge node k to the node scheduler SmediatorUntil m | ═ γ. Wherein, PmIs a node scheduler Smediato％Current sample data distribution, PiIs the sample data distribution of the edge node i, PuIs a uniform distribution of sample data. Then, initializing a new node scheduler, and repeating the above steps until all edge nodes finish the allocation of the node scheduler.
It should be noted that, when the edge node is allocated to the node scheduler based on the KL divergence, a greedy algorithm may be used, that is, as long as the KL divergence is smaller than a certain large value (e.g., infinite), the edge node k is allocated to the node scheduler, so that the computation cost required for allocation is substantially reduced.
After the nodes are allocated to the node schedulers, the preset learning model may be iteratively trained according to the samples of the target edge nodes of each node scheduler.
It should be noted that, for each node scheduler, the initialized preset learning model may be used to sequentially iterate the owned target edge nodes, that is, a gradient descent is performed, and after γ SGDs (random gradient descent) are completed, the node scheduler may obtain the latest learning model parameters as the learning model parameters subsequently participating in the multi-task learning.
And step S230, performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to each node scheduler.
The node scheduling method is applied to multi-task federal learning, the problem that the federal learning using a node scheduler cannot effectively solve the problem that the node sample data is highly non-independent and has the same distribution characteristic is solved, and the performance of the federal learning result in the scene of unbalanced node samples is greatly improved.
In an optional implementation manner, the above-mentioned performing multi-task learning on the preset learning model by using the learning model corresponding to each node scheduler may specifically be implemented by the following steps: and establishing a model matrix and a model relation matrix corresponding to the model matrix by adopting the learning model parameters corresponding to each node scheduler, and updating the model matrix based on the model relation matrix to obtain a target model matrix.
In the process, the target model matrix is obtained by updating the model matrix, and the multi-task learning of the learning model is realized. Because the node scheduling integrates the samples of all edge nodes, and the model matrix is constructed through the learning model parameters corresponding to all the node schedulers, compared with the traditional construction aiming at each edge node, the redundant calculation can be obviously reduced.
In an optional implementation manner, the updating the model matrix based on the model relationship matrix to obtain the target model matrix may be implemented in the following manner: constructing a target optimization function of multi-task learning by adopting a model matrix and a model relation matrix; and updating the model matrix, and taking the model matrix when the target function is optimal as the target model matrix.
The multi-task learning is a process of performing optimization solution on an objective optimization function according to a model matrix and a model relation matrix which are iterated continuously. The constructed objective optimization function may be:
where W is a model matrix of d m, W ═ W (W1,w2,…,wm) (ii) a Omega is a relation matrix of m by m and represents the relation between m models; w is atIs a model parameter of the node scheduler t, and the parameter dimension is d; x is the number oftAnd ytRespectively, the data samples corresponding to the node scheduler t and the related data label information corresponding to the node scheduler t, the length of which is nt；lt() Representing the loss function. Wherein the content of the first and second substances,
is an L2 regularized expression whose purpose is to control the parameter range, prevent model overfitting, λ1And λ2For regularizing the parameter, λ1>0，λ2>0。
Usually, the update of W requires the support of a model relation matrix, and the update of Ω requires the support of W, and here, the update can be performed by alternately fixing W and Ω.
In the process, the optimized objective function is optimized and solved by constructing the objective optimization function, so that an objective model matrix meeting the solving conditions of the objective optimization function is obtained, multi-task learning is completed, and the precision of the learning model is improved.
In an optional implementation manner, the foregoing may be specifically implemented by the following manners when the model matrix is updated: and performing one round of updating on the model matrix by taking a single node scheduler as a unit to obtain a new model matrix, so that the target edge node corresponding to each node scheduler performs a new round of iterative training on the preset learning model based on the new model matrix.
The edge nodes in the node scheduler are firstly trained to achieve local balance, so that the scale of the model matrix is reduced, the model matrix is updated by taking the node scheduler as a unit, and the calculation scale of a single round can be greatly reduced.
In an alternative embodiment, a round of updating is performed on the model matrix by taking a single node scheduler as a unit, and the round of updating comprises the following steps: and sequentially updating the subproblem parameters corresponding to the multi-task learning by taking the single node scheduler as a unit, updating the model relation matrix based on the updated subproblem parameters, and updating the model matrix based on the updated model relation matrix.
During the first time of matrix optimization, the subproblem parameters of the multitask learning can be initialized. And repeatedly executing the updating process of the model matrix by taking the single node scheduler as a unit to complete a round of updating so that the sample convergence or the iteration times reach the maximum value. The obtained updated new model matrix can be used as a result of one-round learning and returned to each node scheduler, and a new round of iterative training is performed by the target edge node corresponding to the node scheduler.
In addition, when the first matrix optimization is performed, the dual problem variable α corresponding to the sub-problem parameter may be initialized in advance. And (4) through calculating the multi-task learning subproblems, updating dual problem variables corresponding to subproblem parameters by taking a single node scheduler as a unit. Let the dual problem variable of the node scheduler t be αtBy calculating the dual problem variable alphatIs optimized by the quantity Δ αtObtaining the dual problem variable alpha corresponding to the updated sub-problem parametert', wherein αt′＝αt+ΔαtThen returns and alpha to the node scheduler ttCorresponding update information vtWherein v istSatisfy vt＝XtΔαtX in this casetIs formed by the above-mentioned xtAnd ytThe composed data vector. Node scheduler t getsvtThen, based on the data vector XtObtaining updated alpha ', and then obtaining the latest learning model w (alpha') through the local learning model w of each node scheduler. And when all the participated node schedulers finish the step, uploading the new model, and finishing one round of updating, so that the central server can obtain a new model matrix.
After finishing one round of updating, the central server returns the model matrix after one round of updating to the target edge nodes under the node schedulers, each node scheduler iterates the owned target edge nodes in sequence to update the learning model parameters, when all the target edge nodes owned by the node schedulers are updated, the latest learning model parameters are obtained and submitted to the central server, the central server updates the model matrix for the new round, and then the steps are repeated until the multi-task learning problem is met, so that the target optimization function is optimal.
As shown in fig. 4, a flowchart of the multi-task learning based on the Z value is provided, which specifically includes the following steps:
step S401: starting;
step S402: the central server initializes a global learning model;
step S403: the central server initializes the optimizer;
step S404: the central server calculates the Z value according to the sample scale information submitted by the edge node and performs data balance;
step S405: calculating and distributing target edge nodes by each node scheduler based on KL divergence;
step S406: the node scheduler internal target edge node performs local iteration on the global learning model (scheduler model);
step S407: the central server initializes a multi-task learning model matrix W and a relation matrix omega;
step S408: sequentially updating and optimizing W and omega by taking a single node scheduler as a unit;
step S409: judging whether W converges on a node scheduler sample or whether the iteration number reaches the maximum value;
step S410: the node dispatcher distributes the last returned model to a target edge node, and the target edge node performs local verification on the model;
step S411: and (6) ending.
As shown in fig. 5, a block diagram of a multi-task learning system for allocating edge nodes to a node scheduler is provided, and in combination with the idea of node scheduling, a data down-sampling/up-sampling operation is performed on original data of edge nodes according to a distribution of node samples, and then the edge nodes are rearranged by using node schedulers (e.g., scheduler 1, scheduler 2, and scheduler 3). And (4) carrying out SGD iterative training on the target edge nodes in each node scheduler in sequence to finish updating of model parameters. Next, a model matrix W and a model relation matrix Ω are initialized according to the distribution of each node scheduler, and FMTL (federal Multi-Task Learning) is performed to sequentially optimize the W matrix in units of a single node scheduler. And after one round of matrix optimization is completed, distributing the new model matrix to each node scheduler, and entering the next round of learning.
In addition, the multi-task learning method based on the edge nodes can enhance the adaptability of the federal learning model to small Internet of things equipment. Because the small Internet of things equipment has the characteristics of multiple types, non-uniform data scale and different and independent data distribution in some edge computing scenes based on the Internet of things equipment, the method for combining node scheduling and multi-task learning in the disclosure can be used for finishing the individualized machine learning model training of batches aiming at a large number of small Internet of things equipment.
The multi-task learning method based on the edge nodes can enhance the accuracy of the federal learning model, so that the method can be applied to time delay prediction and other scenes. Due to the influence of statistical heterogeneity, the conventional federal learning, namely the federal learning for training a universal model, may have a situation of difficult convergence, and finally, the result is that the accuracy of the output model is low, or the accuracy of a local model is far higher than the global accuracy. In some communication scenarios, predicting latency tends to be less tolerant of errors, and therefore requires greater model accuracy. According to the method, the model individuation of federal learning is achieved through multi-task learning, and the model precision can be greatly improved so as to meet part of design requirements such as a time delay prediction model.
Exemplary embodiments of the present disclosure also provide an edge node-based multitask learning apparatus, as shown in fig. 6, the edge node-based multitask learning apparatus 600 may include:
the sample balance processing module 610 is configured to perform sample balance processing on each edge node according to the sample number set of each edge node, and send a sample balance processing result of each edge node to a corresponding edge node, so that each edge node performs up-sampling or down-sampling according to the sample balance processing result;
a model parameter obtaining module 620, configured to obtain learning model parameters corresponding to each node scheduler, where the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to a sample of a target edge node, and the target edge node is a node allocated to the node scheduler from each edge node;
and a multi-task learning module 630, configured to perform multi-task learning on the preset learning model by using the learning model parameters corresponding to each node scheduler.
In an alternative embodiment, the sample balance processing module 610 may include: the mean variance determining module is used for determining the mean and the variance of the sample number set according to the sample number set of each edge node; the Z value determining module is used for determining the Z value corresponding to the sample number of each edge node according to the mean value and the variance; and the balance processing submodule is used for carrying out sample balance processing on each edge node based on the Z value corresponding to the sample number of each edge node.
In an alternative embodiment, the balancing processing sub-module may be configured to: when the Z value corresponding to the number of the samples of the edge node exceeds a first preset threshold value, taking the edge node as a down-sampling edge node so as to perform sample down-sampling on the down-sampling edge node; when the Z value corresponding to the number of the samples of the edge node is lower than a second preset threshold value, taking the edge node as an up-sampling edge node so as to enable the up-sampling edge node to perform sample up-sampling; the first preset threshold is larger than the second preset threshold.
In an optional implementation, the balancing processing sub-module may be further configured to: determining the down-sampling scale of the down-sampling edge node according to the number of samples of the down-sampling edge node, the Z value corresponding to the number of samples of the down-sampling edge node, a first preset threshold, a mean value and a variance, so that the down-sampling edge node performs sample down-sampling according to the down-sampling scale; and determining the up-sampling scale of the up-sampling edge node according to the number of samples of the up-sampling edge node, the Z value corresponding to the number of samples of the up-sampling edge node, a second preset threshold, a mean value and a variance, so that the up-sampling edge node performs sample up-sampling according to the up-sampling scale.
In an alternative embodiment, the multi-task learning module 630 may include: and the target model matrix obtaining module is used for establishing a model matrix and a model relation matrix corresponding to the model matrix by adopting the learning model parameters corresponding to each node scheduler, and updating the model matrix based on the model relation matrix to obtain the target model matrix.
In an optional implementation manner, the target model matrix obtaining module may further include: the optimization function construction module is used for constructing a target optimization function of multi-task learning by adopting the model matrix and the model relation matrix; and the matrix updating module is used for updating the model matrix and taking the model matrix when the target function is optimal as the target model matrix.
In an optional implementation manner, the matrix updating module may further include: and the updating submodule is used for carrying out one round of updating on the model matrix by taking a single node scheduler as a unit to obtain a new model matrix so as to enable the target edge node corresponding to each node scheduler to carry out a new round of iterative training on the preset learning model based on the new model matrix.
In an alternative embodiment, the update submodule may be configured to: and sequentially updating the subproblem parameters corresponding to the multi-task learning by taking the single node scheduler as a unit, updating the model relation matrix based on the updated subproblem parameters, and updating the model matrix based on the updated model relation matrix.
In an alternative embodiment, the target edge node in the model parameter obtaining module 620 may be a node determined from edge nodes according to the KL divergence of the node scheduler.
The specific details of each part in the above-mentioned edge node-based multitask learning apparatus 600 have been described in detail in the method part embodiment, and details that are not disclosed may refer to the method part embodiment, and thus are not described again.
Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the above-described edge node-based multitask learning method of the present specification. In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing an electronic device to perform the steps according to various exemplary embodiments of the disclosure described in the above-mentioned "exemplary methods" section of this specification, when the program product is run on the electronic device. The program product may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Exemplary embodiments of the present disclosure also provide an electronic device capable of implementing the edge node-based multitask learning method. An electronic device 700 according to such an exemplary embodiment of the present disclosure is described below with reference to fig. 7. The electronic device 700 shown in fig. 7 is only an example and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 7, electronic device 700 may take the form of a general purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one memory unit 720, a bus 730 that connects the various system components (including the memory unit 720 and the processing unit 710), and a display unit 740.
The memory unit 720 stores program code that may be executed by the processing unit 710 to cause the processing unit 710 to perform steps according to various exemplary embodiments of the present disclosure as described in the "exemplary methods" section above in this specification. For example, processing unit 710 may perform any one or more of the method steps of fig. 2-4.
The storage unit 720 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)721 and/or a cache memory unit 722, and may further include a read only memory unit (ROM) 723.
The memory unit 720 may also include programs/utilities 724 having a set (at least one) of program modules 725, such program modules 725 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 700, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 700 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 750. Also, the electronic device 700 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 760. As shown, the network adapter 760 communicates with the other modules of the electronic device 700 via the bus 730. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary embodiments of the present disclosure.
Furthermore, the above-described figures are merely schematic illustrations of processes included in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit, according to exemplary embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the following claims.
Claims (12)
1. A multitask learning method based on edge nodes is characterized by comprising the following steps:
carrying out sample balance processing on each edge node according to the sample number set of each edge node, and sending the sample balance processing result of each edge node to the corresponding edge node so that each edge node carries out up-sampling or down-sampling according to the sample balance processing result;
acquiring learning model parameters corresponding to each node scheduler, wherein the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to samples of target edge nodes, and the target edge nodes are nodes distributed to the node schedulers from the edge nodes;
and performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to the node schedulers.
2. The method of claim 1, wherein the sample balancing each edge node according to its set of sample quantities comprises:
determining a mean and a variance of the sample number set according to the sample number set of each edge node;
determining a Z value corresponding to the sample number of each edge node according to the mean value and the variance;
and carrying out sample balance processing on each edge node based on the Z value corresponding to the sample number of each edge node.
3. The method according to claim 2, wherein the performing sample balancing processing on each edge node based on the Z value corresponding to the sample number of each edge node comprises:
when a Z value corresponding to the number of samples of an edge node exceeds a first preset threshold value, taking the edge node as a down-sampling edge node so as to perform sample down-sampling on the down-sampling edge node;
when a Z value corresponding to the number of samples of an edge node is lower than a second preset threshold value, taking the edge node as an up-sampling edge node so as to enable the up-sampling edge node to perform sample up-sampling;
wherein the first preset threshold is greater than the second preset threshold.
4. The method of claim 3, further comprising:
determining a down-sampling scale of the down-sampling edge node according to the number of samples of the down-sampling edge node, a Z value corresponding to the number of samples of the down-sampling edge node, the first preset threshold, the mean value and the variance, so that the down-sampling edge node performs sample down-sampling according to the down-sampling scale;
and determining the up-sampling scale of the up-sampling edge node according to the number of samples of the up-sampling edge node, the Z value corresponding to the number of samples of the up-sampling edge node, the second preset threshold, the mean value and the variance, so that the up-sampling edge node performs sample up-sampling according to the up-sampling scale.
5. The method according to claim 1, wherein the performing multi-task learning on the preset learning model by using the learning model parameters corresponding to the node schedulers comprises:
and establishing a model matrix and a model relation matrix corresponding to the model matrix by adopting the learning model parameters corresponding to each node scheduler, and updating the model matrix based on the model relation matrix to obtain a target model matrix.
6. The method of claim 5, wherein updating the model matrix based on the model relationship matrix to obtain a target model matrix comprises:
constructing a target optimization function of the multi-task learning by adopting the model matrix and the model relation matrix;
and updating the model matrix, and taking the model matrix when the target function is optimal as the target model matrix.
7. The method of claim 6, wherein said updating the model matrix comprises:
and performing one round of updating on the model matrix by taking a single node scheduler as a unit to obtain a new model matrix, so that the target edge node corresponding to each node scheduler performs a new round of iterative training on the preset learning model based on the new model matrix.
8. The method of claim 7, wherein the performing a round of updating on the model matrix in units of a single node scheduler comprises:
and sequentially updating the sub-problem parameters corresponding to the multi-task learning by taking a single node scheduler as a unit, updating the model relation matrix based on the updated sub-problem parameters, and updating the model matrix based on the updated model relation matrix.
9. The method according to claim 1, wherein the target edge node is a node determined from the edge nodes according to a KL divergence of the node scheduler.
10. An edge node-based multitask learning apparatus, comprising:
the sample balance processing module is used for carrying out sample balance processing on each edge node according to the sample number set of each edge node and sending the sample balance processing result of each edge node to the corresponding edge node so as to enable each edge node to carry out up-sampling or down-sampling according to the sample balance processing result;
a model parameter obtaining module, configured to obtain learning model parameters corresponding to each node scheduler, where the learning model parameters corresponding to the node schedulers are obtained by performing iterative training on a preset learning model according to a sample of a target edge node, and the target edge node is a node allocated to the node scheduler from among the edge nodes;
and the multi-task learning module is used for performing multi-task learning on the preset learning model by adopting the learning model parameters corresponding to the node schedulers.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 9.
12. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the method of any of claims 1 to 9 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111321824.5A CN114020469A (en) | 2021-11-09 | 2021-11-09 | Edge node-based multi-task learning method, device, medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111321824.5A CN114020469A (en) | 2021-11-09 | 2021-11-09 | Edge node-based multi-task learning method, device, medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114020469A true CN114020469A (en) | 2022-02-08 |
Family
ID=80062740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111321824.5A Pending CN114020469A (en) | 2021-11-09 | 2021-11-09 | Edge node-based multi-task learning method, device, medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114020469A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114662340A (en) * | 2022-04-29 | 2022-06-24 | 烟台创迹软件有限公司 | Weighing model scheme determination method and device, computer equipment and storage medium |
-
2021
- 2021-11-09 CN CN202111321824.5A patent/CN114020469A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114662340A (en) * | 2022-04-29 | 2022-06-24 | 烟台创迹软件有限公司 | Weighing model scheme determination method and device, computer equipment and storage medium |
| CN114662340B (en) * | 2022-04-29 | 2023-02-28 | 烟台创迹软件有限公司 | Weighing model scheme determination method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220391771A1 (en) | Method, apparatus, and computer device and storage medium for distributed training of machine learning model | |
| CN108089921B (en) | Server for cloud big data operation architecture and operation resource optimization method thereof | |
| US10650008B2 (en) | Parallel scoring of an ensemble model | |
| US11263052B2 (en) | Determining optimal compute resources for distributed batch based optimization applications | |
| US10719366B1 (en) | Dynamic and selective hardware acceleration | |
| CN110058936B (en) | Method, apparatus and computer program product for determining an amount of resources of a dedicated processing resource | |
| US11574254B2 (en) | Adaptive asynchronous federated learning | |
| JP2023508076A (en) | Elastically run machine learning workloads with application-based profiling | |
| WO2022246833A1 (en) | System, method, and medium for elastic allocation of resources for deep learning jobs | |
| CN110737528A (en) | Method and system for reducing computational cost of executing machine learning tasks | |
| CN112181664A (en) | Load balancing method and device, computer readable storage medium and electronic equipment | |
| CN114924745A (en) | Operation method and device of deep learning compiler and electronic equipment | |
| CN114020469A (en) | Edge node-based multi-task learning method, device, medium and equipment | |
| CN115129463A (en) | Computing power scheduling method, device, system and storage medium | |
| US11410023B2 (en) | Lexicographic deep reinforcement learning using state constraints and conditional policies | |
| WO2023077989A1 (en) | Incremental machine learning for a parametric machine learning model | |
| WO2023025033A1 (en) | Intelligent dataset slicing during microservice handshaking | |
| CN113254200B (en) | Resource arrangement method and intelligent agent | |
| CN115344359A (en) | Computing power resource allocation method, device, computer readable storage medium and equipment | |
| CN114090247A (en) | Method, device, equipment and storage medium for processing data | |
| US10680912B1 (en) | Infrastructure resource provisioning using trace-based workload temporal analysis for high performance computing | |
| CN113342781A (en) | Data migration method, device, equipment and storage medium | |
| US20210089932A1 (en) | Forecasting values utilizing time series models | |
| CN111694670A (en) | Resource allocation method, device, equipment and computer readable medium | |
| TWI822290B (en) | Computer-implemented method, computer system and computer program product related to federated learning for training machine learning models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |