CN115002215B - Cloud government enterprise oriented resource allocation model training method and resource allocation method - Google Patents

Cloud government enterprise oriented resource allocation model training method and resource allocation method Download PDF

Info

Publication number
CN115002215B
CN115002215B CN202210376475.5A CN202210376475A CN115002215B CN 115002215 B CN115002215 B CN 115002215B CN 202210376475 A CN202210376475 A CN 202210376475A CN 115002215 B CN115002215 B CN 115002215B
Authority
CN
China
Prior art keywords
resource allocation
model
cloud
training
reinforcement learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210376475.5A
Other languages
Chinese (zh)
Other versions
CN115002215A (en
Inventor
赵永利
李卓桐
李亚杰
郁小松
张�杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202210376475.5A priority Critical patent/CN115002215B/en
Publication of CN115002215A publication Critical patent/CN115002215A/en
Application granted granted Critical
Publication of CN115002215B publication Critical patent/CN115002215B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04JMULTIPLEX COMMUNICATION
    • H04J3/00Time-division multiplex systems
    • H04J3/16Time-division multiplex systems in which the time allocation to individual channels within a transmission cycle is variable, e.g. to accommodate varying complexity of signals, to vary number of channels transmitted
    • H04J3/1605Fixed allocated frame structures
    • H04J3/1652Optical Transport Network [OTN]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/16Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a cloud government enterprise oriented resource allocation model training method, which is applied to an SDN-based cloud network, wherein the cloud network comprises a controller, a plurality of data centers and an optical transmission network connected with the data centers, and the method comprises the following steps: acquiring cloud network resources from the data center and the optical transmission network by using the controller, constructing a resource abstract model according to the cloud network resources, and initializing a pre-constructed reinforcement learning neural network model according to the resource abstract model; constructing a service training set; and training the reinforcement learning neural network model by utilizing the service training set to obtain a resource allocation model. By applying the method provided by the application, the resource allocation model which can be aimed at the needs of the tenants is obtained, the utilization rate of cloud network resources can be effectively improved, the development and maintenance cost is reduced while the data isolation needs of the tenants are ensured, and the reasonable allocation and on-demand scheduling of the cloud network resources are realized.

Description

Cloud government enterprise oriented resource allocation model training method and resource allocation method
Technical Field
The application relates to the technical field of cloud networks, in particular to a cloud government and enterprise oriented resource allocation model training method and a resource allocation method.
Background
With the continuous development of the cloud computing field, enterprises have become a trend to deploy information systems in the cloud, and cloud awareness and capability of the enterprises are continuously enhanced. An optical transport network (Optical Transport Network, OTN) with high-quality transport pipelines continuously changes along with the requirements of cloud computing, so that the cooperative capability of the cloud network is continuously deepened, and the cloud network convergence is gradually achieved. Some cloud network tenants (such as cloud government and enterprise tenants) have high privacy of partial data, and the problem of data leakage safety does not occur, so that network operators and cloud manufacturers are generally required to provide cloud services with differentiated privacy safety by utilizing a multi-tenant technology, different privacy safety guarantee measures bring different costs, and reasonable distribution and on-demand scheduling of network transmission, calculation and storage resources are also brought by utilizing different data centers of the same cloud manufacturer to provide services for the cloud network tenants.
Disclosure of Invention
Therefore, the application aims to provide a cloud government enterprise oriented resource allocation model training method and a resource allocation method.
Based on the above object, the present application provides a cloud-government-enterprise-oriented resource allocation model training method, which is applied to an SDN-based cloud network, wherein the cloud network comprises a controller, a plurality of data centers and an optical transport network connecting the plurality of data centers, and the method comprises: acquiring cloud network resources from the data center and the optical transmission network by using the controller, constructing a resource abstract model according to the cloud network resources, and initializing a pre-constructed reinforcement learning neural network model according to the resource abstract model; constructing a service training set; and training the reinforcement learning neural network model by utilizing the service training set to obtain a resource allocation model.
Optionally, the reinforcement learning neural network model is constructed based on a graph neural network and reinforcement learning, and the initializing the reinforcement learning neural network model constructed in advance according to the resource abstraction model includes: and initializing hidden layer states of nodes and edges in the reinforcement learning neural network model according to the resource abstraction model.
Optionally, the constructing the service training set includes: acquiring a plurality of virtual services; classifying the virtual services according to the data isolation requirements of the virtual services; constructing a source-sink node pair and a source-sink node path from each classified virtual service to each data center; and taking all the source-sink node pairs and the source-sink node paths as the service training set.
Optionally, the classifying the virtual service according to the data isolation requirement of the virtual service includes: dividing the virtual services belonging to the same virtual tenant into a large class; in each of the major classes, the virtual traffic having the same isolation level is divided into a minor class.
Optionally, the training the reinforcement learning neural network model by using the service training set to obtain a resource allocation model includes: performing multiple rounds of iterative training on the reinforcement learning neural network model, wherein each time the iterative training performs training by applying all source-destination node paths corresponding to one virtual service in the service training set, different times of iterative training applies different virtual services, and for each time of iterative training, performing the following operations: sequencing all source and sink node paths according to the length from short to long, and selecting the first n source and sink node paths as candidate paths, wherein n is an integer greater than 1; performing resource allocation on each candidate path to obtain a plurality of allocation behaviors; modifying the hidden layer state of the reinforcement learning neural network model according to the allocation behavior, and outputting a Q value via the reinforcement learning neural network model; calculating the reward and punishment value of the distribution behavior with the maximum Q value, and accumulating the reward and punishment value into a total reward and punishment value; and responding to the total reward and punishment value reaching a first threshold value and/or the iterative training frequency reaching a second threshold value, and finishing the iterative training to obtain the resource allocation model, wherein the first threshold value and the second threshold value are preset.
Optionally, calculating the punishment value of the distribution behavior with the largest Q value includes: determining the reward and punishment value according to the following formula:
wherein, reward is the punishment value,satisfaction of a punishment value, phi, for the data isolation requirements of the allocation behavior cost And (3) prizing the data overhead cost of the distribution behavior.
Based on the same inventive concept, the application also provides a method for carrying out resource allocation by applying the resource allocation model obtained by any cloud government enterprise oriented resource allocation model training method, which comprises the following steps: acquiring a service request of a tenant; and inputting the service request into the resource allocation model, and outputting a resource allocation scheme by using the resource allocation model.
Based on the same inventive concept, the application also provides a cloud government and enterprise oriented resource allocation model training device, which is applied to an SDN-based cloud network, wherein the cloud network comprises a plurality of data centers of a controller and an optical transmission network connected with the plurality of data centers, and the device comprises: the initialization module is configured to acquire cloud network resources from the data center and the optical transmission network by using the controller, construct a resource abstraction model according to the cloud network resources, and initialize a pre-constructed reinforcement learning neural network model according to the resource abstraction model; the training set construction module is configured to construct a business training set; and the training module is configured to train the reinforcement learning neural network model by utilizing the service training set so as to obtain a resource allocation model.
Based on the same inventive concept, the application also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, and is characterized in that the processor realizes any one of cloud government enterprise oriented resource allocation model training methods when executing the program.
Based on the same inventive concept, the application also provides a non-transitory computer readable storage medium, which stores computer instructions, and is characterized in that the computer instructions are used for enabling a computer to execute any cloud government enterprise oriented resource allocation model training method.
As can be seen from the foregoing, the method for training a resource allocation model provided by the present application is applied to an SDN-based cloud network, where the cloud network includes a controller, a plurality of data centers, and an optical transport network connecting the plurality of data centers, and the method includes: acquiring cloud network resources from the data center and the optical transmission network by using the controller, constructing a resource abstract model according to the cloud network resources, and initializing a pre-constructed reinforcement learning neural network model according to the resource abstract model; constructing a service training set; and training the reinforcement learning neural network model by utilizing the service training set to obtain a resource allocation model. By applying the method provided by the application, the resource allocation model which can be aimed at the needs of the tenants is obtained, the utilization rate of cloud network resources can be effectively improved, the development and maintenance cost is reduced while the data isolation needs of the tenants are ensured, and the reasonable allocation and on-demand scheduling of the cloud network resources are realized.
Drawings
In order to more clearly illustrate the technical solutions of the present application or related art, the drawings that are required to be used in the description of the embodiments or related art will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort to those of ordinary skill in the art.
Fig. 1 is a schematic diagram of an application scenario of a cloud-oriented government enterprise resource allocation model training method according to an embodiment of the present application;
fig. 2 is a flow chart of a cloud government enterprise oriented resource allocation model training method according to an embodiment of the present application;
fig. 3 is a flow refinement diagram of a cloud-facing government enterprise resource allocation model training method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a resource abstraction model according to an embodiment of the present application;
fig. 5 is a schematic diagram of a multi-tenant data isolation technique according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a cloud government enterprise-oriented resource allocation model training device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a resource allocation device according to an embodiment of the present application;
fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the application.
Detailed Description
The present application will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present application more apparent.
It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present application should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present application belongs. The terms "first," "second," and the like, as used in embodiments of the present application, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.
An embodiment of the present application provides a cloud-government-enterprise-oriented resource allocation model training method, which is applied to a cloud network based on an SDN (software defined network), as shown in fig. 1, where the cloud network includes a cloud network convergence controller based on the SDN, a plurality of data centers, a plurality of tenants, and an optical transport network connected to the plurality of data centers, and OSU (Optical Service Unit) is an optical service unit state in the optical transport network, as shown in fig. 2, where the method includes:
step S101, acquiring cloud network resources from the data center and the optical transmission network by using the controller, constructing a resource abstraction model according to the cloud network resources, and initializing a pre-constructed reinforcement learning neural network model according to the resource abstraction model. The controller in the embodiment of the application is a cloud network convergence controller based on SDN, and can be used for visualizing network resource states of an optical transmission network and a plurality of cloud data centers and has uniform cloud network control capability and interface protocol; the controller can centrally manage and monitor tenant service requests, and realize the cloud government and enterprise oriented resource allocation model training method and the resource allocation method in the embodiment of the application.
Step S102, constructing a service training set. A training set is built aiming at the service requirements of tenants, so that a resource allocation model for training based on the training set can better meet the service requirements of the tenants.
And step S103, training the reinforcement learning neural network model by utilizing the service training set to obtain a resource allocation model. In a specific embodiment, the resource allocation model is configured to provide resource allocation services for cloud-based government-enterprise tenants.
The resource allocation model training method provided by the embodiment comprises the following steps: constructing a resource abstraction model according to the cloud network resources, initializing a pre-constructed reinforcement learning neural network model according to the resource abstraction model, and constructing a service training set aiming at tenant demands, so that a resource allocation model trained based on the training set better meets the tenant demands; by applying the resource allocation model, the utilization rate of cloud network resources can be effectively improved, the development and maintenance cost is reduced while the data isolation requirements of tenants are ensured, and the reasonable allocation and on-demand scheduling of the cloud network resources are realized.
In some embodiments, the reinforcement learning neural network model is constructed based on a graph neural network and reinforcement learning. In the prior art, the time space complexity of the algorithm based on the resource optimization model of the heuristic algorithm is huge, the cost is high, and the heuristic algorithm only obtains one solution and needs to be independently executed each time; the resource optimization model based on the multi-modal network and the reinforcement learning is characterized in that the resources and constraints needed to be considered in the network are more, the image recognition process is difficult, the training process is slow, the learning effect is difficult to ensure, in addition, the reinforcement learning is essentially to learn the generated multi-modal network image such as Euclidean data, but the multi-modal network topology is non-Euclidean data, if the network node position is changed but the connection structure is not changed, the non-Euclidean data is unchanged, but the generated multi-modal image is a new characteristic for image recognition, so that the reinforcement learning for learning the multi-modal network image does not have generalization. In view of this, the embodiment of the application constructs the reinforcement learning neural network model based on the graph neural network and reinforcement learning. The graph neural network (Graph Neural Network, GNN) is a neural network directly running on a graph structure, and the embodiment of the application utilizes the non-Euclidean data abstraction capability of the graph neural network to construct a cloud network integrated resource allocation model, so that the technical problems brought by the common algorithm in the prior art can be solved, and the reasonable allocation and on-demand scheduling of cloud network resources during resource allocation by utilizing the resource allocation model provided by the embodiment of the application are ensured.
In some embodiments, initializing a pre-built reinforcement learning neural network model according to the resource abstraction model in step S101 includes: and initializing hidden layer states of nodes and edges in the reinforcement learning neural network model according to the resource abstraction model. Initializing hidden layer states of nodes and edges in the reinforcement learning neural network model according to the resource abstraction model, and preparing for modifying the hidden layer states of the reinforcement learning neural network in the subsequent model training.
In a specific embodiment, as shown in fig. 3, the step S101 may be refined as follows: step S1011, collecting cloud network resources: firstly, a cloud network fusion controller based on SDN needs topology information and various heterogeneous resource information of an OTN network and a cloud data center (namely the data center in the embodiment of the application); step S1012, a cloud-network integrated resource abstraction model is established (as shown in fig. 4, the meaning of each parameter in fig. 4 is shown in table 1, in fig. 4, node 1, node 2 and node 3 are all transmission network nodes, and node 4 is a data center abstraction node): constructing a cloud network integrated resource abstraction model based on topology information and resource states of the OTN and the cloud data center; step S1013, initializing a resource optimization model: initializing hidden layer states of nodes and edges in the GNN according to the resource abstraction model, and completing construction and initialization of the reinforcement learning neural network model based on the GNN and the RL. The topology information includes geographical positions of tenants and the cloud data center, and the heterogeneous resource information includes, but is not limited to, OSU of each link of the optical transport network, computing resources in the cloud data center, number of storage resources and storage database type.
Table 1: resource abstraction model parameter meaning
In some embodiments, as shown in fig. 3, the step S102 includes:
step S201, a plurality of virtual services are acquired. The virtual service may be pre-constructed, and its data structure is the same as the data structure of the service requested by the tenant in actual use.
Step S202, classifying the virtual services according to the data isolation requirements of the virtual services. The data isolation requirements include a virtual business enterprise class and an isolation level, and in a specific embodiment, the data isolation requirements further include whether the cloud data center needs to provide an independent database or a shared database for each tenant's virtual business. The same enterprises and the same isolation level are ordered/classified together, so that the data services shared in the same enterprises are distributed to the same type of database of the same data center as much as possible, the services with similar security requirements are uniformly stored, tenant requests are reasonably scheduled, and redundant privacy security guarantee measures are reduced.
And step 203, constructing source-sink node pairs and source-sink node paths from the virtual service to each data center according to each classified virtual service, and taking all the source-sink node pairs and the source-sink node paths as the service training set. The position of the source node depends on the position of the service request node of the tenant; when cloud manufacturers provide cloud services for tenants, a plurality of data centers of the same cloud manufacturer can serve as service providing points, the positions of sink nodes depend on the geographic positions of the data centers providing the services, and the source sink node paths are data transmission paths between the source nodes and the sink nodes.
The training set obtained in the steps S201 to S203 can meet the data security isolation requirement of any tenant, and reduces the cost brought by the multi-tenant technology of tenant development and operation data isolation while improving the utilization rate of three resources including calculation, storage and transmission; the classification process uniformly stores the services with similar security requirements, reasonably dispatches tenant requests, and reduces redundant privacy security guarantee measures. And constructing a data set for training according to the data isolation requirement, so that the model can learn the correlation between network resources and tenant services in the subsequent training process, correct the network state or topology and ensure better generalization capability.
The multi-tenant technology is a cloud computing platform technology, and enables a large number of tenants to share software and hardware resources of the same stack, each tenant represents an enterprise, a plurality of users are arranged in the tenant, each tenant can use the resources as required, customized configuration can be carried out on software services, and the use of other tenants is not affected. The data isolation refers to that when a plurality of tenants use the same system, service data of the tenants are stored in an isolated mode, and service data processing of different tenants cannot interfere with each other. The multi-tenant technology needs to realize safe and efficient data isolation, so that the safety of tenant data and the overall performance of a multi-tenant platform are ensured. For multi-tenant database management, providing an independent database and a shared database, wherein the independent database creates an independent database for tenant service, and the data are fully isolated, but the cost and the expenditure of the independent database management are relatively high; the shared database is only distinguished by the identification code field of the tenant, and the mode has low management cost and overhead, but has poor data isolation effect. As shown in fig. 5, service data among different tenants are isolated from each other, each has an independent database, the same tenant is divided into a plurality of sub-tenants with data isolated from each other according to different isolation requirements, each sub-tenant also has an independent database, but service data needed by different tenants such as office infrastructure are shared by the sub-tenants. The service training set obtained in the steps S201 to S203 is applied to model training, and the finally obtained model can reasonably distribute an independent database and a shared database according to different service requirements of different tenants, so that the data isolation effect can be ensured, and the management cost and the expenditure can be reduced.
In some embodiments, the step S202 includes: dividing the virtual services belonging to the same virtual tenant into a large class; in each of the major classes, the virtual traffic having the same isolation level is divided into a minor class. And uniformly storing the services with similar security requirements, reasonably scheduling tenant requests, and reducing redundant privacy security guarantee measures.
In some embodiments, as shown in fig. 3, the step S103 includes:
performing multiple rounds of iterative training on the reinforcement learning neural network model, wherein each time the iterative training performs training by applying all source-destination node paths corresponding to one virtual service in the service training set, different times of iterative training applies different virtual services, and for each time of iterative training, performing the following operations of step S301 to step S305:
step 301, sorting all source and destination node paths according to the length from short to long, selecting the first n source and destination node paths as candidate paths, wherein n is an integer greater than 1.
And step S302, carrying out resource allocation on each candidate path to obtain a plurality of allocation behaviors. In a specific embodiment, the allocating resources to each candidate path includes allocating OSU and cloud resources to each candidate path and selecting a data storage type.
Step S303, modifying the hidden layer state of the reinforcement learning neural network model according to the allocation behavior, and outputting a Q value via the reinforcement learning neural network model.
Step S304, calculating the punishment and punishment value of the distribution behavior with the maximum Q value; in a specific embodiment, the allocation behavior corresponding to the largest Q value is selected as the allocation result by a reinforcement learning epsilon greedy algorithm (epsilon greedy).
And step S305, accumulating the reward and punishment values into total reward and punishment values.
And responding to the total reward and punishment value reaching a first threshold value and/or the iterative training frequency reaching a second threshold value, and finishing the iterative training to obtain the resource allocation model, wherein the first threshold value and the second threshold value are preset.
In a more specific embodiment, the step S303 is specifically: after each resource allocation behavior is finished, modifying information parameters of a hidden layer of the reinforcement learning neural network model according to the allocation behavior, and outputting an output hidden value o of each node after T step sizes are mutually transmitted by nodes by using GNN m+n Output hidden value o of all nodes m+n The required Q value is obtained through a vector product and a layer of fully connected neural network, wherein the T step length is continuously adjusted according to the information parameter during training.
In some embodiments, calculating the punishment value of the distribution behavior with the largest Q value in the step S304 includes: determining the reward and punishment value according to the following formula:
wherein, reward is the punishment value,a data isolation requirement satisfaction punishment value for the allocation behaviour, for example, when resources that do not satisfy the data isolation requirement of an enterprise cannot be allocated or allocated>Is a larger negative number; phi (phi) cost And adding a cost price punishment value of a database type to the data center, wherein the cost price punishment value of the data cost of the distribution behavior is lower than that of the shared database.
Based on the same inventive concept, an embodiment of the present application further provides a method for performing resource allocation by using a resource allocation model obtained by any one of the cloud government enterprise-oriented resource allocation model training methods, as shown in fig. 3, including:
step S401, a tenant sends a service request;
step S402, acquiring a service request of a tenant;
step S403, inputting the service request to the resource allocation model, and outputting a resource allocation scheme by using the resource allocation model, where the resource allocation scheme includes a data center geographic location of the service, an allocated network path, a calculation storage and transmission resource, and a provided data storage type.
According to the resource allocation method provided by the embodiment of the application, the best allocation strategy is obtained by utilizing the trained resource allocation model, so that the utilization rate of cloud network resources can be effectively improved, the development and maintenance cost is reduced while the data isolation requirements of tenants are ensured, the reasonable allocation and on-demand scheduling of cloud network resources are realized, and the utilization rate of heterogeneous resources of tenant service allocation is greatly improved.
In a specific embodiment, the resource allocation method further includes:
acquiring a service request of a government enterprise tenant on the cloud;
and inputting the service request into the resource allocation model, and outputting a resource allocation scheme aiming at the cloud government enterprise tenant by utilizing the resource allocation model.
It should be noted that, the method of the embodiment of the present application may be performed by a single device, for example, a computer or a server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the method of an embodiment of the present application, the devices interacting with each other to accomplish the method.
It should be noted that the foregoing describes some embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Based on the same inventive concept, the application also provides a cloud government enterprise oriented resource allocation model training device, which is applied to an SDN-based cloud network, wherein the cloud network comprises a controller, a plurality of data centers and an optical transmission network connected with the data centers, as shown in fig. 6, and the device comprises:
an initialization module 10 configured to acquire cloud network resources from the data center and the optical transport network by using the controller, construct a resource abstraction model according to the cloud network resources, and initialize a reinforcement learning neural network model constructed in advance according to the resource abstraction model;
a training set construction module 20 configured to construct a business training set;
a training module 30 is configured to train the reinforcement learning neural network model with the business training set to obtain a resource allocation model.
The resource allocation model training device provided in this embodiment includes: the initialization module 10 and the training set construction module 20 construct a resource abstraction model according to the cloud network resources, initialize a pre-constructed reinforcement learning neural network model according to the resource abstraction model, and construct a service training set according to the needs of tenants, so that a resource allocation model trained based on the training set better meets the business needs of the tenants; by applying the resource allocation model, the utilization rate of cloud network resources can be effectively improved, the development and maintenance cost is reduced while the data isolation requirements of tenants are ensured, and the reasonable allocation and on-demand scheduling of the cloud network resources are realized.
In some embodiments, the reinforcement learning neural network model is constructed based on a graph neural network and reinforcement learning, and the initialization module 10 is further configured to: and initializing hidden layer states of nodes and edges in the reinforcement learning neural network model according to the resource abstraction model.
In some embodiments, the training set construction module 20 includes:
an acquisition unit configured to acquire a plurality of virtual services;
the classification unit is configured to classify the virtual service according to the data isolation requirement of the virtual service;
and the construction unit is configured to construct source-destination node pairs and source-destination node paths from the virtual service to each data center according to each classified virtual service, and takes all the source-destination node pairs and the source-destination node paths as the service training set.
In some embodiments, the classification unit is further configured to: dividing the virtual services belonging to the same virtual tenant into a large class; in each of the major classes, the virtual traffic having the same isolation level is divided into a minor class.
In some embodiments, the training module 30 is further configured to: performing multiple rounds of iterative training on the reinforcement learning neural network model, wherein each time the iterative training performs training by applying all source-destination node paths corresponding to one virtual service in the service training set, different times of iterative training applies different virtual services, and for each time of iterative training, performing the following operations:
sequencing all source and sink node paths according to the length from short to long, and selecting the first n source and sink node paths as candidate paths, wherein n is an integer greater than 1;
performing resource allocation on each candidate path to obtain a plurality of allocation behaviors;
modifying the hidden layer state of the reinforcement learning neural network model according to the allocation behavior, and outputting a Q value via the reinforcement learning neural network model;
calculating the reward and punishment value of the distribution behavior with the maximum Q value, and accumulating the reward and punishment value into a total reward and punishment value;
and responding to the total reward and punishment value reaching a first threshold value and/or the iterative training frequency reaching a second threshold value, and finishing the iterative training to obtain the resource allocation model, wherein the first threshold value and the second threshold value are preset.
In some embodiments, said calculating a punishment value for said dispensing action with a maximum Q value comprises:
determining the reward and punishment value according to the following formula:
wherein, reward is the punishment value,satisfaction of a punishment value, phi, for the data isolation requirements of the allocation behavior cost And (3) prizing the data overhead cost of the distribution behavior.
Based on the same inventive concept, the application also provides a device for performing resource allocation by applying the resource allocation model obtained by the cloud government enterprise oriented resource allocation model training device, which corresponds to the method of any embodiment, as shown in fig. 7, and the device comprises:
the acquiring module 40 is configured to acquire a service request of the tenant.
An output module 50 configured to input the service request to the resource allocation model, and to output a resource allocation scheme using the resource allocation model.
The resource allocation device provided by the embodiment of the application obtains the optimal allocation strategy by utilizing the trained resource allocation model, can effectively improve the utilization rate of cloud network resources, reduces development and maintenance costs while guaranteeing the data isolation requirements of tenants, realizes reasonable allocation and on-demand scheduling of the cloud network resources, and greatly improves the utilization rate of heterogeneous resources of tenant service allocation.
For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of each module may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
The device of the above embodiment is used for implementing the resource allocation model training method and the resource allocation method for cloud government enterprises in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Based on the same inventive concept, the application also provides an electronic device corresponding to the method of any embodiment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the cloud government enterprise oriented resource allocation model training method and the resource allocation method when executing the program.
Fig. 8 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 910, a memory 920, an input/output interface 930, a communication interface 940, and a bus 950. Wherein processor 910, memory 920, input/output interface 930, and communication interface 940 implement communication connections among each other within the device via a bus 950.
The processor 910 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 920 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 920 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 920 and executed by processor 910.
The input/output interface 930 is used to connect with input/output modules to achieve information input and output. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
The communication interface 940 is used to connect a communication module (not shown in the figure) to enable communication interaction between the present device and other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 950 includes a path for transferring information between components of the device (e.g., processor 910, memory 920, input/output interface 930, and communication interface 940).
It should be noted that although the above device only shows the processor 910, the memory 920, the input/output interface 930, the communication interface 940, and the bus 950, in the implementation, the device may include other components necessary to achieve normal operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
The electronic device of the foregoing embodiment is configured to implement the cloud-facing government enterprise resource allocation model training method and the cloud-facing government enterprise resource allocation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Based on the same inventive concept, the application also provides a non-transitory computer readable storage medium corresponding to the method of any embodiment, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute the cloud government enterprise oriented resource allocation model training method according to any embodiment.
The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
The computer instructions stored in the storage medium of the foregoing embodiments are used to make the computer execute the cloud-facing resource allocation model training method and the resource allocation method according to any one of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described herein.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the application (including the claims) is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined within the idea of the application, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the application as described above, which are not provided in detail for the sake of brevity.
Additionally, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures, in order to simplify the illustration and discussion, and so as not to obscure the embodiments of the present application. Furthermore, the devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and also in view of the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present application are to be implemented (i.e., such specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative in nature and not as restrictive.
While the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, and the like, which are within the spirit and principles of the embodiments of the application, are intended to be included within the scope of the application.

Claims (8)

1. A cloud-government-enterprise-oriented resource allocation model training method, which is characterized in that the method is applied to an SDN-based cloud network, the cloud network comprises a controller, a plurality of data centers and an optical transport network connecting the plurality of data centers, and the method comprises:
acquiring cloud network resources from the data center and the optical transmission network by using the controller, constructing a resource abstract model according to the cloud network resources, and initializing a pre-constructed reinforcement learning neural network model according to the resource abstract model;
constructing a service training set, comprising:
acquiring a plurality of virtual services;
classifying the virtual services according to the data isolation requirements of the virtual services;
constructing a source-sink node pair and a source-sink node path from each classified virtual service to each data center;
taking all the source-sink node pairs and the source-sink node paths as the service training set;
training the reinforcement learning neural network model by using the service training set to obtain a resource allocation model, wherein the training comprises the following steps:
performing multiple rounds of iterative training on the reinforcement learning neural network model, wherein each time the iterative training is performed by applying all source and destination node paths corresponding to one virtual service in the service training set, different times of iterative training are performed by applying different virtual services, and for each time of iterative training, the following operations are performed:
sequencing all source and sink node paths according to the length from short to long, and selecting the first n source and sink node paths as candidate paths, wherein n is an integer greater than 1;
performing resource allocation on each candidate path to obtain a plurality of allocation behaviors;
modifying a hidden layer state of the reinforcement learning neural network model according to the allocation behavior, and outputting a Q value through the reinforcement learning neural network model;
calculating the reward and punishment value of the distribution behavior with the maximum Q value, and accumulating the reward and punishment value into a total reward and punishment value;
and responding to the total reward and punishment value reaching a first threshold value and/or the iterative training frequency reaching a second threshold value, and finishing the iterative training to obtain the resource allocation model, wherein the first threshold value and the second threshold value are preset.
2. The cloud-based government enterprise resource allocation model training method of claim 1, wherein the reinforcement learning neural network model is constructed based on a graph neural network and reinforcement learning, and wherein initializing the reinforcement learning neural network model constructed in advance according to the resource abstraction model comprises:
and initializing hidden layer states of nodes and edges in the reinforcement learning neural network model according to the resource abstraction model.
3. The cloud-oriented resource allocation model training method of the government enterprises of claim 1, wherein classifying the virtual business according to the data isolation requirement of the virtual business comprises:
dividing the virtual services belonging to the same virtual tenant into a large class;
in each of the major classes, the virtual traffic having the same isolation level is divided into a minor class.
4. The cloud-government-enterprise-oriented resource allocation model training method according to claim 1, wherein calculating the reward and punishment value of the allocation behavior with the largest Q value comprises:
determining the reward and punishment value according to the following formula:
wherein,for the punishment and punishment value, +.>The data isolation requirements for the allocation act satisfy a punishment value,and (3) prizing the data overhead cost of the distribution behavior.
5. A method for allocating resources by using a resource allocation model obtained by the cloud government enterprise-oriented resource allocation model training method according to any one of claims 1 to 4, comprising:
acquiring a service request of a tenant;
and inputting the service request into the resource allocation model, and outputting a resource allocation scheme by using the resource allocation model.
6. A cloud-government-enterprise-oriented resource allocation model training device, wherein the device is applied to an SDN-based cloud network, the cloud network including a controller, a plurality of data centers, and an optical transport network connecting the plurality of data centers, the device comprising:
the initialization module is configured to acquire cloud network resources from the data center and the optical transmission network by using the controller, construct a resource abstraction model according to the cloud network resources, and initialize a pre-constructed reinforcement learning neural network model according to the resource abstraction model;
a training set construction module configured to construct a business training set, comprising:
acquiring a plurality of virtual services;
classifying the virtual services according to the data isolation requirements of the virtual services;
constructing a source-sink node pair and a source-sink node path from each classified virtual service to each data center;
taking all the source-sink node pairs and the source-sink node paths as the service training set;
a training module configured to train the reinforcement learning neural network model with the business training set to obtain a resource allocation model, comprising:
performing multiple rounds of iterative training on the reinforcement learning neural network model, wherein each time the iterative training is performed by applying all source and destination node paths corresponding to one virtual service in the service training set, different times of iterative training are performed by applying different virtual services, and for each time of iterative training, the following operations are performed:
sequencing all source and sink node paths according to the length from short to long, and selecting the first n source and sink node paths as candidate paths, wherein n is an integer greater than 1;
performing resource allocation on each candidate path to obtain a plurality of allocation behaviors;
modifying a hidden layer state of the reinforcement learning neural network model according to the allocation behavior, and outputting a Q value through the reinforcement learning neural network model;
calculating the reward and punishment value of the distribution behavior with the maximum Q value, and accumulating the reward and punishment value into a total reward and punishment value;
and responding to the total reward and punishment value reaching a first threshold value and/or the iterative training frequency reaching a second threshold value, and finishing the iterative training to obtain the resource allocation model, wherein the first threshold value and the second threshold value are preset.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 4 when the program is executed by the processor.
8. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 4.
CN202210376475.5A 2022-04-11 2022-04-11 Cloud government enterprise oriented resource allocation model training method and resource allocation method Active CN115002215B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210376475.5A CN115002215B (en) 2022-04-11 2022-04-11 Cloud government enterprise oriented resource allocation model training method and resource allocation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210376475.5A CN115002215B (en) 2022-04-11 2022-04-11 Cloud government enterprise oriented resource allocation model training method and resource allocation method

Publications (2)

Publication Number Publication Date
CN115002215A CN115002215A (en) 2022-09-02
CN115002215B true CN115002215B (en) 2023-12-05

Family

ID=83023988

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210376475.5A Active CN115002215B (en) 2022-04-11 2022-04-11 Cloud government enterprise oriented resource allocation model training method and resource allocation method

Country Status (1)

Country Link
CN (1) CN115002215B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103338163A (en) * 2013-07-16 2013-10-02 清华大学 Software-defined network controller supporting scheduling of dynamic elastic resource
CN108449286A (en) * 2018-03-01 2018-08-24 北京邮电大学 Network bandwidth resources distribution method and device
CN109039884A (en) * 2017-06-12 2018-12-18 瞻博网络公司 Use the network path prediction and selection of machine learning
CN110519664A (en) * 2019-08-06 2019-11-29 北京邮电大学 The configuration method and device of transceiver in software definition optical-fiber network
CN111106999A (en) * 2019-12-27 2020-05-05 国网江苏省电力公司信息通信分公司 IP-optical network communication service joint distribution method and device
CN111144574A (en) * 2018-11-06 2020-05-12 北京嘀嘀无限科技发展有限公司 Artificial intelligence system and method for training learner model using instructor model
CN111200566A (en) * 2019-12-17 2020-05-26 北京邮电大学 Network service flow information grooming method and electronic equipment
WO2020228143A1 (en) * 2019-05-15 2020-11-19 福州大学 Cloud software service resource allocation method based on qos model self-correction
US10873533B1 (en) * 2019-09-04 2020-12-22 Cisco Technology, Inc. Traffic class-specific congestion signatures for improving traffic shaping and other network operations
CN112383477A (en) * 2020-10-22 2021-02-19 国网电力科学研究院有限公司 Routing and spectrum allocation method and device for data center optical network

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10097372B2 (en) * 2014-01-09 2018-10-09 Ciena Corporation Method for resource optimized network virtualization overlay transport in virtualized data center environments
US9602427B2 (en) * 2014-02-06 2017-03-21 Nec Corporation Cloud service embedding with shared protection in software-defined flexible-grid optical transport networks
US10671938B2 (en) * 2016-01-27 2020-06-02 Bonsai AI, Inc. Artificial intelligence engine configured to work with a pedagogical programming language to train one or more trained artificial intelligence models
US11461145B2 (en) * 2019-01-28 2022-10-04 EMC IP Holding Company LLC Building neural networks for resource allocation for iterative workloads using reinforcement learning
US11392843B2 (en) * 2019-04-01 2022-07-19 Accenture Global Solutions Limited Utilizing a machine learning model to predict a quantity of cloud resources to allocate to a customer

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103338163A (en) * 2013-07-16 2013-10-02 清华大学 Software-defined network controller supporting scheduling of dynamic elastic resource
CN109039884A (en) * 2017-06-12 2018-12-18 瞻博网络公司 Use the network path prediction and selection of machine learning
CN108449286A (en) * 2018-03-01 2018-08-24 北京邮电大学 Network bandwidth resources distribution method and device
CN111144574A (en) * 2018-11-06 2020-05-12 北京嘀嘀无限科技发展有限公司 Artificial intelligence system and method for training learner model using instructor model
WO2020228143A1 (en) * 2019-05-15 2020-11-19 福州大学 Cloud software service resource allocation method based on qos model self-correction
CN110519664A (en) * 2019-08-06 2019-11-29 北京邮电大学 The configuration method and device of transceiver in software definition optical-fiber network
US10873533B1 (en) * 2019-09-04 2020-12-22 Cisco Technology, Inc. Traffic class-specific congestion signatures for improving traffic shaping and other network operations
CN111200566A (en) * 2019-12-17 2020-05-26 北京邮电大学 Network service flow information grooming method and electronic equipment
CN111106999A (en) * 2019-12-27 2020-05-05 国网江苏省电力公司信息通信分公司 IP-optical network communication service joint distribution method and device
CN112383477A (en) * 2020-10-22 2021-02-19 国网电力科学研究院有限公司 Routing and spectrum allocation method and device for data center optical network

Also Published As

Publication number Publication date
CN115002215A (en) 2022-09-02

Similar Documents

Publication Publication Date Title
Xiao et al. Distributed optimization for energy-efficient fog computing in the tactile internet
US11218546B2 (en) Computer-readable storage medium, an apparatus and a method to select access layer devices to deliver services to clients in an edge computing system
US11265369B2 (en) Methods and systems for intelligent distribution of workloads to multi-access edge compute nodes on a communication network
NL2029029B1 (en) Methods and apparatus to coordinate edge platforms
EP3974980A1 (en) Methods, apparatus, and articles of manufacture for workload placement in an edge environment
Ghanbari et al. Resource allocation mechanisms and approaches on the Internet of Things
US9667749B2 (en) Client-initiated leader election in distributed client-server systems
KR20210081227A (en) End-to-end quality of service in edge computing environments
EP4180953A1 (en) Orchestrator execution planning using a distributed ledger
US20190114206A1 (en) System and method for providing a performance based packet scheduler
US10620928B2 (en) Global cloud applications management
US10027596B1 (en) Hierarchical mapping of applications, services and resources for enhanced orchestration in converged infrastructure
CN106856438A (en) A kind of method of Network instantiation, device and NFV systems
US20210117134A1 (en) Technologies for storage and processing for distributed file systems
Chiang et al. Management and orchestration of edge computing for iot: A comprehensive survey
US20240039860A1 (en) Methods, systems, apparatus, and articles of manufacture to manage network communications in time sensitive networks
US20220021729A1 (en) Efficient data processing in a mesh network of computing devices
CN115002215B (en) Cloud government enterprise oriented resource allocation model training method and resource allocation method
US20230196182A1 (en) Database resource management using predictive models
US11853810B2 (en) Edge time sharing across clusters via dynamic task migration based on task priority and subtask result sharing
CN113132445A (en) Resource scheduling method, device, network system and storage medium
US11586626B1 (en) Optimizing cloud query execution
Sampath Towards Seamless Edge Computing in Connected Vehicles
Wang et al. A Virtual Optical Network Mapping Algorithm Based on Machine Learning
US20220166830A1 (en) Mobile kube-edge auto-configuration

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant