WO2021000694A1 - 一种部署业务的方法以及调度装置 - Google Patents

一种部署业务的方法以及调度装置 Download PDF

Info

Publication number
WO2021000694A1
WO2021000694A1 PCT/CN2020/094442 CN2020094442W WO2021000694A1 WO 2021000694 A1 WO2021000694 A1 WO 2021000694A1 CN 2020094442 W CN2020094442 W CN 2020094442W WO 2021000694 A1 WO2021000694 A1 WO 2021000694A1
Authority
WO
WIPO (PCT)
Prior art keywords
qos
execution
path
service
deployed
Prior art date
Application number
PCT/CN2020/094442
Other languages
English (en)
French (fr)
Inventor
肖磊
金波
申思
李有强
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20834405.1A priority Critical patent/EP3968606B1/en
Publication of WO2021000694A1 publication Critical patent/WO2021000694A1/zh
Priority to US17/563,555 priority patent/US11838389B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/5041—Network service management, e.g. ensuring proper service fulfilment according to agreements characterised by the time relationship between creation and deployment of a service
    • H04L41/5054—Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H04L47/80—Actions related to the user profile or the type of traffic
    • H04L47/805—QOS or priority aware
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/61—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services

Definitions

  • This application relates to the field of cloud computing, and in particular to a method for deploying services and a scheduling device.
  • a scheduling device is usually deployed in the cloud platform for selecting an execution path for deploying the service to be deployed according to the requirements of the service to be deployed. Specifically, according to the type of service to be deployed, the scheduling device selects at least one execution device that meets the requirements of the service to be deployed and is used to execute the service to be deployed, thereby determining the execution path including the selected at least one execution device. Deploy the service to be deployed on the execution path.
  • the execution device that meets the requirements of the service to be deployed is selected according to the quality of service (QoS) of each execution device.
  • QoS quality of service
  • the scheduling device only pays attention to the QoS of one or a few execution devices. This is because different services to be deployed have different requirements for the QoS of the executing device. For example, communication services to be deployed have higher QoS requirements for the execution equipment used for forwarding, and lower QoS requirements for the execution equipment used for calculation; on the contrary, the financial services to be deployed have higher requirements for the execution equipment used for forwarding.
  • the QoS requirement is low, and the QoS requirement of the execution device used for calculation is high.
  • the execution path is selected based on the QoS of a certain execution device or several execution devices, which results in the execution path selected by the scheduling device can only roughly meet the needs of the service to be deployed, so the selected execution path may not meet the service to be deployed Demand.
  • This application provides a method for deploying services and a scheduling device, which are used to calculate the QoS of the execution path, and select the execution path for deploying the service to be deployed according to the QoS of the execution path, so that the selection of the execution path is more accurate and avoids selection
  • the execution path cannot meet the needs of the business to be deployed.
  • the first aspect of the embodiments of the present application provides a method for deploying a service, which specifically includes: the scheduling device may first determine the executable path (also called the execution path) of the service to be deployed according to the identifier of the service to be deployed, and the execution path At least two execution devices are included; after that, the scheduling device further obtains the QoS of each execution device of the execution path, and calculates the QoS of the execution path according to the QoS of each execution device of the execution path; finally, compares the execution The QoS of the path and the preset QoS of the service to be deployed (the preset QoS of the service to be deployed is set by the user in advance in the scheduling device.
  • the scheduling device may first determine the executable path (also called the execution path) of the service to be deployed according to the identifier of the service to be deployed, and the execution path At least two execution devices are included; after that, the scheduling device further obtains the QoS of each execution device of the execution path, and calculates the QoS of the
  • the scheduling device After the user has set the preset QoS, the scheduling device obtains the preset QoS of the service to be deployed Whether the relationship between QoS) conforms to the preset rule, if it is determined to conform to the preset rule, the scheduling device will deploy the service to be deployed on the execution path, and the execution devices on the execution path will be used to execute the service to be deployed.
  • the QoS of each execution device on the integrated execution path is proposed for the first time to determine the ability of the execution path to provide services, obtain the QoS of the execution path, and select the execution path for deploying the service to be deployed according to the QoS of the execution path. Therefore, the QoS of each execution device on the execution path is integrated to determine the ability of the execution path to provide services, and the ability to accurately measure the execution path to provide services (that is, the QoS of the execution path) is provided, so that the selection of the execution path is more accurate and avoids The selected execution path cannot meet the needs of the business to be deployed.
  • the scheduling device may also obtain the QoS of the transmission link between every two execution devices of the execution path. At this time, the scheduling device calculates the QoS of the execution path by calculating the QoS of the execution path according to the QoS of each execution device on the execution path and the QoS of the transmission link between every two execution devices.
  • the QoS of the transmission link between each execution device of the execution path is also considered.
  • the scheduling device in addition to obtaining the QoS of each execution device of the execution path, the scheduling device also obtains the QoS of the transmission link between each execution device, and determines the execution according to the QoS of each execution device and the QoS of each transmission link. The QoS of the path, so that the QoS calculated on the entire execution path is more accurate.
  • the scheduling device obtains the QoS of each execution device of the execution path by the following method: First, the scheduling device determines the QoS calculation formula corresponding to each execution device in the execution path. There is a corresponding QoS calculation formula; after that, the scheduling device sends each QoS calculation formula to the corresponding execution device, and the corresponding execution device calculates the QoS of the execution device according to the QoS calculation formula; finally, the scheduling device executes Each execution device on the path obtains the QoS of the execution device.
  • the scheduling device obtains the QoS of the execution device (that is, by sending the corresponding QoS calculation formula to each execution device), which is practical.
  • the scheduling apparatus obtains the QoS of the transmission link between every two execution devices of the execution path by obtaining the following methods: First, the scheduling apparatus determines the transmission between every two execution devices in the execution path Link, and determine the QoS calculation formula corresponding to each transmission link, and a transmission link corresponds to a QoS calculation formula; then, the scheduling device sends the QoS calculation formula corresponding to each transmission link to the transmission link At least one of the two execution devices of the route, and the at least one execution device calculates the QoS of the transmission link according to the corresponding QoS calculation formula; finally, the scheduling device obtains the corresponding transmission link from the at least one execution device. QoS of the road.
  • the scheduling device obtains the QoS of the transmission link between two execution devices (that is, by sending each execution device corresponding to the transmission link) QoS calculation formula), with wide applicability.
  • the scheduling device calculates the QOS of the execution path according to the QoS of each execution device on the execution path. Specifically, it may also be: first determine the weight of the QoS of each execution device in the execution path, and the execution device The weight of QoS indicates the importance of the execution device in the execution path; finally, the QoS of the execution path is determined according to the QoS of each execution device and the weight of the corresponding QoS.
  • the scheduling device further clarifies the importance of each execution device in the execution path through the weight of the QoS of the execution device, which is more targeted.
  • the scheduling device may determine the execution path by first determining N (N ⁇ 1) execution paths according to the resources required by the service to be deployed, so that each execution device in the N execution paths includes The specification of the resource is not lower than the specification of the resource required by the service to be deployed; after that, the scheduling device will calculate the QoS of the N execution paths to obtain the QoS of the N execution paths.
  • the scheduling device determines the N execution paths If the relationship between any one of the QoS and the preset QoS does not meet the preset rules, the scheduling device will adjust the resource specifications of the execution equipment on any execution path until it obtains the execution path that meets the preset QoS conditions (which can be called The adjusted execution path); finally, the scheduling device will deploy the service to be deployed on the adjusted execution path.
  • the scheduling device will perform resource adjustment so that the service to be deployed is deployed on the adjusted execution path , Ensure the reliability and stability of the service to be deployed.
  • the scheduling device may further The QoS of the execution path is predicted to obtain the predicted QoS of the execution path.
  • the scheduling device further determines that the relationship between the predicted QoS of the execution path and the preset QoS conforms to the preset rules, and the scheduling device determines to deploy the service to be deployed on the execution path on.
  • the scheduling device not only compares the QoS in the current cycle of each execution path, but also compares the QoS in the next cycle of each execution path. Only the QoS in the current cycle of the execution path and the QoS in the next cycle of the execution path are compared.
  • the relationship between the QoS and the preset QoS is in compliance with the preset relationship, and the above-mentioned service to be deployed will be deployed on the execution path, which further guarantees the stability of the service to be deployed.
  • the second aspect of the embodiments of the present application provides a scheduling device.
  • the scheduling device implements the method in the above-mentioned embodiment through hardware or through hardware execution of corresponding software.
  • the hardware or the software includes one or more of the above-mentioned implementations.
  • the scheduling device may include:
  • the path selection module is configured to: determine the execution path of the service to be deployed, the execution path including at least two execution devices; obtain the quality of service QoS of each execution device; calculate the QoS of the execution path according to the QoS of each execution device ; Obtain the preset QoS of the service to be deployed; determine that the relationship between the QoS of the execution path and the preset QoS meets the preset rules;
  • the service deployment module is configured to: after the path selection module determines that the relationship between the QoS of the execution path and the preset QoS conforms to the preset rule, deploy the service to be deployed on the execution path, and the at least two execution devices are used to execute respectively The business to be deployed.
  • the path selection module is specifically used for:
  • the QoS of the path calculates the QoS of the execution path.
  • the path selection module is specifically used to:
  • the path selection module is specifically configured to: determine at least one second QoS calculation formula corresponding to the at least one transmission link; and send each second QoS calculation formula to the corresponding transmission link Finally, obtain the QoS of each transmission link obtained by the execution device on the at least one transmission link according to the corresponding second QoS calculation formula.
  • the path selection module is specifically configured to: determine the weight of each QoS in the QoS of each execution device; and determine the QoS of the execution path according to each QoS and the corresponding weight.
  • the path selection module is specifically configured to: determine N execution paths according to the resources required by the service to be deployed, wherein the specifications of the resources included in the execution device on each execution path are not lower than the The specifications of the resources required for the service to be deployed; determine that the relationship between the QoS of each execution path of the N execution paths and the preset QoS does not meet the preset rule, and adjust the resource specifications of the execution devices on any execution path until Obtain the execution path that meets the preset QoS conditions.
  • the path selection module determines that the relationship between the QoS of the execution path and the preset QoS meets the preset rule
  • the path selection module is specifically used to: perform the QoS of the execution path in the next cycle Predict to obtain the predicted QoS of the execution path, the QoS of the execution path is the QoS in the current cycle of the execution path; determine that the relationship between the predicted QoS of the execution path and the preset QoS meets the preset rule;
  • the service deployment module is specifically used to deploy the service to be deployed on the execution path after the path selection module determines that the relationship between the predicted QoS of the execution path and the preset QoS meets the preset rule.
  • the third aspect of the embodiments of the present application provides a computing device.
  • the computing device includes one or more processors and one or more memories; wherein the one or more memories store one or more computer instructions, when the computer When the instructions are executed by the one or more processors, the computing device is caused to execute any one of the possible implementation methods of the foregoing embodiments.
  • the fourth aspect of the embodiments of the present application provides a computer-readable storage medium that stores instructions in the computer-readable storage medium, and when it runs on a computer, the computer can execute any one of the possible implementation methods of the above-mentioned embodiments .
  • Figure 1 is a system framework diagram of an embodiment of the application
  • FIG. 2 is a schematic diagram of a path b1 of service B in an embodiment of this application;
  • FIG. 3 is a schematic diagram of a method for deploying services according to an embodiment of the application
  • FIG. 4 is a schematic diagram of the path b2 of service B after resource adjustment of the scheduling device according to the embodiment of the application;
  • FIG. 5 is a schematic diagram of the flow of QoS reporting to the scheduling device in an embodiment of the application
  • FIG. 6 is a schematic diagram of a method for deploying services according to an embodiment of the application.
  • FIG. 7 is a schematic diagram of a scheduling device according to an embodiment of the application.
  • FIG. 8 is another schematic diagram of a scheduling device according to an embodiment of the application.
  • Cloud computing is an emerging business service model and computing model. It provides various computing and storage services to users who access cloud computing through the Internet. This service does not rely on local computers, but on at least one cloud. server. Cloud servers can provide various resources such as computing, storage, and network, and provide services to users based on the resources provided by cloud servers. When a user needs to execute a service to be deployed, resources on the cloud server can be allocated to the service to be deployed according to the needs of the service to be deployed. In other words, the on-demand distribution of cloud servers can allocate cloud server resources according to task requirements "within the means of expenditure".
  • QoS refers to the ability of an executing device or service to provide services for a specific service to be deployed
  • QoS is the value of QoS.
  • the execution path is selected based on the QoS of one or several execution devices, which causes the execution path selected by the scheduling device to roughly meet the needs of the service to be deployed, and the selected execution path may not meet the service to be deployed. Demand.
  • the scheduling device is used to input a user request (such as inputting a certain item to be deployed) Business) Allocate the resources of the cloud server. Specifically, it is responsible for the processing of user input tasks, the storage of status data, the scheduling of resources, etc. According to the user request, the scheduling device uniformly schedules the various execution devices involved in the user request under the cloud server.
  • the scheduling device can schedule execution device 1, execution device 2,..., execution device n according to the specifications of the resources required by the service to be deployed (assuming the The execution devices involved in the deployment service are the resources included in the execution device 1, the execution device 2,..., the execution device n).
  • the smallest functional unit in the cloud server can be called an execution device, and the resource specification included in the execution device is not less than the resource specification required by the service to be deployed.
  • the execution device represents an alternative device for deploying a service to be deployed
  • the execution path represents an alternative path for deploying a service to be deployed.
  • Execution devices can include computing devices, storage devices, and network devices.
  • Computing devices include physical machines, virtual machines, containers, and so on.
  • the scheduling device can be deployed on the same cloud server together with the functional device that executes specific services, that is to say, corresponding software/hardware is placed in the cloud server to implement resource scheduling;
  • the device can also be deployed on multiple different cloud servers, such as multiple virtual machines together as a scheduling device to implement resource scheduling; the scheduling device can also be deployed separately (the scheduling device shown in Figure 1 is deployed separately), The details are not limited here.
  • the execution path includes the execution device and the transmission link between the execution devices through which the data related to the business passes when completing a business.
  • VPC virtual private cloud
  • VTEP Virtual Tunnel Endpoint 1 of the VPC
  • VTEP1 is connected to VTEP2
  • VTEP2 may be a physical switch.
  • Figure 2 illustrates a physical switch—Top of Rack (ToR).
  • VPC, ToR, bare metal 1, SAN transmission link between VPC and ToR 1, transmission link between ToR and bare metal 1, transmission link 2 between bare metal 1 and SAN
  • the execution path of B may be referred to as path b1)
  • VPC, ToR, bare metal 1, SAN are the execution devices of the execution path
  • the execution path of the service includes at least one execution device.
  • the scheduling device can determine the execution path of the service to be deployed in many ways, which are not specifically limited here. The following illustrates one of the determination methods:
  • the scheduling device receives the user request, and the user request carries the service identifier of the service to be deployed.
  • the scheduling device obtains the service identifier, and can determine the type of resources required by the service to be deployed by the user based on the service identifier (the service identifier has a mapping relationship with the resource type of the service to be deployed).
  • the scheduling device also needs to obtain the specifications of the resources required by the service to be deployed, and then the scheduling device determines the execution path according to the types and specifications of the resources required by the service to be deployed.
  • business B is a storage business, assuming that the storage resource required by business B is 2GB (that is, the resource specification of business B is storage resource 2GB), when the user enters business B, the scheduling device can obtain it at the same time
  • the identifier of the service B (assuming the service identifier of the service B is "1122")
  • the scheduling device can determine the type and specification of the resources required by the service B according to the service identifier "1122" as the above-mentioned storage resource 2GB, and then
  • the scheduling device determines that the execution equipment (such as: execution equipment VPC, ToR, bare metal 1, SAN can provide 2GB storage resources) under the cloud server can provide 2GB storage resources, then the execution equipment VPC, ToR,
  • the path formed by bare metal 1, SAN and the transmission link 1, transmission link 2, and transmission link 3 between them (that is, the path b1 in Figure 2) is determined by the scheduling device as the executable path of the service to be deployed (It
  • the determined execution path only meets the requirements of resources required by the service to be deployed.
  • further judgment is used to determine whether to use the execution path as the execution path for deploying the service to be deployed.
  • the scheduling device After the scheduling device determines the execution path, it will further obtain the QoS of each execution device of the execution path.
  • the QoS of the execution device is determined according to the specifications of the resources included on the execution device. How many resources the execution device can provide to the service to be deployed is called the specification of the resources included in the execution device.
  • There may be multiple ways for the scheduling apparatus to obtain the QoS of each execution device which is not specifically limited here. The following illustrates several ways to obtain the QoS of each execution device on the execution path:
  • the QoS of each execution device on the execution path is calculated by the QoS calculation formula.
  • the scheduling device may send the QoS calculation formula corresponding to each of the foregoing execution devices (that is, one execution device corresponds to a QoS calculation formula) to the respective execution device, and each execution device calculates the QoS calculation formula according to the obtained QoS calculation formula.
  • the QoS of the execution device for example, when the execution device is a physical machine, the QoS of the physical machine can be calculated based on the calculation formulas of the host's central processing unit (CPU) usage, memory usage, bandwidth, delay, etc. Come.
  • CPU central processing unit
  • each of the foregoing execution devices sends the QoS of the execution device calculated according to the QoS calculation formula to the scheduling apparatus.
  • the above-mentioned QoS calculation formula corresponding to the execution device may be built in the scheduling device in advance.
  • the scheduling device determines each execution device of the execution path, the QoS calculation formula of each execution device is sent to the corresponding Execution equipment; the QoS calculation formula can also exist in the equipment A (including virtual equipment, physical equipment, etc.) connected to the scheduling device.
  • the scheduling device determines the execution path, it can send to the above-mentioned equipment connected to the scheduling device A sends a request instruction, which is used to trigger the device A to send the QoS calculation formula corresponding to each execution device on the execution path to the scheduling device, and then the scheduling device sends each QoS calculation formula to the corresponding execution device.
  • the first QoS calculation formula may also be built into each execution device in advance. After the scheduling device determines the execution path, the scheduling device will send instructions to each execution device of the execution path, and the instruction is used to trigger each execution device according to The built-in QoS calculation formula calculates the QoS of the execution device. In the embodiment of the present application, there is no limitation on how the executing device obtains the QoS calculation formula.
  • the QoS of each execution device mentioned above is obtained from the mapping relationship between the resource specification and QoS included in each execution device on the execution path.
  • mapping relationship library stores the mapping relationship between the specifications of the resources included in each execution device and the QoS of the corresponding execution device.
  • the scheduling device can obtain the specifications of the resources included in each execution device at the current time, and find the execution at the current time according to the mapping relationship between the specifications of the resources included in the execution device and the QoS of the execution device in the mapping relationship library.
  • the QoS of the device For ease of understanding, the following examples are used for illustration:
  • the scheduling device determines that the execution path of service C is path c
  • the execution devices included in path c include device a, device b, and device c. At this time, the scheduling device needs to obtain the QoS of equipment a, equipment b, and equipment c.
  • the scheduling equipment will first obtain the specifications of the resources included in equipment a, equipment b, and equipment c at the current moment (assuming equipment a, equipment b ,
  • the specifications of the resources included in the device c at the current moment are x1, y2, and z3 respectively), and look for the corresponding when the specifications of the resources included in the device a are x1 in the mapping library (Table 1 shows a mapping library)
  • Table 1 shows a mapping library
  • the scheduling device After obtaining the QoS of each execution device on the execution path, the scheduling device will calculate the QoS of the execution path according to the QoS of each execution device mentioned above.
  • the QoS of the execution path there are multiple implementation manners for calculating the QoS of the execution path, which are not specifically limited here. The following illustrates several ways to calculate the QoS of the execution path:
  • the user can independently set the QoS of the service (also known as the service to be deployed), which may be that the scheduling device first intelligently recommends to the user according to the service type and nature of the service to be deployed input by the user.
  • a commonly used QoS for deploying services can be called conventional QoS
  • the user refers to the recommended conventional QoS and then sets the QoS of the service to be deployed by himself (the user can directly use the conventional QoS as the preset QoS without modification , It can also be used as the preset QoS based on the conventional QoS after slight adjustment according to their own preferences, which is not limited here); users can also directly set the QoS of the service to be deployed according to their own needs to get the preset QoS.
  • the basis for setting the preset QoS is not specifically limited here. After the user completes the QoS setting of the service to be deployed, the scheduling device can obtain the preset QoS for the service to be deployed.
  • step 304 can be performed before step 301, can also be performed after step 303, can also be performed between any two steps between step 301 and step 303, and can also be performed with step 301. Any one of the steps to step 303 is performed at the same time, and the details are not limited here.
  • the scheduling device After the scheduling device obtains the QoS of the execution path of the service to be deployed and the user's preset QoS for the service to be deployed, it will determine whether the relationship between the QoS of the execution path and the preset QoS conforms to the preset rule. If the scheduling device determines the execution The relationship between the QoS of the path and the preset QoS conforms to the preset rule, and the service to be deployed is deployed on the execution path.
  • the preset rule may be that the QoS of the execution path is greater than the preset QoS, or the QoS of the execution path minus the preset QoS and the value obtained is greater than a preset positive value.
  • the preset rules are not limited here.
  • the embodiment of this application it is proposed for the first time how to obtain the QoS of the executable path (that is, the execution path) of the service to be deployed, and compare the preset QoS of the service to be deployed with the QoS of the execution path to determine that the service to be deployed is finally executed Whether the time is deployed on the execution path.
  • the embodiment of the present application also proposes for the first time that the QoS of the service to be deployed can be set by the user, which fully takes into account the preferences of different users for the QoS of the service to be deployed, and improves the service satisfaction of the service to be deployed.
  • the QoS of the transmission link between each execution device may also be considered. That is, before step 303, the scheduling device can further obtain the QoS of the transmission link between each execution device of the execution path, then the QoS of the execution path obtained in step 303 is based on the QoS and execution of each execution device of the execution path. The QoS of each transmission link in the path is calculated.
  • the scheduling apparatus obtains the QoS of each execution device of the execution path according to the method described in the embodiment corresponding to FIG. 3, which will not be repeated here.
  • the scheduling device obtains the QoS of the transmission link: first, the scheduling device determines the transmission link involved in the execution path, and determines the QoS calculation formula corresponding to each transmission link, and then calculates the QoS corresponding to each transmission link The calculation formula is sent to one of the two execution devices on the corresponding transmission link, and the execution device that has obtained the QoS calculation formula calculates the QoS of the corresponding transmission link and sends the QoS of the transmission link to the scheduling device.
  • the scheduling device determines the path b1 of service B, it can determine that the transmission links of path b1 include transmission link 1, transmission link 2, transmission link 3, and determine
  • the QoS calculation formulas corresponding to transmission link 1, transmission link 2, and transmission link 3 respectively are calculation formula 21, calculation formula 22, and calculation formula 23.
  • the two execution devices on transmission link 1 are VPC and ToR
  • the two execution devices on transmission link 2 are ToR and bare machine 1
  • the two execution devices on transmission link 3 are bare machine 1 and SAN; after that, the scheduling device can send the calculation formula 21 to the VPC or ToR (assuming to send to VPC), send calculation formula 22 to ToR or bare metal 1 (assumed to be sent to bare metal 1), send calculation formula 23 to bare metal 1 or SAN (assumed to be sent to SAN), and then VPC, bare metal 1, and SAN are respectively based on the calculation formula 21.
  • Calculation formula 22 and calculation formula 23 calculate the QoS of transmission link 1, transmission link 2, and transmission link 3.
  • VPC, bare metal 1, and SAN respectively send the obtained QoS of the above three transmission links to the scheduler Device.
  • the scheduling device may also send the QoS calculation formula corresponding to the transmission link to two execution devices on the transmission link, so that both execution devices perform the same transmission Calculate the QoS of the link.
  • the scheduling device not only obtains the QoS of each execution device of the execution path, but also obtains the QoS of the transmission link between each execution device, and determines the execution according to the QoS of each execution device and the QoS of each transmission link.
  • the QoS of the path makes the QoS calculated on the entire execution path more accurate.
  • the execution path determined by the scheduling device can be obtained according to, but not limited to, the following method: First, N (N ⁇ 1) execution paths are determined according to the service identifier, wherein each execution path The specifications of the resources included in the execution equipment are not lower than the specifications of the resources required by the service to be deployed.
  • the method for the scheduling device to determine the QoS of each of the N execution paths and the method of obtaining the QoS of each execution path can be referred to Steps 301-305 in the foregoing embodiment will not be repeated here.
  • the scheduling device will first determine several execution paths (such as path d1, path d2, and path d3) according to the specifications of the resources required by the service to be deployed (for example, the bandwidth required by service D is not less than 100kbps). Service D provides bandwidth resources not less than 100kbps).
  • the scheduling device determines that the resource value required by service B is satisfied (assuming that the resource value required by service B is 2GB), there is only path b1, then the scheduling device can directly follow the above-mentioned embodiment
  • the method of obtaining the QoS of the execution path obtains the QoS of path b1, and compares the QoS of path b1 with the preset QoS set by the user for service B. If the QoS of path b1 matches the preset QoS of service B, Set rules, then deploy business B on the path b1.
  • the scheduling device determines that the execution path that meets the resource value required by service B is not only path b1, but also path b2 and path b3, then the execution paths of service B are these three Any one of the execution path.
  • the scheduling device can obtain the QoS of path b1, path b2, and path b3 according to the method of obtaining the QoS of the execution path described in the above embodiment, and then obtain the QoS of path b1, path b2, and path b3. Compare with the user's preset QoS for service B in advance, and determine which execution path service B will ultimately be deployed on according to the comparison result.
  • the scheduling device determines that the relationship between the QoS of only one of the N execution paths and the preset QoS meets the preset rule, the service to be deployed is deployed on the execution path where the QoS and the preset QoS meet the preset rule.
  • the QoS of the execution path (the aforementioned execution path QoS refers to the QoS in the current cycle of the execution path) to predict to obtain the predicted QoS in the next cycle of the execution path.
  • the scheduling device determines that the relationship between the predicted QoS of the execution path and the preset QoS of the service to be deployed also conforms to the prediction. After the rule is set, the service to be deployed is deployed on the execution path. Taking service B as an example, the QoS of path b1 can be further predicted to obtain the predicted QoS in the next cycle of path b1.
  • the scheduling transpose determines the relationship between the predicted QoS of path b1 and the preset QoS of service B. After the preset rules, service B will be deployed on path b1.
  • the scheduling device determines that the relationship between the QoS of at least two execution paths and the preset QoS among the N execution paths is in compliance with the preset rule, the service is deployed on one of the at least two execution paths.
  • the execution path to be deployed to determine business B can be:
  • the scheduling device determines that the relationship between the QoS of the N execution paths and the preset QoS does not conform to the preset rule, it performs resource adjustment.
  • the scheduling device can adjust the resources of the above N execution paths according to the resource value required by the service to be deployed, by adjusting any execution path
  • the resource specifications of the execution device on the upper limit until the execution path that meets the preset QoS conditions (which may be referred to as the adjusted execution path) is obtained, and then the service to be deployed is deployed on the adjusted execution path.
  • the scheduling device finds that the relationship between the QoS calculated by the current service B in each of the optional execution paths (that is, path b1, path b2, and path b3) and the preset QoS are all Failure to comply with the preset rules will trigger the adjustment of back-end resources.
  • the back-end of the scheduling device can deploy other services (such as service D, service E, etc.) on the optional execution path of service B by means of migration, suspension, restriction, etc.
  • the resource usage of each execution device is reduced to reduce resource contention for service B, and to improve the QoS of service B on these execution devices, so as to make the QoS of service B on the execution path formed by these execution devices Has also been promoted.
  • the scheduling device may adjust the resource adjustment method to increase the QoS of the execution device.
  • the addition of resources on the execution device of the execution path improves the QoS of the execution device.
  • the specific method for resource adjustment is not limited here.
  • the QoS of each optional execution path of service B (that is, path b1, path b2, and path b3) may be found to change, so as long as the QoS of each execution path after the change is changed
  • the preset QoS is compared, and if the scheduling device determines that there is at least one execution path after resource adjustment, the relationship between the QoS and the preset QoS conforms to the preset rule, the execution path to be deployed for service B can be determined in a similar manner to the above embodiment ; If the scheduling device determines that the relationship between the QoS of an execution path and the preset QoS after adjusting the resources still conforms to the preset rule, the resource adjustment is continued until there is at least a relationship between the QoS of the execution path and the preset QoS after adjustment.
  • Preset rules Take Figure 4 as an example to illustrate: After resource adjustment, the scheduling device determines that the relationship between the QoS of service B's path b2 and the preset QoS meets the preset rules, and then deploys service B on path b2, which includes VPC, High-quality multi-layer virtual switch OVS, virtual machine 2, distributed block storage, transmission link 4 connected through VPC VTEP1 and OVS VTEP3, transmission link 5 connecting OVS and virtual machine 2, connecting virtual machine 2 and Transmission link 6 for distributed block storage.
  • VPC High-quality multi-layer virtual switch OVS
  • virtual machine 2 distributed block storage
  • transmission link 4 connected through VPC VTEP1 and OVS VTEP3
  • transmission link 5 connecting OVS and virtual machine 2
  • Transmission link 6 for distributed block storage.
  • the process of determining the execution path of the service to be deployed is: the scheduling device first determines the N execution paths that meet the specifications of the resources required by the service to be deployed according to the identifier of the service to be deployed, and calculates the number of execution paths of the N execution paths. N QoS, and then determine how many of the N QoS and the preset QoS relationship meet the preset rules (assuming that there are W QoS meets the preset rules), and finally according to the above-mentioned determination method from this W An execution path corresponding to a QoS is selected among the QoS, and the service to be deployed is deployed on the selected execution path.
  • the scheduling device can calculate the QoS of these N execution paths one by one according to a preset method, and calculate the QoS of one execution path.
  • the QoS of the execution path is compared with the preset QoS, and when the scheduling device determines that the relationship between the QoS of the execution path and the preset QoS meets the preset rules, it directly deploys the service to be deployed on the execution path that meets the requirements.
  • the QoS of the remaining execution path is no longer calculated, thereby reducing the amount of calculation.
  • the scheduling device will first calculate the QoS of path b1, and after calculating the QoS of path b1, compare the QoS with the preset QoS, assuming that the result of the comparison is that the relationship between the QoS of path b1 and the preset QoS does not conform to the preset Rule, the scheduling device continues to calculate the QoS of path b2, and after calculating the QoS of path b2, it compares the QoS with the preset QoS, assuming that the result of the comparison is that the relationship between the QoS of path b2 and the preset QoS conforms to the preset According to the rule, the scheduling device directly determines that the path b2 is the path to be deployed by the service B, and deploys the service B on the path b2 without calculating the QoS of the path b3.
  • the scheduling device can also directly determine an execution path according to the specifications of the resources required by the service to be deployed (that is, it is not necessary to first determine N satisfying service resources to be deployed). Specification execution path), each time an execution path is determined, the QoS of the execution path is calculated and compared with the preset QoS. If the scheduling device determines that the QoS of the execution path matches the preset QoS, it directly The execution path is determined to be the execution path of the service to be deployed, and the service to be deployed is deployed on the execution path, without determining whether there are other execution paths that meet the specifications of the resources required by the service to be deployed.
  • the scheduling device directly It is determined that the path b2 is the execution path for service B to be deployed, and the service B is deployed on the path b2, regardless of whether there are other execution paths that can meet the resource specifications required by the service B.
  • the scheduling device determines that the relationship between the QoS and the preset QoS of all optional execution paths (ie, N execution paths) of the service to be deployed does not meet the preset rules
  • the scheduling device Execution resource adjustment can achieve scheduling across execution paths of virtual machines, containers, physical machines, etc., while existing cloud servers cannot do this.
  • DRS distributed resource scheduler
  • the DRS system can only perform virtual machine-level migration according to the performance status of the host, and cannot achieve unified scheduling across virtual machines, containers, and physical machines.
  • VMware's DRS can only pass Basic CPU and memory can adjust the back-end resources, but cannot switch and adjust the execution path of the business to be deployed.
  • the scheduling device may classify in advance the execution paths of all services that can be executed under the cloud server, for example, divide the execution paths that can provide high storage resources into a large category
  • the execution paths that can provide high network resources are divided into a large category, and the execution paths that can provide high computing resources are divided into a large category.
  • the execution paths under each category can be divided into several sub-categories according to the scope of the resources that can be provided.
  • Each category and each sub-category has a corresponding identification, so that when a service to be deployed needs to be deployed, according to the specifications of the resources required by the service to be deployed, the scheduling device can quickly determine to meet the needs of the service to be deployed through the category identification All the optional execution paths (ie, N execution paths) of the resource specifications improve the speed of the scheduling device to determine the execution path for deploying the service to be deployed.
  • the scheduling device adjusts the QoS calculation formula corresponding to the execution device improves the QoS of the execution device.
  • the precondition that the scheduling device adjusts the QoS calculation formula is that the specifications of the resources included in the execution path are actually sufficient for the service to be deployed, only due to the QoS calculation
  • the selection of the formula results in insufficient utilization of the resources included in the execution device.
  • the QoS of the execution device host can be based on the host's central processing unit (CPU) usage rate, memory usage rate, bandwidth, and time.
  • CPU central processing unit
  • the extended QoS calculation formula is calculated. If the choices of CPU usage, memory usage, bandwidth, and latency are different, the QoS calculation formula will also be different.
  • the execution device may be an agent (also referred to as an Agent, a kind of software/hardware entity) embedded in the execution device to obtain the execution device sent by the scheduling device.
  • Agent also referred to as an Agent, a kind of software/hardware entity
  • the corresponding QoS calculation formula and/or the QoS calculation formula corresponding to the transmission link and the QoS of the execution device and/or the QoS of the transmission link calculated according to the above QoS calculation formula are sent back to the scheduling device.
  • the scheduling device determines P execution paths according to the resource value required by service E, where P ⁇ 1, these P execution paths include n execution devices, and among these n execution devices
  • Each execution device is embedded with Agent, which can be marked as Agent-1, Agent-2,..., Agent-n.
  • the QoS calculation formula (including the QoS calculation formula corresponding to each execution device and/or the QoS calculation formula corresponding to each transmission link) is sent to the agent of the corresponding execution device, that is, steps p1-p3: p1 in Figure 5
  • Send QoS calculation formula 1 corresponding to execution device 1 (including QoS calculation formula 1 of execution device 1 and/or QoS calculation formula 1 of transmission link 1 corresponding to execution device 1) to Agent-1 on execution device 1;
  • p2 send the QoS calculation formula 2 corresponding to the execution device 2 (including the QoS calculation formula 2 of the execution device 2 and/or the QoS calculation formula 2 of the transmission link 2 corresponding to the execution device 2) to the Agent- on the execution device 2.
  • the agent of each execution device After the agent of each execution device obtains the corresponding QoS calculation formula, it will obtain the QoS of the corresponding execution device according to the QoS calculation formula (if the QoS calculation formula only includes the QoS calculation formula corresponding to the execution device) and the transmission link corresponding to the execution device QoS (if the QoS calculation formula includes the QoS calculation formula corresponding to the executing device and the QoS calculation formula corresponding to the transmission link), that is, steps p4-p6 in Figure 5 (the QoS calculation formula includes the QoS calculation formula corresponding to the executing device and the transmission chain
  • the QoS calculation formula corresponding to the path is shown as an example): p4, Agent-1 calculates the QoS of the execution device 1 and the QoS of the transmission link 1 corresponding to the execution device 1; p5, the Agent-2 calculates the QoS of the execution device 2 and the corresponding execution The QoS of the transmission link 2 of the device 2; ...; p6, Agent-n
  • each device sends the calculated QoS of the execution device and the QoS corresponding to the transmission link of the execution device to the scheduling device, that is, steps p7-p9 in Figure 5:
  • Agent-1 will execute the QoS and transmission of device 1
  • the QoS of link 1 is sent to the scheduling device;
  • Agent-2 will execute the QoS of device 2 and the QoS of transmission link 2 to the scheduling device;
  • p9 Agent-n will execute the QoS of device n, the transmission chain
  • the QoS of way n is sent to the scheduling device.
  • the scheduling device can calculate the QoS of the service E on the P execution paths based on the QoS of these execution devices and the QoS of the transmission link.
  • step p7 is executed after step p4 and step p4 is executed after step p4. It is sufficient to execute after p1, step p8 after step p5, step p5 after step p2, step p9 after step p6, and step p6 after step p3, and the details are not limited here.
  • the scheduling device since the smallest functional unit of all devices (including virtual machines, physical machines, containers, etc.) under the cloud server when executing services can be called execution devices, the scheduling device uses the Agent embedded in the execution device To realize the sending and receiving of QoS calculation formula, the calculation and transmission of QoS, which is convenient for unified management. If a new execution device needs to be added under the cloud server, it is only necessary to embed an Agent in the newly added execution device to realize unified management. easy and convenient.
  • User login Portal refers to a portal site, a web-based application that mainly provides personalization, single sign-on, content integration from different sources, and the presentation layer for storing information systems), and enter what the user wants to do Deploy the business.
  • the scheduling device obtains relevant characteristics of the service to be deployed.
  • the scheduling device can obtain the type, nature, priority and other related characteristics of the service to be deployed.
  • the scheduling device intelligently recommends the conventional QoS of the service to be deployed to the user.
  • the scheduling device intelligently recommends the conventional QoS of the service to be deployed to users according to the relevant characteristics of the service to be deployed, that is, a QoS commonly adopted by most users when performing the service to be deployed.
  • the user sets the preset QoS of the service to be deployed.
  • the user refers to the conventional QoS recommended by the scheduling device to set the QoS of the service to be deployed.
  • the user can directly use the conventional QoS as the preset QoS without modification, or adjust slightly according to their own preferences on the basis of the conventional QoS The latter is used as the preset QoS, which is not specifically limited here.
  • step 603 can also be omitted, and the user can directly set the QoS of the service to be deployed to obtain the preset QoS according to their own needs.
  • the scheduling device obtains the preset QoS set by the user.
  • the scheduling device can obtain the preset QoS for the service to be deployed.
  • the scheduling device determines the specifications of resources required by the service to be deployed.
  • the scheduling device may further determine the resource specifications of the service to be deployed according to the acquired relevant characteristics of the service to be deployed. It should be noted that, in addition to step 606 can be performed after step 602 to step 605, step 606 can also be performed between any two steps of step 602 to step 605, which is not specifically limited here.
  • the scheduling device determines m QoS calculation formulas respectively corresponding to Q execution paths of the service to be deployed and m execution devices of the Q execution paths.
  • the scheduling device After the scheduling device determines the resource value required by the service to be deployed, it will automatically generate Q execution paths (Q ⁇ 1) that meet the specifications of the resource required by the service to be deployed according to the overall system status of the cloud server, and further determine these Q A total of m execution devices included in the execution path and m QoS calculation formulas corresponding to these m execution devices.
  • These m QoS calculation formulas include m QoS calculation formulas corresponding to m execution devices and m transmission links.
  • the QOS calculation formula for the execution device is used to calculate the QoS of the corresponding execution device, and the QoS calculation formula corresponding to the transmission link is used to calculate the QoS of the transmission link corresponding to the execution device.
  • the definitions of the QoS calculation formula corresponding to the execution device and the QoS calculation formula corresponding to the transmission link are as described in the embodiment corresponding to FIG. 3, and will not be repeated here.
  • the scheduling device determines the weight occupied by the execution device of each execution path in the Q execution paths.
  • the scheduling device After the scheduling device determines the execution device of each execution path in the Q execution paths, it will further determine the weight of each execution device, and the weight of the execution device represents the importance of the execution device in the corresponding execution path.
  • the scheduling device respectively sends corresponding m QoS calculation formulas to the Agents embedded in the m execution devices.
  • each execution device under the cloud server is respectively embedded with an Agent, and the execution device realizes the collection of QoS calculation formula, the calculation and transmission of QoS through the embedded Agent.
  • the scheduling device determines m execution devices of the Q execution paths, the scheduling device will send the QoS calculation formula corresponding to the m execution devices to the agents of the m execution devices.
  • Each Agent in the m execution devices calculates the QoS of the corresponding execution device and the QoS of the transmission link corresponding to the execution device according to the corresponding QoS calculation formula.
  • the agents of the m execution devices After the agents of the m execution devices respectively obtain the corresponding QoS calculation formula, they will calculate the QoS of the corresponding execution device and the QoS of the transmission link corresponding to the execution device according to the QoS calculation formula.
  • how the Agent of the executing device obtains the QoS of the corresponding executing device and the QoS of the transmission link corresponding to the executing device according to the QoS calculation formula is the same as the calculation of the QoS of the executing device and the transmission link in the corresponding embodiment in FIG. QoS is similar and will not be repeated here.
  • Each Agent in the m execution devices respectively sends the QoS of the corresponding execution device and the QoS of the corresponding transmission link to the scheduling apparatus.
  • Each Agent in the m execution devices sends the QoS of the corresponding execution device and the QoS of the corresponding transmission link obtained according to the QoS calculation formula to the scheduling device.
  • the scheduling device calculates the QoS of the Q execution paths and predicts the QoS of each execution path in the next cycle.
  • the scheduling device calculates the QoS of the Q execution paths according to the acquired QoS of the m execution devices and the QoS of the corresponding transmission link, and predicts the QoS of each execution path in the next cycle.
  • how the scheduling device calculates the QoS of the Q execution paths is similar to the QoS of the execution paths in the embodiment corresponding to FIG. 3, and will not be repeated here.
  • the final execution path of the service to be deployed can be determined according to the above-mentioned embodiment. , I won’t repeat it here.
  • the scheduling device since the scheduling device determines an execution path as the execution path of the service to be deployed based on the QoS in the current cycle of the execution path, but the execution device of the execution path not only runs the service to be deployed, Other services may also be run, which may change the resources included in the execution device.
  • the scheduling device finds that the selected execution path cannot meet the resource requirements of the to-be-deployed service during the execution of the service to be deployed, it needs to make non-prediction-based back-end adjustments to the resources of the execution path, for example, the data routing of service F
  • the scheduling device finds that the virtual machine resources at the back end of the current execution path are insufficient, which cannot meet the requirements of the calculation delay of the business F, then the unified load balancing will automatically forward the business F to the physical machine Calculate on resources.
  • the scheduling device can further predict the QoS of each execution path in the next cycle.
  • the scheduling device finds that the QoS in the next cycle of the execution path where the service to be deployed is deployed cannot meet the pre-determined QoS during the execution of the service to be deployed.
  • the execution path of the service to be deployed can be adjusted to the execution path where the QoS in the next cycle meets the preset rules according to the predicted QoS of each execution path in the next cycle to ensure the continuity of the service to be deployed Sex and stability.
  • the scheduling device determines that the calculated relationship between the QoS of the Q execution paths and the preset QoS does not meet the preset rules, the scheduling device performs resource adjustment, that is, adjusts the resource specifications of the execution equipment on any execution path until the preset QoS is satisfied.
  • resource adjustment that is, adjusts the resource specifications of the execution equipment on any execution path until the preset QoS is satisfied.
  • Set the execution path of the QoS condition (which can be referred to as the adjusted execution path), and deploy the service to be deployed on the adjusted execution path.
  • the manner in which the scheduling device performs resource adjustment is similar to the manner in which the resource adjustment is performed in the foregoing embodiment, and will not be repeated here.
  • the execution path will be different according to the different computing resources. Taking the delay as the QoS, taking the computing resources and network resources as the influence, an example of an execution path solution model:
  • c [c_1,c_2,...,c_n]
  • r [r_1,r_2,...,r_n]
  • degree of completion ⁇ [ ⁇ _1, ⁇ _2,..., ⁇ _n] ⁇ T
  • ⁇ [ ⁇ _(r ⁇ 1,) ⁇ _(r ⁇ 2,)..., ⁇ _(r ⁇ n)] ⁇ .
  • ⁇ _1), Take out all the best execution paths that can be obtained at the same time that meet the conditions, the restriction conditions are Among them, the definition: ⁇ _i t_i/(t_i 0), ⁇ _i is the minimum tolerance value, which is determined according to the maximum waiting time t_(i0) of a single task. In general, t_(i0) ⁇ t_i, ⁇ is the CPU resource idle rate (unit Is %), R is the idle rate of network resources (unit is %).
  • the embodiment of the present application can divide the dispatching device into functional modules according to the above example of the method for deploying services.
  • each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 7 shows a schematic diagram of a scheduling device, and the scheduling device provided in an embodiment of the present application may include:
  • the path selection module 701 is used for:
  • the execution path includes at least two execution devices, and obtain the quality of service QoS of each execution device; calculate the QoS of the execution path according to the QoS of each execution device; obtain the execution path of the service to be deployed Preset QoS, and finally, determine that the relationship between the QoS of the execution path and the preset QoS meets the preset rules;
  • the service deployment module 702 is used to:
  • the path selection module determines that the relationship between the QoS of the execution path and the preset QoS conforms to the preset rule, the service to be deployed is deployed on the execution path, and the at least two execution devices are used to execute the service to be deployed respectively.
  • the path selection module 701 is specifically configured to:
  • the QoS of at least one transmission link between the at least two execution devices is obtained, and the QoS of each execution device and the at least one transmission link are obtained.
  • the QoS of the path calculates the QoS of the execution path.
  • the path selection module 701 is specifically used to:
  • the path selection module 701 is specifically configured to:
  • the path selection module 701 is specifically configured to:
  • the weight of each QoS in the QoS of each execution device is determined, and the QoS of the execution path is determined according to each QoS and the corresponding weight.
  • the path selection module 701 is specifically configured to:
  • N execution paths according to the resources required by the service to be deployed, wherein the specifications of the resources included in the execution device on each execution path are not lower than the specifications of the resources required by the service to be deployed; if each of the N execution paths If the relationship between the QoS of an execution path and the preset QoS does not conform to the preset rule, resource adjustment is performed, that is, the resource specifications of the execution devices on any execution path are adjusted until an execution path meeting the preset QoS conditions is obtained.
  • the path selection module 701 is specifically further configured to:
  • the QoS of the execution path is the QoS in the current cycle of the execution path; in addition, the predicted QoS and preset QoS of the execution path will be determined The relationship also meets the preset rules;
  • the service deployment module 702 is specifically used for:
  • the path selection module determines that the relationship between the predicted QoS of the execution path and the preset QoS conforms to the preset rule, the service to be deployed is deployed on the execution path.
  • FIG. 7 describes the scheduling device in the embodiment of the present application from the perspective of a modular functional entity.
  • the following describes the scheduling device in the embodiment of the present application from the perspective of hardware processing.
  • this embodiment is A schematic diagram of an embodiment of the scheduling device, which specifically includes:
  • the scheduling device may have relatively large differences due to different configurations or performance, and may include one or more central processing units (CPU) 822 (for example, one or more processors) and one or more memories 830, One or more computer instructions 842 or data 844 are stored on the memory 830.
  • the memory 830 may be short-term storage or persistent storage.
  • the memory 830 may include one or more modules (not shown in the figure), and each module may include operations on a series of computer instructions in the scheduling device.
  • the central processing unit 822 may be configured to communicate with the memory 830, and execute operations of a series of computer instructions in the memory 830 on the scheduling device.
  • the scheduling device may also include one or more power sources 826, one or more wired or wireless network interfaces 850, one or more input and output interfaces 858, and/or one or more operating systems 841.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the usable medium may be a magnetic medium, (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state hard disk).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请实施例公开了一种部署业务的方法，该方法综合执行路径上各个执行设备的QoS来判断该执行路径提供服务的能力，即获取执行路径的QoS，并根据执行路径的QoS选择部署待部署业务的执行路径，从而综合执行路径上的各个执行设备的QoS来判断该执行路径提供服务的能力，使得执行路径的选择更为精准，避免了选择的执行路径无法满足待部署业务的需要的情况。

Description

一种部署业务的方法以及调度装置 技术领域
本申请涉及云计算领域，尤其涉及一种部署业务的方法以及调度装置。
背景技术
云平台中通常部署有调度装置，用于根据待部署业务的需求来选择部署该待部署业务的执行路径。具体地，调度装置根据待部署业务的类型，选择符合待部署业务需求的、用于执行该待部署业务的至少一个执行设备，从而确定包括所选择的至少一个执行设备的执行路径，在确定的执行路径上部署该待部署业务。
通常根据各执行设备的服务质量(Quality of Service，QoS)来选择符合待部署业务需求的执行设备。在确定执行路径时，调度装置仅关注某一个或者某几个执行设备的QoS。这是由于不同待部署业务对执行设备的QoS的要求不同。例如，通信类的待部署业务对用于转发的执行设备的QoS要求较高，对用于计算的执行设备的QoS要求较低；相反，金融类的待部署业务对用于转发的执行设备的QoS要求较低，对用于计算的执行设备的QoS要求较高。然而，基于某一个或者某几个执行设备的QoS选择执行路径，这导致调度装置所选择的执行路径只能在大致上满足待部署业务的需求，那么所选择的执行路径可能无法满足待部署业务的需求。
发明内容
本申请提供了一种部署业务的方法以及调度装置，用于计算执行路径的QoS，并根据执行路径的QoS选择部署待部署业务的执行路径，使得执行路径的选择更为精准，避免了选择的执行路径无法满足待部署业务需要的情况。
本申请实施例第一方面提供了一种部署业务的方法，具体包括：调度装置首先可以根据待部署业务的标识确定该待部署业务可执行的路径(也可称为执行路径)，该执行路径包括有至少两个执行设备；之后，调度装置进一步获取该执行路径的每一个执行设备的QoS，并根据该执行路径的每一个执行设备的QoS计算得到该执行路径的QoS；最后，比较该执行路径的QoS与该待部署业务的预设QoS(该待部署业务的预设QoS由用户事先在调度装置中设置，用户设置完预设QoS之后，调度装置就获取到了该待部署业务的预设QoS)的关系是否符合预设规则，如果确定符合预设规则，调度装置将会在该执行路径上部署该待部署业务，并且该执行路径上的执行设备都将用于执行该待部署业务。
在本申请实施例中，首次提出了综合执行路径上各个执行设备的QoS来判断该执行路径提供服务的能力，获取执行路径的QoS，并根据执行路径的QoS选择部署待部署业务的执行路径，从而综合执行路径上的各个执行设备的QoS来判断该执行路径提供服务的能力，并提供了精确计量执行路径提供服务的能力(即执行路径的QoS)，使得执行路径的选择更为精准，避免了选择的执行路径无法满足待部署业务需要的情况。
在一种可能的实现方式中，调度装置在根据每个执行路径的QoS计算执行路径的QoS之前，调度装置还可以获取该执行路径每两个执行设备之间的传输链路的QoS。此时调度 装置计算执行路径的QoS就是根据该执行路径上每个执行设备的QoS以及每两个执行设备之间的传输链路的QoS来计算得到该执行路径的QoS。
在本申请实施例中，为了使获取到的执行路径的QoS更加精确，除了考虑该执行路径的各个执行设备的QoS，还考虑到了该执行路径各个执行设备之间传输链路的QoS。也就是说，调度装置除了获取执行路径的各个执行设备的QoS之外，还会获取各个执行设备之间的传输链路的QoS，根据各个执行设备的QoS和各个传输链路的QoS来确定执行路径的QoS，从而使得整个执行路径上计算得到的QoS更精确。
在一种可能的实现方式中，调度装置获取执行路径的每一个执行设备的QoS可以通过如下方式获取：首先调度装置确定执行路径中的每个执行设备分别对应的QoS计算公式，一个执行设备就对应有一个QoS计算公式；之后，调度装置将每个QoS计算公式发送至对应的执行设备，由对应的执行设备根据该QoS计算公式计算得到该执行设备的QoS；最后，调度装置再从该执行路径的每个执行设备处获取到该执行设备的QoS。
在本申请实施例中，具体阐述了调度装置如何获取执行设备的QoS(即通过向各执行设备发送对应的QoS计算公式)，具备实操性。
在一种可能的实现方式中，调度装置获取执行路径的每两个执行设备之间的传输链路的QoS可以通过如下方式获取：首先调度装置确定执行路径中每两个执行设备之间的传输链路，并确定每个传输链路分别对应的QoS计算公式，一个传输链路对应有一个QoS计算公式；之后，调度装置将每个传输链路对应的QoS计算公式发送至连接成该传输链路的两个执行设备中的至少一个，并由该至少一个执行设备根据对应的QoS计算公式计算得到该传输链路的QoS；最后，调度装置再从上述至少一个执行设备处获取到对应传输链路的QoS。
在本申请实施例中，与调度装置获取执行设备的QoS类似，这里进一步阐述了调度装置如何获取两个执行设备之间传输链路的QoS(即通过向各执行设备发送与该传输链路对应的QoS计算公式)，具备广泛适用性。
在一种可能的实现方式中，调度装置根据执行路径上每个执行设备的QoS计算执行路径的QOS具体还可以是：首先确定该执行路径中每个执行设备的QoS所占的权重，执行设备的QoS所占权重表示该执行设备在执行路径的重要程度；最后，根据每个执行设备的QoS及对应QoS所占权重来确定执行路径的QoS。
在本申请实施例中，调度装置通过执行设备的QoS的权重进一步明确了执行路径中每个执行设备的重要程度，更具有针对性。
在一种可能的实现方式中，调度装置确定执行路径可以是先根据待部署业务所需资源确定N(N≥1)个执行路径，使得这N个执行路径中每个执行上的执行设备包括的资源的规格不低于该待部署业务所需资源的规格；之后，调度装置将对这N个执行路径的QoS进行计算，得到N个执行路径的QoS，如果调度装置确定这N个执行路径的QoS中的任何一个QoS与预设QoS的关系都不符合预设规则，则调度装置就会调整任意执行路径上的执行设备的资源规格直至获得满足预设QoS条件的执行路径(可称为调整后的执行路径)；最后，调度装置将会在上述调整后的执行路径上部署该待部署业务。
在本申请实施例中，进一步阐释了当N个执行路径的QoS与预设QoS的关系都不符合预设规则时，调度装置将会执行资源调整使得待部署业务部署在调整后的执行路径上， 保证了待部署业务的可靠性和稳定性。
在一种可能的实现方式中，调度装置确定执行路径的QoS(即执行路径当前周期内的QoS)与预设QoS的关系符合预设规则之后，调度装置还可以对该执行路径下一周期内的QoS进行预测，得到该执行路径的预测QoS，调度装置进一步确定该执行路径的预测QoS与预设QoS的关系符合也符合预设规则，则调度装置确定将该待部署业务部署在该执行路径上。
在本申请实施例中，调度装置不仅会比较各个执行路径当前周期内的QoS，还会比较各执行路径下一周期内的QoS，只有执行路径当前周期内的QoS以及执行路径下一周期内的QoS与预设QoS的关系均符合预设关系，才会在该执行路径上部署上述待部署业务，这样就进一步保障了待部署业务的稳定性。
本申请实施例第二方面提供了一种调度装置，该调度装置通过硬件或通过硬件执行相应的软件实现如上述实施例中的方法，所述硬件或所述软件包括一个或多个与上述实施例中的方法相对应的模块。例如，该调度装置可以包括：
路径选择模块以及业务部署模块；
该路径选择模块，用于：确定待部署业务的执行路径，该执行路径包括至少两个执行设备；获取每个执行设备的服务质量QoS；根据该每个执行设备的QoS计算该执行路径的QoS；获取该待部署业务的预设QoS；确定该执行路径的QoS与该预设QoS的关系符合预设规则；
该业务部署模块，用于：在路径选择模块确定执行路径的QoS与预设QoS的关系符合预设规则之后，在该执行路径上部署该待部署业务，该至少两个执行设备用于分别执行该待部署业务。
在一种可能的实现方式中，该路径选择模块具体用于：
在该根据每个执行设备的QoS计算该执行路径的QoS之前，获取该至少两个执行设备之间的至少一个传输链路的QoS；并根据该每个执行设备的QoS和该至少一个传输链路的QoS计算该执行路径的QoS。
在一种可能的实现方式中，该路径选择模块具体还用于：
确定每个执行设备对应的至少两个第一QoS计算公式，一个执行设备对应一个QoS计算公式；并将每个第一QoS计算公式发送至对应的执行设备；最后，获取每个执行设备根据对应的第一QoS计算公式得出的每个执行设备的QoS。
在一种可能的实现方式中，该路径选择模块具体用于：确定该至少一个传输链路对应的至少一个第二QoS计算公式；并将每个第二QoS计算公式发送至对应的传输链路上的执行设备；最后，获取该至少一个传输链路上的执行设备根据对应的第二QoS计算公式得出的每个传输链路的QoS。
在一种可能的实现方式中，该路径选择模块具体用于：确定每个执行设备的QoS中每个QoS所占权重；并根据每个QoS以及对应的权重确定该执行路径的QoS。
在一种可能的实现方式中，该路径选择模块具体用于：根据该待部署业务所需资源确定N个执行路径，其中，每个执行路径上的执行设备包括的资源的规格不低于该待部署业务所需资源的规格；确定该N个执行路径的每一个执行路径的QoS与该预设QoS的关系均不符合该预设规则，并调整任意执行路径上的执行设备的资源规格直至获得满足 预设QoS条件的执行路径。
在一种可能的实现方式中，在路径选择模块确定执行路径的QoS与预设QoS的关系符合预设规则之后，该路径选择模块具体还用于：对该执行路径下一周期内的QoS进行预测，得到该执行路径的预测QoS，该执行路径的QoS为该执行路径当前周期内的QoS；确定该执行路径的预测QoS与预设QoS的关系符合预设规则；
该业务部署模块具体用于：在路径选择模块确定执行路径的预测QoS与预设QoS的关系符合预设规则之后，在该执行路径上部署该待部署业务。
本申请实施例第三方面提供一种计算设备，该计算设备包括一个或多个处理器、一个或多个存储器；其中，该一个或多个存储器存储有一个或多个计算机指令，当该计算机指令被该一个或多个处理器执行时，使得该计算设备执行上述实施例任意一种可能实现方式的方法。
本申请实施例第四方面提供一种计算机可读存储介质，该计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机可以执行上述实施例任意一种可能实现方式的方法。
附图说明
为了更清楚地说明本申请实施例的技术方法，下面将对实施例中所需使用的附图作以简单地介绍。
图1为本申请实施例的一个系统框架图；
图2为本申请实施例业务B的路径b1的示意图；
图3为本申请实施例部署业务的方法的一个示意图；
图4为本申请实施例调度装置经过资源调整后业务B的路径b2的示意图；
图5为本申请实施例中QoS上报给调度装置的流程示意图；
图6为本申请实施例部署业务的方法的一个示意图；
图7为本申请实施例调度装置的一个示意图；
图8为本申请实施例调度装置的另一个示意图。
具体实施方式
下面将结合本申请中的附图，对本申请提供的实施例中的方案进行描述。
首先，需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
云计算是一种新兴的商业服务模式和计算模型，它通过互联网给接入云计算的用户提供各种计算、存储等服务，这种服务不是依托于本地计算机，而是依托于至少一台云服务器。云服务器可提供计算、存储和网络等各种资源，基于云服务器提供的资源为用户提供服务。当用户需要执行待部署业务，可根据待部署业务的需要给该待部署业务分 配云服务器上的资源。也就是说，云服务器这种按需分配的特性可以根据任务需求“量入为出”地分配云服务器的资源。
目前传统的分配云服务器的资源的方式主要有两种：一种是基于不同优先级策略的资源分配，即给予高优先级的待部署业务更多的资源，低优先级的待部署业务则获得更少的资源。然而这种方式给予优先级高的待部署业务充足的资源使其执行，能够保证用户对该待部署业务的执行结果具有较高的满意度，但往往会造成资源的过剩以致浪费，而那些优先级低的待部署业务，通常由于得不到适量的资源而处于饥饿状态，不能满足用户最基本的使用体验。这种由于优先级造成的资源两级分化，导致资源的利用率不高。基于此，为了提高服务满意度及提高资源利用率，另一种方式就是基于QoS的资源分配，具体地，根据特定的待部署业务所需的QoS进行资源分配。QoS是指执行设备或者服务为特定的待部署业务提供服务的能力，QoS即QoS的值。然而，基于某一个或者某几个执行设备的QoS选择执行路径，这导致调度装置所选择的执行路径只能在大致上满足待部署业务的需求，导致所选择的执行路径可能无法满足待部署业务的需求。
在详细介绍本申请实施例之前，先对本申请实施例所涉及的系统架构图进行说明，具体如图1所示，其中，调度装置用于根据用户输入的用户请求(如：输入某项待部署业务)对云服务器的资源进行分配，具体的，负责用户输入任务的处理、状态数据的保存、资源的调度等，根据用户请求，调度装置统一调度云服务器下该用户请求涉及的各个执行设备包括的资源，如图1中，当用户输入一个待部署业务时，调度装置根据该待部署业务所需资源的规格就可以调度执行设备1、执行设备2、……、执行设备n(假设该待部署业务涉及的执行设备就是执行设备1、执行设备2、……、执行设备n)所包括的资源。需要说明的是，云服务器中的最小功能单元都可以称为执行设备，执行设备包括的资源规格不小于待部署业务所需的资源规格。在本申请实施例中，执行设备表示部署待部署业务的备选设备，执行路径表示部署待部署业务的备选路径。执行设备可以包括计算设备、存储设备以及网络设备等。计算设备包括物理机、虚拟机、容器等。还需要说明的是，该调度装置可以和执行具体业务的功能装置一起部署在同一个云服务器上，也就是说是在该云服务器中置入相应的软件/硬件来实现资源的调度；该调度装置也可以部署在多个不同的云服务器上，如多个虚拟机共同作为调度装置来实现资源的调度；该调度装置还可以单独部署(图1中所示的调度装置就为单独部署)，具体此处不做限定。
此外，执行路径则包括完成一个业务时该业务相关的数据所经过的执行设备和各执行设备之间的传输链路，以图2为例进行说明：若业务B是一个存储业务，那么要执行业务B，业务B相关的数据首先到达私有网络(virtual private cloud，VPC)，在VPC中转发至VPC的隧道端点(Virtual Tunnel Endpoint，VTEP)1。VTEP1与VTEP2相连，VTEP2可以是物理交换机，图2中示例性地绘制出一种物理交换机—架顶交换机(Top of Rack，ToR)。数据经VTEP1与VTEP2之间形成的传输链路1到达ToR，然后经过传输链路2到达裸机1，最后经过传输链路3到达最终的存储区域网络(storage area network，SAN)，最终完成业务B的执行。此时，VPC、ToR、裸机1、SAN、VPC与ToR之间的传输链路1、ToR与裸机1之间的传输链路2、裸机1与SAN之间的传输链路3就构成该业务B的执行路径(可称为路径b1)，VPC、ToR、裸机1、SAN就是该执行路径的执行设备，业务的执行路径至少包括一个执行设备。
请参照图3，本申请实施例一种部署业务的方法的具体实现方式如下：
301、确定待部署业务的执行路径，该执行路径包括至少两个执行设备。
调度装置确定待部署业务的执行路径可以有很多种方式，具体此处不做限定。下面示意其中一种确定方式：
当用户申请云服务器上的资源来执行待部署业务时，调度装置接收到用户请求，用户请求携带有该待部署业务的业务标识。调度装置获取到业务标识，根据该业务标识就可以确定用户要执行的待部署业务所需要的资源的类型(业务标识与待部署业务的资源类型具备映射关系)。此外，调度装置还需要获取到该待部署业务所需资源的规格，然后调度装置根据该待部署业务所需资源的类型和规格来确定执行路径。以业务B为例，业务B为一个存储业务，假设业务B需要的存储资源为2GB(即业务B的资源的规格为存储资源2GB)，当用户输入业务B时，调度装置同时就能够获取到该业务B的标识(假设该业务B的业务标识为“1122”)，调度装置根据该业务标识“1122”就能够确定业务B所需资源的类型和规格为上述的存储资源2GB，之后，在满足存储资源2GB的前提下，调度装置确定云服务器下能够提供2GB存储资源的执行设备(如：执行设备VPC、ToR、裸机1、SAN能够提供2GB存储资源)，那么该执行设备VPC、ToR、裸机1、SAN以及它们之间的传输链路1、传输链路2、传输链路3所构成的路径(即图2中的路径b1)就被调度装置确定为该待部署业务可执行的路径(也可称为执行路径)。
此时确定的执行路径仅满足待部署业务所需资源的要求，在下面的步骤中，通过进一步判断来确定是否将该执行路径作为部署该待部署业务的执行路径。
302、获取每个执行设备的QoS。
调度装置确定好执行路径之后，将进一步获取上述执行路径的每个执行设备的QoS。执行设备的QoS根据执行设备上包括的资源的规格确定。执行设备能够给待部署业务提供多少资源就称为执行设备包括的资源的规格。调度装置获取上述每个执行设备的QoS的方式可以有多种，具体此处不做限定。以下示意几种获取执行路径上每个执行设备的QoS的方式：
A、由QoS计算公式计算得到执行路径上每个执行设备的QoS。
调度装置可以将与上述每个执行设备分别对应的QoS计算公式(即一个执行设备对应有一个QoS计算公式)发送至各自对应的执行设备，每个执行设备根据获取到的QoS计算公式计算得到该执行设备的QoS，例如，当执行设备为物理机，物理机的QoS可以是根据主机的中央处理器(central processing unit，CPU)使用率、内存使用率、带宽、时延等的计算公式计算而来。之后，上述每个执行设备再将根据该QoS计算公式计算得到的该执行设备的QoS发送给调度装置。
需要说明的是，上述与执行设备对应的QoS计算公式可以是事先内置在调度装置中，当调度装置确定了执行路径的各个执行设备时，再将每个执行设备的QoS计算公式发送至对应的执行设备；QoS计算公式也可以是存在于与该调度装置有连接的设备A(包括虚拟设备、物理设备等)中，当调度装置确定了执行路径后，可以向与调度装置有连接的上述设备A发送请求指令，该请求指令用于触发该设备A向调度装置发送执行路径上各个执行设备对应的QoS计算公式，之后，调度装置再将各个QoS计算公式发送至对应的执行设备。该第一QoS计算公式还可以是事先就内置于各个执行设备中的，当调度装置 确定了执行路径后，调度装置会给执行路径的各个执行设备发送指令，该指令用于触发各个执行设备根据自身内置的QoS计算公式计算该执行设备的QoS。在本申请实施例中，对执行设备如何获取到QoS计算公式不做限定。
B、由执行路径上每个执行设备包括的资源的规格与QoS的映射关系得到上述每个执行设备的QoS。
调度装置上可以有一个映射关系库，该映射关系库中存有每个执行设备的包括的资源的规格与对应执行设备的QoS的映射关系，当需要获取执行路径的每个执行设备的QoS时，那么调度装置可以获取到上述每个执行设备当前时刻所包括的资源的规格，并在映射关系库中根据执行设备所包括的资源的规格与执行设备的QoS的映射关系查找到当前时刻该执行设备的QoS。为便于理解，下面举例进行说明：
若调度装置确定了业务C的执行路径为路径c，该路径c包括的执行设备有设备a、设备b、设备c。此时调度装置就需要获取到设备a、设备b、设备c的QoS，那么调度装置会先获取到设备a、设备b、设备c在当前时刻所包括的资源的规格(假设设备a、设备b、设备c当前时刻所包括的资源的规格分别为x1、y2、z3)，并在映射关系库(表1示意了一种映射关系库)中查找设备a包括的资源的规格为x1时对应的设备a的QoS(假设为a-QoS-1)、设备b包括的资源的规格为y2时对应的设备b的QoS(假设为b-QoS-2)、设备c包括的资源的规格为z3时对应的设备c的QoS(假设为c-QoS-3)，这样调度装置就获取到设备a、设备b、设备c当前时刻的QoS。
表1、映射关系库
Figure PCTCN2020094442-appb-000001
303、根据每个执行设备的QoS计算执行路径的QoS。
调度装置获取到执行路径上各个执行设备的QoS之后，将根据上述各个执行设备的QoS计算执行路径的QoS。在本申请实施例中，计算执行路径的QoS有多种实现方式，具体此处不做限定。以下示意几种计算执行路径的QoS的方式：
A、对执行路径的各个执行设备的QoS取算术平均值得到。
调度装置对获取到的执行路径的各个执行设备的QoS相加后取平均值得到执行路径的QoS。即执行路径的QoS(可记为路径QoS)为：路径QoS＝(QoS1+QoS2+……+QoSi+……+QoSn)/n，其中，QoSi为执行路径的第i执行设备的QoS。例如，以图2业务B的路径b1为例进行说明：假设VPC由对应的计算公式11计算得到的QoS为QoS1、ToR由对应的计算公式12计算得到的QoS为QoS2、裸机1由对应的计算公式13计算得到的QoS为QoS3、SAN由对应的计算公式14计算得到的QoS为QoS4。那么业务B在该路径b1的QoS就为：b1路径QoS＝(QoS1+QoS2+QoS3+QoS4)/4。
B、对执行路径的各个执行设备的QoS做加权求和运算得到。
调度装置确定执行路径之后，还将进一步确定执行路径各个执行设备的权重，执行设备所占权重表示该执行设备在执行路径的重要程度。之后，调度装置对该执行路径的各个执行设备的QoS做加权求和运算，得到执行路径的QoS。即执行路径的QoS(可记为路径QoS)为：路径QoS＝a1*QoS1+a2*QoS2+……+ai*QoSi+……+an*QoSn，其中，QoSi为执行路径的第i执行设备的QoS，ai为第i执行设备的权重。例如，依然以图2业务B的路径b1为例进行说明：假设VPC由对应的计算公式11计算得到的QoS为QoS1且权重为0.1、ToR由对应的计算公式12计算得到的QoS为QoS2且权重为0.1、裸机1由对应的计算公式13计算得到的QoS为QoS3且权重为0.5、SAN由对应的计算公式14计算得到的QoS为QoS4且权重为0.3。那么业务B在该路径b的QoS就为：b1路径QoS＝0.1*QoS1+0.1*QoS2+0.5*QoS3+0.3*QoS4。
304、获取待部署业务的预设QoS。
在本申请实施例中，用户可以对业务(也可称为待部署业务)的QoS自主设置，可以是调度装置首先根据用户输入的待部署业务的业务类型、性质等智能推荐给用户针对该待部署业务的普遍采用的一个QoS(可称为常规QoS)，然后用户参考该推荐的常规QoS再自行设置该待部署业务的QoS(用户可以不做修改而直接就将该常规QoS作为预设QoS，也可以在该常规QoS的基础上根据自身喜好稍作调整后作为预设QoS，具体此处不做限定)；用户也可以完全按照自己的需求直接对该待部署业务的QoS进行设置得到预设的QoS，具体此处对预设QoS如何设定的依据不做限定。用户对待部署业务的QoS设置完成之后，调度装置就可以获取到针对该待部署业务的预设QoS。
需要说明的是，步骤304可以是在步骤301之前进行，也可以是在步骤303之后进行，还可以是在步骤301至步骤303之间的任何两个步骤之间进行，还可以是与步骤301至步骤303中的任意一个步骤同时进行，具体此处不做限定。
305、确定执行路径的QoS与预设QoS的关系符合预设规则，在该执行路径上部署该待部署业务。
调度装置获取到待部署业务在执行路径的QoS以及用户针对该待部署业务的预设QoS之后，将判断执行路径的QoS与该预设QoS的关系是否符合预设规则，如果调度装置确定该执行路径的QoS与该预设QoS的关系符合预设规则，则在该执行路径上部署该待部署业务。需要说明的是，在本申请实施例中，预设规则可以是执行路径的QoS大于预设QoS，也可以是执行路径的QoS减去预设QoS后得到的数值大于一个预设的正数值，具体此处对预设规则不做限定。
在本申请实施例中，首次提出了如何获得待部署业务可执行的路径(即执行路径)的QoS，并通过比较待部署业务的预设QoS与执行路径的QoS来决定待部署业务最终在执行时是否部署在该执行路径上。此外，本申请实施例还首次提出了待部署业务的QoS可以由用户自行设置，充分考虑到了不同用户对同样待部署业务的QoS的偏好，提高了待部署业务的服务满意度。
需要说明的是，在本申请的一些实施方式中，为了使获取到的执行路径的QoS更加精确，除了考虑执行路径的各个执行设备的QoS，还可以考虑各个执行设备之间传输链路的QoS，即在步骤303之前，调度装置还可以进一步获取执行路径的各个执行设备之间的传输链路的QoS，那么步骤303获取到的执行路径的QoS就是根据执行路径的各个执行设 备的QoS以及执行路径中各个传输链路的QoS计算得到。
在本申请实施例中，调度装置获取执行路径各个执行设备的QoS可以依照图3对应的实施例所述的方式得到，此处不予赘述。这里具体阐述调度装置如何获取传输链路的QoS：首先调度装置确定执行路径中涉及到的传输链路，并确定每个传输链路对应的QoS计算公式，之后将每个传输链路对应的QoS计算公式发送至对应传输链路上的两个执行设备中的一个，由获取到该QoS计算公式的执行设备计算对应传输链路的QoS并将该传输链路的QoS发送给调度装置。以图2业务B的路径b1为例进行说明，调度装置确定业务B的路径b1之后，就可以确定路径b1的传输链路包括传输链路1、传输链路2、传输链路3，并确定出传输链路1、传输链路2、传输链路3分别对应的QoS计算公式为计算公式21、计算公式22、计算公式23，其中传输链路1上的两个执行设备为VPC和ToR，传输链路2上的两个执行设备为ToR和裸机1，传输链路3上的两个执行设备为裸机1和SAN；之后，调度装置可以将计算公式21发送至VPC或ToR(假设发送给VPC)、将计算公式22发送至ToR或裸机1(假设发送给裸机1)、将计算公式23发送至裸机1或SAN(假设发送给SAN)，之后VPC、裸机1、SAN就分别根据计算公式21、计算公式22、计算公式23计算得到传输链路1、传输链路2、传输链路3的QoS，最后VPC、裸机1、SAN分别将得到的上述3个传输链路的QoS发送给调度装置。
需要说明的是，在本申请的一些实施方式中，调度装置还可以将传输链路对应的QoS计算公式发送给传输链路上的两个执行设备，使得这两个执行设备都对该同一传输链路的QoS进行计算，当两个执行设备向调度装置返回该传输链路的两个QoS时，将上述获取到的同一传输链路上的两个QoS取算术平均值作为执行路径的传输链路的QoS。在本申请实施例中，调度装置除了获取执行路径的各个执行设备的QoS，还获取各个执行设备之间的传输链路的QoS，根据各个执行设备的QoS和各个传输链路的QoS来确定执行路径的QoS，使得整个执行路径上计算得到的QoS更精确。
这里需要说明的是，在本申请的上述实施例中，调度装置确定执行路径可以根据但不限于如下方式得到：先根据业务标识确定N(N≥1)个执行路径，其中，每个执行路径的执行设备包括的资源的规格不低于该待部署业务所需资源的规格，调度装置确定这N个执行路径中每一个执行路径的QoS的方法以及获取每个执行路径的QoS的方法可以参照上述实施例中步骤301-305，此处不予赘述。也就是说，调度装置会先根据待部署业务所需资源的规格(如：业务D所需带宽资源不低于100kbps)确定几个执行路径(如：路径d1、路径d2、路径d3都能够给业务D提供不低于100kbps的带宽资源)。这里需要分几种情况分别进行说明如何确定待部署业务最终部署的执行路径：
A、当N＝1时，即执行路径只有一个时。
以图2对应的业务B为例进行说明，若调度装置确定满足业务B所需资源值(假设业务B所需资源值为存储资源2GB)只有路径b1，那么调度装置可以直接依照上述实施例中所述的获取执行路径的QoS的方式获取到路径b1的QoS，并将路径b1的QoS与用户对业务B设置的预设QoS进行比较，若路径b1的QoS与业务B的预设QoS符合预设规则，则将业务B部署在该路径b1上。
B、当N≥2时，即执行路径至少有两个时。
依然以图2对应的业务B为例进行说明，若调度装置确定满足业务B所需资源值的 执行路径不只有路径b1，还有路径b2和路径b3，那么业务B的执行路径就是这3个执行路径中的任意一个。此时，调度装置可以依据上述实施例中所述的获取执行路径的QoS的方式分别获取到路径b1、路径b2和路径b3的QoS，之后，将获取到路径b1、路径b2和路径b3的QoS分别与用户对业务B预先设置的预设QoS进行比较，根据比较结果确定业务B最终部署在哪个执行路径上，这里根据比较结果分别进行论述：
a、若调度装置确定N个执行路径中只有一个执行路径的QoS与预设QoS的关系符合预设规则，则将待部署业务部署在QoS与预设QoS符合预设规则的那个执行路径上。
依然以图2对应的业务B为例进行说明，若路径b1、路径b2和路径b3中只有一个执行路径(如路径b1)的QoS与预设QoS的关系符合预设规则，那么则将业务B部署在路径b1上。
需要说明的是，在本申请的一些实施方式中，为使得待部署业务执行时更稳定(如：不用频繁切换目标执行路径)，还可以进一步对该执行路径的QoS(上述所称的执行路径的QoS均指执行路径当前周期内的QoS)进行预测，得到执行路径下一周期内的预测QoS，调度装置确定执行路径的预测QoS与该待部署业务的预设QoS的关系也符合所述预设规则后，就将该待部署业务部署在该执行路径上。以业务B为例，即还可以进一步对该路径b1的QoS进行预测，得到路径b1下一周期内的预测QoS，调度转置确定路径b1的预测QoS与业务B的预设QoS的关系也符合所述预设规则后，就会将业务B部署在路径b1上。
b、若调度装置确定N个执行路径中的至少两个执行路径的QoS与预设QoS的关系均符合预设规则，则将业务部署在这至少两个执行路径中的的其中一个上。
依然以图2对应的业务B为例进行说明，若路径b1、路径b2和路径b3中部分执行路径(如路径b1和路径b2)或全部执行路径的QoS与预设QoS的关系均符合预设规则(可将这些符合预设规则的执行路径统称为待选择执行路径)，那么确定业务B待部署的执行路径可以是：
1)确定每个待选择执行路径的执行设备包括的资源的规格，选择将业务B部署在执行设备包括的资源最多的执行路径上。
2)将业务B部署在待选择执行路径的QoS最好的那个执行路径上。
3)将业务B部署在待选择执行路径中传输链路数量最少的那个执行路径上。
4)对所有待选择执行路径下一周期的QoS进行预测，得到每个待选择执行路径的预测QoS，若调度装置确定所有待选择执行路径的预测QoS中存在预测QoS(可称为目标预测QoS)与预设QoS的关系符合预设规则，则当该目标预测QoS只有一个时，就将业务B部署在该目标预测QoS对应的待选择执行路径上；当该目标预测QoS有至少两个时，则可以根据上述方式1)、2)、3)、4)中类似的方式将业务B部署对应的执行路径上。由上述可知，在调度装置确定有多个执行路径的QoS与预设QoS的关系都符合预设规则的情况下，如何从这多个执行路径中确定一个执行路径作为部署该待部署业务的执行路径有多种实现方式，具体此处不做限定。
c、若调度装置确定N个执行路径的QoS与预设QoS的关系均不符合预设规则，则执行资源调整。
若调度装置确定N个执行路径的QoS与预设QoS的关系均不符合预设规则，调度装 置可以根据待部署业务所需资源值对上述N个执行路径的资源进行调整，通过调整任意执行路径上的执行设备的资源规格直至获得满足预设QoS条件的执行路径(可称为调整后的执行路径)，之后，将待部署业务部署在该调整后的执行路径上。
依然以图2对应的业务B为例进行说明，若调度装置发现当前业务B在每个可选的执行路径(即路径b1、路径b2和路径b3)计算得到的QoS与预设QoS的关系均不符合预设规则，则会触发后端资源调整，调度装置后端可以通过迁移、暂停、限制等手段将业务B的可选的执行路径上部署的其他业务(如业务D、业务E等)在这条执行路径上各个执行设备占用的资源使用量降低，以减少对业务B的资源争抢，提升业务B在这些执行设备的QoS，从而使得业务B在这些执行设备构成的执行路径的QoS也得以提升。也就是说，通过改变执行路径涉及的执行设备上其他业务的资源预留或者减少资源争抢，可以提升业务B在该执行路径的QoS。需要说明的是，调度装置对资源调整的方式除了可以是通过迁移、暂停、限制等手段增加执行路径的执行设备所包括的资源的规格使得执行设备的QoS提升之外，还可以是额外在该执行路径的执行设备上增加资源使得执行设备的QoS提升，具体此处对资源调整的方式不做限定。之后，经过上述资源调整之后，业务B的每个可选的执行路径(即路径b1、路径b2和路径b3)的QoS可能都会发现变化，那么只要对变化后的每个执行路径的QoS再与预设QoS进行比较，若调度装置确定至少存在一个执行路径经过资源调整后的QoS与预设QoS的关系符合预设规则，就可以根据上述实施例类似的方式确定出业务B待部署的执行路径；若调度装置确定调整资源后依然没有一个执行路径的QoS与预设QoS的关系符合预设规则，则再继续进行资源调整，直至调整后至少存在一个执行路径的QoS与预设QoS的关系符合预设规则。以图4为例进行说明：经过资源调整后，调度装置确定业务B的路径b2的QoS与预设QoS的关系符合预设规则，则将业务B部署在路径b2上，该路径b2包括VPC、高质量的多层虚拟交换机OVS、虚机2、分布式块存储、通过VPC的VTEP1与OVS的VTEP3连接的传输链路4、连接OVS与虚机2的传输链路5、连接虚机2与分布式块存储的传输链路6。
上述实施例确定待部署业务待部署的执行路径的过程是：调度装置先根据待部署业务的标识确定满足待部署业务所需资源的规格的N个执行路径，并计算得到这N个执行路径的N个QoS，再确定这N个QoS中有几个QoS与预设的QoS的关系满足预设规则(假设存在W个QoS符合预设规则)，最后再根据上述所述的确定方式从这W个QoS中选出一个QoS对应的执行路径，并将该待部署业务部署在选出的执行路径上。
需要说明的是，在本申请的一些其他实施方式中，调度装置在确定了N个执行路径之后，可以按照预设方法逐条计算这N个执行路径的QoS，计算出一个执行路径的QoS，就将该执行路径的QoS与预设的QoS进行比较，当调度装置确定这个执行路径的QoS与预设的QoS的关系符合预设规则，就直接将该待部署业务部署在这个满足要求的执行路径上，而不再计算剩余的执行路径的QoS，从而减少了计算量。为便于理解，以图2对应的业务B为例进行说明：若调度装置根据对应的业务标识确定当前业务B可选的执行路径有3个(即N＝3)，分别为路径b1、路径b2和路径b3，调度装置将根据预设方法(假设预设方法是哪条执行路径先被确定下来，就先计算哪条执行路径的QOS)确定计算这3个执行执行路径的QoS的先后顺序分别为先计算路径b1的QoS、再计算路径b2的QoS、最后计算路径b3的QoS。那么调度装置将首先计算路径b1的QoS，计算得到路径b1的 QoS之后就将该QoS与预设的QoS进行比较，假设比较的结果为路径b1的QoS与预设的QoS的关系不符合预设规则，则调度装置继续计算路径b2的QoS，计算得到路径b2的QoS之后就将该QoS与预设的QoS进行比较，假设比较的结果为路径b2的QoS与预设的QoS的关系符合预设规则，则调度装置直接就确定该路径b2为业务B待部署的路径，并在该路径b2上部署业务B，而不再计算路径b3的QoS。
还需要说明的是，在本申请的一些实施方式中，调度装置还可以直接根据待部署业务所需的资源的规格确定出一个执行路径(即不需要先确定出N个满足待部署业务资源的规格的执行路径)，每确定出一个执行路径就对该执行路径的QoS进行计算，并与预设QoS比较，若调度装置确定该执行路径的QoS与预设的QoS符合预设规则，则直接确定该执行路径为待部署业务待部署的执行路径，并将该待部署业务部署在该执行路径上，而不再去确定是否还有其他执行路径满足待部署业务所需资源的规格。依然以图2对应的业务B为例进行说明：若调度装置根据业务B所需资源的规格首先确定出当前业务B的一个执行路径(假设为路径b1)，那么调度装置就直接计算路径b1的QOS，并将路径b1的QoS与预设的QoS进行比较，假设比较的结果为路径b1的QoS与预设的QoS的关系不符合预设规则，那么调度装置继续根据业务B所需资源的规格再确定出当前业务B的另一个执行路径(假设为路径b2)，并计算路径b2的QOS，假设比较的结果为路径b2的QoS与预设的QoS的关系符合预设规则，那么调度装置直接就确定该路径b2为业务B待部署的执行路径，并将业务B部署在该路径b2上，而不再考虑是否还存在其他执行路径可满足业务B所需的资源的规格。
从图1以及图4中可以看出，当调度装置确定待部署业务的所有可选的执行路径(即N个执行路径)的QoS与预设QoS的关系都不符合预设规则时，调度装置执行资源调整可以做到跨虚机、容器、物理机等执行路径的调度，而目前已有的云服务器无法做到这一点，以分布式资源调度程序(distributed resource scheduler，DRS)系统为例，DRS系统目前也只能根据主机的性能状况做虚机层面的迁移，而无法做到跨虚机、容器、物理机等的统一调度；再以虚拟机VMware为例，VMware的DRS也只能通过基本的CPU、内存能做后端资源的调整，无法做到对待部署业务的执行路径进行切换调整。
需要说明的是，在本申请的一些实施方式中，调度装置可以事先对所有能够在云服务器下执行的业务的执行路径进行分类，如，将能提供高存储资源的执行路径分为一大类、能提供高网络资源的执行路径分为一大类、能提供高计算资源的执行路径分为一大类等，每一类别下的执行路径又可以根据能提供资源的范围分成几小类。每大类以及每小类都有相应的标识，这样当有待部署业务需要部署时，根据该待部署业务所需资源的规格，调度装置可以通过类别标识更快的确定满足该待部署业务所需资源的规格的所有可选的执行路径(即N个执行路径)，提高了调度装置的确定部署该待部署业务的执行路径的速度。
还需要说明的是，在本申请的一些实施方式中，除了可以是对执行路径涉及的执行设备的资源进行调整使得执行设备根据QoS计算公式计算得到的QoS得到提升之外，还可以是调度装置调整执行设备对应的QoS计算公式使得执行设备的QoS得到提升，调度装置通过调整QoS计算公式的前提是对于待部署业务来说执行路径所包括的资源的规格其实是够用的，只是由于QoS计算公式根据的选择不同导致执行设备的所包括的资源没 有被充分利用，例如，执行设备主机的QoS可以是根据主机的中央处理器(central processing unit，CPU)使用率、内存使用率、带宽、时延等的QoS计算公式计算而来。如果CPU使用率、内存使用率、带宽、时延等这些的选择不同，QoS计算公式也会不同。
需要说明的是，在本申请的一些实施方式中，执行设备可以是通过内嵌在执行设备中的代理(也可称为Agent，一种软/硬件实体)来获取调度装置发送的与执行设备对应的QoS计算公式和/或与传输链路对应的QoS计算公式并将根据上述QoS计算公式计算得到的执行设备的QoS和/或传输链路的QoS发送回调度装置。以图5为例进行说明：若调度装置根据业务E所需资源值确定了P个执行路径，其中P≥1，这P个执行路径中包括有n个执行设备，这n个执行设备中的每一个执行设备都嵌入有Agent，可分别记为Agent-1、Agent-2、……、Agent-n，调度装置确定了P个执行路径的n个执行设备后就会将与这些执行设备对应的QoS计算公式(包括每个执行设备对应的QoS计算公式和/或每个传输链路对应的QoS计算公式)发送至对应的执行设备的Agent，即图5中的步骤p1-p3：p1、将与执行设备1对应的QoS计算公式1(包括执行设备1的QoS计算公式1和/或执行设备1对应的传输链路1的QoS计算公式1)发送至执行设备1上的Agent-1；p2、将与执行设备2对应的QoS计算公式2(包括执行设备2的QoS计算公式2和/或执行设备2对应的传输链路2的QoS计算公式2)发送至执行设备2上的Agent-2；……；p3、将与执行设备n对应的QoS计算公式n(包括执行设备n的QoS计算公式n和/或执行设备n对应的传输链路n的QoS计算公式n)发送至执行设备n上的Agent-n。各执行设备的Agent获取到对应的QoS计算公式之后，将根据QoS计算公式得到对应执行设备的QoS(若QoS计算公式仅包括执行设备对应的QoS计算公式)以及对应该执行设备的传输链路的QoS(若QoS计算公式包括执行设备对应的QoS计算公式和传输链路对应的QoS计算公式)，即图5中的步骤p4-p6(以QoS计算公式包括执行设备对应的QoS计算公式和传输链路对应的QoS计算公式为例进行示意)：p4、Agent-1计算执行设备1的QoS以及对应执行设备1的传输链路1的QoS；p5、Agent-2计算执行设备2的QoS以及对应执行设备2的传输链路2的QoS；……；p6、Agent-n计算执行设备n的QoS以及对应执行设备n的传输链路3的QoS。最后，各设备将计算得到的执行设备的QoS以及对应该执行设备传输链路的QoS发送至调度装置，即图5中的步骤p7-p9：p7、Agent-1将执行设备1的QoS、传输链路1的QoS发送至调度装置；p8、Agent-2将执行设备2的QoS、传输链路2的QoS发送至调度装置；……；p9、Agent-n将执行设备n的QoS、传输链路n的QoS发送至调度装置。最后，调度装置就可以根据这些执行设备的QoS和传输链路的QoS计算得到业务E在这P个执行路径的QoS。需要说明的是，在图5中，步骤p1-p3之间、步骤p4-p6之间、步骤p7-p9之间不存在先后顺序，只需保证步骤p7在步骤p4之后执行、步骤p4在步骤p1之后执行、步骤p8在步骤p5之后执行、步骤p5在步骤p2之后执行、步骤p9在步骤p6之后执行、步骤p6在步骤p3之后执行即可，具体此处不做限定。
在本申请实施例中，由于在云服务器下的所有设备(包括虚拟机、物理机、容器等)在执行业务时的最小功能单元都可以称为执行设备，调度装置通过嵌入执行设备中的Agent来实现QoS计算公式的收发、QoS的计算以及发送，便于统一管理，若云服务器下需要增加新的执行设备，也只需要在该新加入的执行设备中内嵌一个Agent就可实现统 一管理，简单方便。
为便于理解，下面从用户开始设置待部署业务的QoS开始，详细介绍部署业务的方法的一个具体的实施例，具体如图6所示：
601、登录并输入待部署业务。
用户登录Portal(Portal是指一个门户网站，是一个基于web的应用程序，主要提供个性化、单点登录、不同来源的内容整合以及存放信息系统的表示层)，并输入用户想要进行的待部署业务。
602、调度装置获取待部署业务的相关特性。
用户通过Portal输入待部署业务后，调度装置就可以获取到该待部署业务的类型、性质、优先级等相关特性。
603、调度装置向用户智能推荐待部署业务的常规QoS。
调度装置根据待部署业务的相关特性向用户智能推荐该待部署业务的常规QoS，即大部分用户在进行该待部署业务时所普遍采用的一个QoS。
604、用户设置待部署业务的预设QoS。
用户参考调度装置推荐的常规QoS对待部署业务的QoS进行设置，例如，用户可以不做修改而直接就将该常规QoS作为预设QoS，也可以在该常规QoS的基础上根据自身喜好稍作调整后作为预设QoS，具体此处不做限定。
在本申请的一些实施方式中，若用户不需要参考常规QoS，那么步骤603也可省略，用户可以完全按照自己的需求直接对该待部署业务的QoS进行设置得到预设QoS。
605、调度装置获取用户设置的预设QoS。
用户对待部署业务的QoS设置完成之后，调度装置就可以获取到针对该待部署业务的预设QoS。
606、调度装置确定待部署业务所需资源的规格。
调度装置可以根据获取到的待部署业务的相关特性进一步确定该待部署业务所述资源的规格。需要说的是，步骤606除了可以在步骤602至步骤605之后进行之外，步骤606也可以是在步骤602至步骤605中任何两个步骤之间进行，具体此处不做限定。
507、调度装置确定待部署业务的Q个执行路径、Q个执行路径的m个执行设备分别对应的m个QoS计算公式。
调度装置确定了待部署业务所需资源值以后，将根据云服务器的整体系统状况自动生成满足该待部署业务所需资源的规格的Q个执行路径(Q≥1)，并进一步确定这Q个执行路径总共包含的m个执行设备以及这m个执行设备分别对应的m个QoS计算公式，这m个QoS计算公式包括m个执行设备分别对应的m个QoS计算公式和m个传输链路对应的QOS计算公式，其中执行设备对应的QoS计算公式用于计算对应执行设备的QoS，传输链路对应的QoS计算公式用于计算执行设备对应传输链路的QoS。在本申请实施例中，对执行设备对应的QoS计算公式、传输链路对应的QoS计算公式的定义如上述图3对应的实施例所述，此处不予赘述。
608、调度装置确定Q个执行路径中每个执行路径的执行设备所占权重。
调度装置确定了Q个执行路径中的每个执行路径的执行设备之后，将进一步确定每个执行设备所占权重，执行设备所占权重表示该执行设备在对应的执行路径的重要程度。
609、调度装置给m个执行设备中嵌入的Agent分别发送对应的m个QoS计算公式。
在本申请实施例中，云服务器下的每个执行设备中都分别嵌有一个Agent，执行设备通过嵌入的Agent来实现QoS计算公式的收取、QoS的计算以及发送。当调度装置确定了Q个执行路径的m个执行设备后，那么调度装置就会将对应这m个执行设备的QoS计算公式发送至这m个执行设备的Agent。
610、m个执行设备中的各个Agent根据对应的QoS计算公式计算对应执行设备的QoS、对应该执行设备的传输链路的QoS。
m个执行设备的Agent分别获取到对应的QoS计算公式之后，将根据该QoS计算公式计算对应执行设备的QoS以及对应该执行设备的传输链路的QoS。在本申请实施例中，执行设备的Agent具体如何根据QoS计算公式得到对应执行设备的QoS以及对应该执行设备传输链路的QoS与上述图3对应实施例计算执行设备的QoS与传输链路的QoS类似，此处不予赘述。
611、m个执行设备中的各个Agent分别向调度装置发送对应执行设备的QoS和对应传输链路的QoS。
m个执行设备中的各个Agent将根据QoS计算公式得到的对应执行设备的QoS和对应传输链路的QoS发送至调度装置。
612、调度装置计算Q个执行路径的QoS并对各执行路径下一周期内的QoS进行预测。
调度装置根据获取到的m个执行设备的QoS和对应传输链路的QoS计算得到这Q个执行路径的QoS，并对各执行路径下一周期内的QoS进行预测。在本申请实施例中，调度装置如何计算得到Q个执行路径的QoS与图3对应的实施例中计算执行路径的QoS类似，此处不予赘述。
在本申请实施例中，若这Q个执行路径中存在至少一个执行路径的QoS与预设QoS的关系符合预设规则，则可以根据上述实施例中阐述的如何确定待部署业务最终的执行路径，此处不予赘述。在本申请实施例中，由于调度装置确定一个执行路径作为待部署业务待部署的执行路径是根据该执行路径当前周期内的QoS，然而执行路径的执行设备上不仅只会运行该待部署业务，还可能会运行其他业务，这就使得执行设备所包括的资源可能会发生变化。如果调度装置在待部署业务执行过程中发现已选的执行路径不能满足该待部署业务的资源需求，则需要对该执行路径的资源做非基于预测的后端调整，例如，业务F的数据走到统一负载均衡这个执行路径上时，调度装置发现当前这个执行路径后端的虚机资源不足，从而无法满足该业务F计算时延的要求，那么统一负载均衡会自动将该业务F转发到物理机资源上进行计算。调度装置还可以进一步对各执行路径下一周期内的QoS进行预测，例如，如果调度装置在待部署业务执行过程中发现部署有该待部署业务的执行路径下一周期内的QoS已不能满足预设规则，则可以根据预测的各执行路径的下一周期内的QoS将该待部署业务的执行路径调整到下一周期内的QoS满足预设规则的执行路径上来，以保证待部署业务的连续性和稳定性。
613、若Q个执行路径的QoS与预设QoS的关系均不满足预设规则，则执行资源调整。
如果调度装置确定计算得到的Q个执行路径的QoS与预设QoS的关系均不符合预设规则，则调度装置执行资源调整，即调整任意执行路径上的执行设备的资源的规格直至获得满足预设QoS条件的执行路径(可称为调整后的执行路径)，并将待部署业务部署在 该调整后的执行路径上来。调度装置执行资源调整的方式与上述实施例资源调整的方式类似，此处不予赘述。
对于用户设置的针对待部署业务的预设QoS，根据计算资源的不同，执行路径也会不同，以时延为QoS，取计算资源和网络资源为影响，示例一种执行路径的求解模型：
δ＝Φ(c)θ(r)*γ(c,r)
其中，δ为任务的完成度(单位为％)，即确保待部署业务完成的概率，Φ为计算资源的概率密度，θ为网络资源概率密度，γ为联合影响函数，定义为γ(c,r)＝(t(c,r))/t -，其中，t -为任务的平均执行时间(单位为微秒)，t(c,r)为计算资源和网络资源为(c,r)时对应的执行时间。
对于单次任务请求，定义c＝[c_1,c_2,…,c_n]，r＝[r_1,r_2,…,r_n]，完成度〖δ＝[δ_1,δ_2,…,δ_n]〗^T，任务(比如一个http request)的总执行时间t＝[t_(1)δ^(-1),t_2δ^(-1),…,t_iδ^(-1),…,t_nδ^(-1)]^T，其中t_(i)代表单次任务的平均执行时间。
Figure PCTCN2020094442-appb-000002
θ＝[θ_(r^1,)θ_(r^2,)…,θ_(r^n)]τ。其中，求解矩阵参数c、r则通过如下方式得到：确定目标函数为H＝MIN(〖||t||〗_1,〖||c||〗_1,〖||r||〗_1),取出满足条件的所有可能同时得到的最佳执行路径，限制条件为
Figure PCTCN2020094442-appb-000003
其中，定义：ε_i＝t_i/(t_i 0)，ε_i为最小容忍值，根据单次任务最大等待时间t_(i0)确定，一般情况下t_(i0)≥t_i，ρ为CPU资源闲置率(单位为％)，R为网络资源闲置率(单位为％)。
本申请实施例可以根据上述部署业务的方法的示例对调度装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
例如，图7示出了一种调度装置的示意图，本申请实施例提供的调度装置可以包括：
路径选择模块701，用于：
确定待部署业务的执行路径，该执行路径包括至少两个执行设备，并获取每个执行设备的服务质量QoS；根据该每个执行设备的QoS计算该执行路径的QoS；获取该待部署业务的预设QoS，最后，确定该执行路径的QoS与该预设QoS的关系符合预设规则；
业务部署模块702，用于：
若路径选择模块确定了执行路径的QoS与预设QoS的关系符合预设规则之后，在该执行路径上部署该待部署业务，该至少两个执行设备用于分别执行该待部署业务。
优选的，在本申请的一些实施方式中，该路径选择模块701具体用于：
在该根据每个执行设备的QoS计算该执行路径的QoS之前，获取该至少两个执行设备之间的至少一个传输链路的QoS，并根据该每个执行设备的QoS和该至少一个传输链路的QoS计算该执行路径的QoS。
优选的，在本申请的一些实施方式中，该路径选择模块701具体还用于：
确定每个执行设备对应的至少两个第一QoS计算公式，一个执行设备对应一个QoS计算公式，并将每个第一QoS计算公式发送至对应的执行设备，最后，获取该每个执行 设备根据对应的第一QoS计算公式得出的该每个执行设备的QoS。
优选的，在本申请的一些实施方式中，该路径选择模块701具体用于：
确定该至少一个传输链路对应的至少一个第二QoS计算公式，并将每个第二QoS计算公式发送至对应的传输链路上的执行设备，最后，获取该至少一个传输链路上的执行设备根据对应的第二QoS计算公式得出的每个传输链路的QoS。
优选的，在本申请的一些实施方式中，该路径选择模块701具体用于：
确定该每个执行设备的QoS中每个QoS所占权重，并根据该每个QoS以及对应的权重确定该执行路径的QoS。
优选的，在本申请实施例中，该路径选择模块701具体用于：
根据该待部署业务所需资源确定N个执行路径，其中，每个执行路径上的执行设备包括的资源的规格不低于该待部署业务所需资源的规格；若该N个执行路径的每一个执行路径的QoS与该预设QoS的关系均不符合该预设规则，则执行资源调整，即调整任意执行路径上的执行设备的资源规格直至获得满足预设QoS条件的执行路径。
优选的，在本申请实施例中，在路径选择模块确定执行路径的QoS与预设QoS的关系符合预设规则之后，该路径选择模块701具体还用于：
对该执行路径下一周期内的QoS进行预测，得到该执行路径的预测QoS，该执行路径的QoS为该执行路径当前周期内的QoS；此外，还将确定执行路径的预测QoS与预设QoS的关系也符合预设规则；
该业务部署模块702具体用于：
在路径选择模块确定执行路径的预测QoS与预设QoS的关系符合预设规则之后，在该执行路径上部署该待部署业务。
图7对应的实施例中的调度装置具体的功能以及结构用于实现前述图3至图6中由调度装置进行处理的步骤，具体此处不予赘述。
图7从模块化功能实体的角度对本申请实施例中的调度装置进行了描述，下面从硬件处理的角度对本申请实施例中的调度装置进行描述，如图8所示，为本申请实施例中调度装置的一个实施例示意图，具体包括：
该调度装置可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器(Central Processing Units，CPU)822(例如一个或一个以上处理器)和一个或一个以上存储器830，一个或一个以上计算机指令842或数据844存储在存储器830上。其中，存储器830可以是短暂存储或持久存储。存储器830可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对调度装置中的一系列计算机指令的操作。更进一步地，中央处理器822可以设置为与存储器830通信，在调度装置上执行存储器830中的一系列计算机指令的操作。
该调度装置还可以包括一个或一个以上电源826，一个或一个以上有线或无线网络接口850，一个或一个以上输入输出接口858，和/或，一个或一个以上操作系统841。
上述图3至图6所描述的部署业务的方法中的步骤由调度装置基于该图8所示的结构实现。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如DVD)、或者半导体介质(例如固态硬盘)等。

Claims (16)

  1. 一种部署业务的方法，其特征在于，所述方法包括：
    确定待部署业务的执行路径，所述执行路径包括至少两个执行设备；
    获取每个执行设备的服务质量QoS；
    根据所述每个执行设备的QoS计算所述执行路径的QoS；
    获取所述待部署业务的预设QoS；
    确定所述执行路径的QoS与所述预设QoS的关系符合预设规则，在所述执行路径上部署所述待部署业务，所述至少两个执行设备用于分别执行所述待部署业务。
  2. 根据权利要求1所述的方法，其特征在于，在所述根据所述每个执行设备的QoS计算所述执行路径的QoS之前，所述方法还包括：
    获取所述至少两个执行设备之间的至少一个传输链路的QoS；
    所述根据所述每个执行设备的QoS计算所述执行路径的QoS包括：
    根据所述每个执行设备的QoS和所述至少一个传输链路的QoS计算所述执行路径的QoS。
  3. 根据权利要求1-2中任一项所述的方法，其特征在于，所述获取每个执行设备的服务质量QoS包括：
    确定每个执行设备对应的至少两个第一QoS计算公式，一个执行设备对应一个QoS计算公式；
    将每个第一QoS计算公式发送至对应的执行设备；
    获取所述每个执行设备根据对应的第一QoS计算公式得出的所述每个执行设备的QoS。
  4. 根据权利要求2-3中任一项所述的方法，其特征在于，所述获取所述至少两个执行设备之间的至少一个传输链路的QoS，包括：
    确定所述至少一个传输链路对应的至少一个第二QoS计算公式；
    将每个第二QoS计算公式发送至对应的传输链路上的执行设备；
    获取所述至少一个传输链路上的执行设备根据对应的第二QoS计算公式得出的每个传输链路的QoS。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，所述根据所述每个执行设备的QoS计算所述执行路径的QoS包括：
    确定所述每个执行设备的QoS中每个QoS所占权重；
    根据所述每个QoS以及对应的权重确定所述执行路径的QoS。
  6. 根据权利1-5中任一项所述的方法，其特征在于，所述确定待部署业务的执行路径包括：
    根据所述待部署业务所需资源确定N个执行路径，其中，每个执行路径上的执行设备包括的资源的规格不低于所述待部署业务所需资源的规格；
    所述方法还包括：
    确定所述N个执行路径的每一个执行路径的QoS与所述预设QoS的关系均不符合所述预设规则，调整任意执行路径上的执行设备的资源规格直至获得满足预设QoS条件的执行路径。
  7. 根据权利要求1-5中任一项所述的方法，其特征在于，所述确定所述执行路径的QoS与所述预设QoS的关系符合预设规则，在所述执行路径上部署所述待部署业务包括：
    确定所述执行路径的QoS与所述预设QoS的关系符合所述预设规则，对所述执行路径下一周期内的QoS进行预测，得到所述执行路径的预测QoS，所述执行路径的QoS为所述执行路径当前周期内的QoS；
    确定所述执行路径的预测QoS与所述预设QoS的关系符合所述预设规则，在所述执行路径上部署所述待部署业务。
  8. 一种调度装置，其特征在于，包括：路径选择模块以及业务部署模块；
    所述路径选择模块，用于：
    确定待部署业务的执行路径，所述执行路径包括至少两个执行设备；
    获取每个执行设备的服务质量QoS；
    根据所述每个执行设备的QoS计算所述执行路径的QoS；
    获取所述待部署业务的预设QoS；
    确定所述执行路径的QoS与所述预设QoS的关系符合预设规则；
    所述业务部署模块，用于：
    在所述路径选择模块确定所述执行路径的QoS与所述预设QoS的关系符合预设规则之后，在所述执行路径上部署所述待部署业务，所述至少两个执行设备用于分别执行所述待部署业务。
  9. 根据权利要求8所述的调度装置，其特征在于，所述路径选择模块具体用于：
    在所述根据每个执行设备的QoS计算所述执行路径的QoS之前，获取所述至少两个执行设备之间的至少一个传输链路的QoS；
    根据所述每个执行设备的QoS和所述至少一个传输链路的QoS计算所述执行路径的QoS。
  10. 根据权利要求8-9中任一项所述的调度装置，其特征在于，所述路径选择模块具体还用于：
    确定每个执行设备对应的至少两个第一QoS计算公式，一个执行设备对应一个QoS计算公式；
    将每个第一QoS计算公式发送至对应的执行设备；
    获取所述每个执行设备根据对应的第一QoS计算公式得出的所述每个执行设备的QoS。
  11. 根据权利要求9-10中任一项所述的调度装置，其特征在于，所述路径选择模块具体用于：
    确定所述至少一个传输链路对应的至少一个第二QoS计算公式；
    将每个第二QoS计算公式发送至对应的传输链路上的执行设备；
    获取所述至少一个传输链路上的执行设备根据对应的第二QoS计算公式得出的每个传输链路的QoS。
  12. 根据权利要求8-11中任一项所述的调度装置，其特征在于，所述路径选择模块具体用于：
    确定所述每个执行设备的QoS中每个QoS所占权重；
    根据所述每个QoS以及对应的权重确定所述执行路径的QoS。
  13. 根据权利要求8-12中任一项所述的调度装置，其特征在于，所述路径选择模块具体用于：
    根据所述待部署业务所需资源确定N个执行路径，其中，每个执行路径上的执行设备包括的资源的规格不低于所述待部署业务所需资源的规格；
    确定所述N个执行路径的每一个执行路径的QoS与所述预设QoS的关系均不符合所述预设规则，调整任意执行路径上的执行设备的资源规格直至获得满足预设QoS条件的执行路径。
  14. 根据权利要求8-12中任一项所述的调度装置，其特征在于，在所述路径选择模块确定所述执行路径的QoS与所述预设QoS的关系符合所述预设规则之后，所述路径选择模块具体还用于：
    对所述执行路径下一周期内的QoS进行预测，得到所述执行路径的预测QoS，所述执行路径的QoS为所述执行路径当前周期内的QoS；
    确定所述执行路径的预测QoS与所述预设QoS的关系符合所述预设规则；
    所述业务部署模块具体用于：
    在所述路径选择模块确定所述执行路径的预测QoS与所述预设QoS的关系符合所述预设规则之后，在所述执行路径上部署所述待部署业务。
  15. 一种计算设备，其特征在于，所述计算设备包括存储器和处理器，所述处理器执行所述存储器存储的计算机指令，使得所述计算设备执行权利要求1-7中任一项所述的方法。
  16. 一种计算机可读存储介质，包括指令，当所述指令在计算机上运行时，使得所述计算机执行如权利要求1-7中任一项所述的方法。
PCT/CN2020/094442 2019-06-29 2020-06-04 一种部署业务的方法以及调度装置 WO2021000694A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20834405.1A EP3968606B1 (en) 2019-06-29 2020-06-04 Method for deploying services and scheduling apparatus
US17/563,555 US11838389B2 (en) 2019-06-29 2021-12-28 Service deployment method and scheduling apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910581292.5 2019-06-29
CN201910581292.5A CN110430236B (zh) 2019-06-29 2019-06-29 一种部署业务的方法以及调度装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/563,555 Continuation US11838389B2 (en) 2019-06-29 2021-12-28 Service deployment method and scheduling apparatus

Publications (1)

Publication Number Publication Date
WO2021000694A1 true WO2021000694A1 (zh) 2021-01-07

Family

ID=68408881

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/094442 WO2021000694A1 (zh) 2019-06-29 2020-06-04 一种部署业务的方法以及调度装置

Country Status (4)

Country Link
US (1) US11838389B2 (zh)
EP (1) EP3968606B1 (zh)
CN (1) CN110430236B (zh)
WO (1) WO2021000694A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114172853A (zh) * 2021-12-01 2022-03-11 阿里巴巴（中国）有限公司 流量转发及裸机服务器的配置方法、装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110430236B (zh) 2019-06-29 2020-11-03 华为技术有限公司 一种部署业务的方法以及调度装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080181117A1 (en) * 2006-08-25 2008-07-31 Alcatel Lucent METHOD OF PROVIDING QoS FOR A FLOW
CN102404137A (zh) * 2011-08-09 2012-04-04 江苏欣网视讯科技有限公司 一种IP网络下基于业务流QoS监测的故障定位方法
CN104539535A (zh) * 2015-01-28 2015-04-22 浪潮电子信息产业股份有限公司 一种数据传输路径确定方法及装置
WO2015168888A1 (zh) * 2014-05-07 2015-11-12 华为技术有限公司 网络拥塞控制方法和控制器
CN110430236A (zh) * 2019-06-29 2019-11-08 华为技术有限公司 一种部署业务的方法以及调度装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8982709B2 (en) * 2007-01-31 2015-03-17 Hewlett-Packard Development Company, L.P. Selecting service nodes for an end-to-end service path from a reduced search space
KR101054055B1 (ko) * 2008-05-16 2011-08-03 한국과학기술원 이동 노드를 위한 ｎｓｉｓ 프로토콜 기반 사전자원예약장치 및 방법
CN102204209B (zh) * 2011-05-06 2014-01-22 华为技术有限公司 跨媒介网络中的QoS协商方法和系统及网络设备
CN104104602B (zh) * 2014-08-06 2018-02-09 湖南网数科技有限公司 一种QoS信息预测的方法及装置
CN104518993A (zh) * 2014-12-29 2015-04-15 华为技术有限公司 云化网络通信路径的分配方法、装置及系统
CN108243044B (zh) * 2016-12-27 2021-08-24 山东财兜信息科技有限公司 业务部署的方法与装置
CN108494601B (zh) * 2018-04-04 2020-02-21 西安电子科技大学 层次化确定网络中的多约束双路径路由方法
CN109729557A (zh) * 2018-12-28 2019-05-07 上海昭宁信息科技有限公司 一种无线组网的方法和终端
US10979316B2 (en) * 2019-05-31 2021-04-13 Juniper Networks, Inc. Dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080181117A1 (en) * 2006-08-25 2008-07-31 Alcatel Lucent METHOD OF PROVIDING QoS FOR A FLOW
CN102404137A (zh) * 2011-08-09 2012-04-04 江苏欣网视讯科技有限公司 一种IP网络下基于业务流QoS监测的故障定位方法
WO2015168888A1 (zh) * 2014-05-07 2015-11-12 华为技术有限公司 网络拥塞控制方法和控制器
CN104539535A (zh) * 2015-01-28 2015-04-22 浪潮电子信息产业股份有限公司 一种数据传输路径确定方法及装置
CN110430236A (zh) * 2019-06-29 2019-11-08 华为技术有限公司 一种部署业务的方法以及调度装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3968606A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114172853A (zh) * 2021-12-01 2022-03-11 阿里巴巴（中国）有限公司 流量转发及裸机服务器的配置方法、装置
CN114172853B (zh) * 2021-12-01 2024-04-16 阿里巴巴（中国）有限公司 流量转发及裸机服务器的配置方法、装置

Also Published As

Publication number Publication date
EP3968606C0 (en) 2024-03-20
US11838389B2 (en) 2023-12-05
EP3968606B1 (en) 2024-03-20
CN110430236B (zh) 2020-11-03
CN110430236A (zh) 2019-11-08
EP3968606A4 (en) 2022-06-22
EP3968606A1 (en) 2022-03-16
US20220124172A1 (en) 2022-04-21

Similar Documents

Publication Publication Date Title
US10341208B2 (en) File block placement in a distributed network
US11275609B2 (en) Job distribution within a grid environment
US10657106B2 (en) Method, computing device, and distributed file system for placement of file blocks within a distributed file system
CN112153700B (zh) 一种网络切片资源管理方法及设备
CN108243044B (zh) 业务部署的方法与装置
Ge et al. GA-based task scheduler for the cloud computing systems
JP6501918B2 (ja) 計算ワークフローにおいてサービス品質を確保するためのシステム及び方法
CN106233276B (zh) 网络可访问块存储装置的协调准入控制
US9417903B2 (en) Storage management for a cluster of integrated computing systems comprising integrated resource infrastructure using storage resource agents and synchronized inter-system storage priority map
US9141432B2 (en) Dynamic pending job queue length for job distribution within a grid environment
CN101167054B (zh) 用于在多个数据中心之间选择性卸载工作负荷的方法和装置
CN1610347B (zh) 在基于群集的系统内管理性能及资源利用率的方法和设备
WO2017167025A1 (zh) 一种实现任务调度的方法、装置及计算机存储介质
US20170149681A1 (en) Assessment and dynamic provisioning of computing resources for multi-tiered application
US11102143B2 (en) System and method for optimizing resource utilization in a clustered or cloud environment
KR20160132852A (ko) 광역 네트워크를 통한 데이터 전송을 위한 장기 스케줄의 계산 기법
CN110262897B (zh) 一种基于负载预测的Hadoop计算任务初始分配方法
WO2021000694A1 (zh) 一种部署业务的方法以及调度装置
Li et al. An efficient scheduling optimization strategy for improving consistency maintenance in edge cloud environment
CN114625500A (zh) 云环境下拓扑感知的微服务应用调度的方法及应用
Ben Jemaa et al. Analytical models for QoS-driven VNF placement and provisioning in wireless carrier cloud
Jung et al. Ostro: Scalable placement optimization of complex application topologies in large-scale data centers
CN114978913B (zh) 一种基于切链的服务功能链跨域部署方法及系统
US20080235705A1 (en) Methods and Apparatus for Global Systems Management
KR102563329B1 (ko) 컨테이너에 대한 리소스 간 의존성 스케줄링 방법 및 이를 수행하는 네트워크 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20834405

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020834405

Country of ref document: EP

Effective date: 20211207

NENP Non-entry into the national phase

Ref country code: DE