CN113760516A - Elastic expansion method, device, equipment and medium in multi-cloud environment - Google Patents
Elastic expansion method, device, equipment and medium in multi-cloud environment Download PDFInfo
- Publication number
- CN113760516A CN113760516A CN202010495551.5A CN202010495551A CN113760516A CN 113760516 A CN113760516 A CN 113760516A CN 202010495551 A CN202010495551 A CN 202010495551A CN 113760516 A CN113760516 A CN 113760516A
- Authority
- CN
- China
- Prior art keywords
- cloud
- cloud platforms
- application
- resource
- platforms
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The application provides an elastic expansion method under a cloudy environment, which comprises the following steps: modeling resources provided by a plurality of cloud platforms to obtain resource models of the cloud platforms, and adjusting application instances according to the resource models of the cloud platforms and an elastic expansion strategy when monitoring index values of the application instances deployed on the cloud platforms meet preset conditions. Therefore, under the environment of a plurality of cloud platforms, automatic and application examples stretch out and draw back across the cloud elasticity without manual intervention, the elastic stretching efficiency is improved, and resource waste or service quality difficulty caused by manual intervention is avoided. The whole expansion process does not need manual participation, the elastic expansion of the application example across the cloud platform in the minute level or even the second level can be realized, the resource utilization rate is improved, and the service quality is guaranteed.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to an elastic scaling method, apparatus, device, and computer-readable storage medium in a cloud environment.
Background
Cloud technology (cloud technology) refers to a hosting technology for unifying series of resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The service provider providing the service through the cloud technology according to the user requirement is the cloud service provider.
An application can be deployed on a cloud platform provided by a cloud service provider to provide services to the outside. In order to guarantee the service quality and save resources, an elastic scaling method is proposed in the industry. Specifically, specific indexes of the application, such as a processor (CPU) utilization rate and a memory utilization rate, are monitored, and when an index value of the specific index reaches a preset threshold, the number of application instances is adjusted according to a set elastic scaling strategy.
However, the elastic scaling method is mainly suitable for applications deployed on a single cloud platform. For applications deployed in a multi-cloud platform, i.e., a multi-cloud environment, manual intervention is often required, and rapid elastic expansion and contraction are difficult to realize, so that resource waste or service quality is difficult to guarantee.
Disclosure of Invention
The application provides an elastic expansion method in a cloud environment, and solves the problems that manual intervention is needed in the related technology, rapid elastic expansion is difficult to realize, and resource waste or service quality is difficult to guarantee. The application also provides a device, equipment, a computer readable storage medium and a computer program product corresponding to the method.
In a first aspect, the application provides an elastic expansion method in a cloudy environment. The method is used for realizing automatic cloud-crossing elastic expansion of the application examples in the environment of a plurality of cloud platforms, manual intervention is not needed, the elastic expansion efficiency is improved, and resource waste or service quality difficulty caused by manual intervention is avoided.
Specifically, each cloud platform of the plurality of cloud platforms is provided with at least one resource. The resource may be a hardware resource or a software resource that provides a service. Resources may also be divided into computing resources, storage resources, and network resources according to function. For example, the computing resources may include processor resources, such as Central Processing Unit (CPU) resources. The memory resources may include memory (memory) resources, external memory resources, and the like. The memory resource may be an internal memory, and the external memory resource may be a hard disk, an optical disk, a flash disk, and the like.
In specific implementation, the cross-cloud elastic expansion controller may first model resources provided by a plurality of cloud platforms to obtain resource models of the plurality of cloud platforms. The resource model can realize mapping conversion of the same type of resources among a plurality of cloud platforms, for example, mapping conversion of storage resources between a cloud platform a and a cloud platform B. Then, when the monitoring index value of the application instance deployed on the multiple cloud platforms meets a preset condition, for example, the monitoring index value reaches a set threshold, the cloud-crossing elastic scaling controller may adjust the application instance according to the resource models of the multiple cloud platforms and the elastic scaling policy. Because manual intervention is not needed, the elastic expansion of the application instance across the cloud platform in the minute level or even the second level can be realized, the resource utilization rate is improved, and the service quality is guaranteed.
In some possible implementations, the cross-cloud elastic scaling controller may configure one or more elastic scaling strategies. For example, the cross-cloud elastic scaling controller may configure one or more of an affinity (affinity) policy, an anti-affinity (anti-affinity) policy, a cost-first policy, and a performance-first policy. The affinity policy specifically refers to configuring the application instances in similar regions (regions) of the same cloud platform, where the similar regions include the same region, and even similar nodes (including the same node) in the same region. The anti-affinity policy specifically refers to avoiding configuration of application instances in a proximate region and proximate nodes in the same region. The cost priority policy is a policy for performing elastic scaling with the aim of minimizing cost, and the performance priority policy is a policy for performing elastic scaling with the aim of maximizing performance.
Based on this, the cross-cloud elastic scaling controller may adjust the application instance according to the resource models of the multiple cloud platforms and any one or more of an affinity policy, an anti-affinity policy, a cost priority policy, and a performance priority policy. Therefore, elastic expansion and contraction of the application examples can be achieved according to user requirements.
In some possible implementation manners, when the cross-cloud elastic scaling controller elastically scales the application instances, the number of the application instances may be adjusted according to the resource models and the elastic scaling strategies of the multiple cloud platforms. Such as adding an application instance or deleting an application instance. Of course, the cross-cloud elastic scaling controller may also adjust the configuration of the application instance according to the resource models of the multiple cloud platforms and the elastic scaling policy. For example, computing resources, storage resources, and/or network resources of an application instance are promoted or reduced.
In some possible implementations, the cross-cloud elastic scaling controller may debug the application instance as follows. Specifically, the cross-cloud elastic scaling controller determines a target cloud platform according to the resource models of the plurality of cloud platforms and the elastic scaling strategy, and then creates a new application instance through an Application Programming Interface (API) of the target cloud platform. The new instance is deployed on the target cloud platform.
In some possible implementation manners, after creating a new application instance, when the monitoring index value of the application instances deployed on the plurality of cloud platforms does not meet a preset condition, the cross-cloud elastic scaling controller deletes the new application instance through the API.
In some possible implementations, the plurality of cloud platforms includes a plurality of public cloud platforms, or a plurality of private cloud platforms, or a hybrid cloud platform formed by at least one public cloud platform and at least one private cloud platform. Multiple public cloud platforms can be selected to deploy application instances in consideration of cost. Multiple private cloud platforms may be selected to deploy the application instance in view of security issues. By comprehensively considering the cost problem and the safety problem, a mixed cloud platform formed by at least one public cloud platform and at least one private cloud platform can be selected to deploy the application instance.
In a second aspect, the present application provides an elastic expansion device in a cloudy environment. The device comprises:
the modeling unit is used for modeling resources provided by a plurality of cloud platforms to obtain resource models of the cloud platforms;
and the adjusting unit is used for adjusting the application examples according to the resource models and the elastic scaling strategies of the cloud platforms when the monitoring index values of the application examples deployed on the cloud platforms meet preset conditions.
In some possible implementations, the adjusting unit is specifically configured to:
and adjusting the application examples according to the resource models of the cloud platforms and any one or more of an affinity strategy, an anti-affinity strategy, a cost priority strategy and a performance priority strategy.
In some possible implementations, the adjusting unit is specifically configured to:
adjusting the number of the application instances according to the resource models of the cloud platforms and an elastic expansion strategy; alternatively, the first and second electrodes may be,
and adjusting the configuration of the application instance according to the resource models of the cloud platforms and the elastic scaling strategy.
In some possible implementations, the adjusting unit is specifically configured to:
determining a target cloud platform according to the resource models of the cloud platforms and an elastic expansion strategy;
creating a new application instance through an Application Programming Interface (API) of the target cloud platform.
In some possible implementations, the adjusting unit is further configured to:
after a new application instance is created, when the monitoring index values of the application instances deployed on the plurality of cloud platforms do not meet preset conditions, deleting the new application instance through the API.
In some possible implementations, the plurality of cloud platforms includes a plurality of public cloud platforms, or a plurality of private cloud platforms, or a hybrid cloud platform formed by at least one public cloud platform and at least one private cloud platform.
In a third aspect, the present application provides an apparatus comprising a processor and a memory. The processor and the memory are in communication with each other. The processor is configured to execute the instructions stored in the memory to cause the apparatus to perform the elastic scaling method in a cloudy environment according to the first aspect or any one of the implementations of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and the instructions instruct an apparatus to perform the elastic scaling method in the cloudy environment according to the first aspect or any implementation manner of the first aspect.
In a fifth aspect, the present application provides a computer program product comprising instructions that, when run on a device, cause the device to perform the method for elastic scaling in a cloudy environment according to the first aspect or any one of the implementations of the first aspect.
The present application can further combine to provide more implementations on the basis of the implementations provided by the above aspects.
Drawings
In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings used in the embodiments will be briefly described below.
Fig. 1 is a schematic view of an application scenario of an elastic scaling method in a cloud environment according to an embodiment of the present application;
fig. 2 is an architecture diagram of an elastic expansion and contraction method in a cloudy environment according to an embodiment of the present disclosure;
fig. 3 is an architecture diagram of an elastic expansion and contraction method in a cloudy environment according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an elastic expansion controller according to an embodiment of the present disclosure;
fig. 5 is a flowchart of an elastic scaling method in a cloudy environment according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of elastic expansion in a cloudy environment according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an apparatus according to an embodiment of the present application.
Detailed Description
The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.
Some technical terms referred to in the embodiments of the present application will be first described.
Cloud technology (cloud technology) refers to a hosting technology for unifying series of resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The service provider that provides services through the cloud technology according to the user requirements is the cloud service provider, and the services provided by the cloud service provider through the cloud technology are also called cloud services.
The cloud service comprises the following service modes: infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). In some implementations, the cloud services can also include a Function as a Service (FaaS) schema.
In the IaaS mode, a service provider provides hardware resources, a user deploys an operating system, middleware, a runtime library and the like, and then installs software.
In the PaaS mode, a service provider not only provides hardware resources, but also deploys an operating system, provides middleware, a runtime library and the like, and a user installs software by himself.
In the SaaS mode, a service provider provides hardware resources, deploys an operating system, and provides basic environments such as middleware and a runtime library. In addition, the service provider provides software that the user can use directly.
In FaaS mode, software (e.g., applications) is abstracted into functions. Function launching is performed only when the program is called. The program is not started when not called, and thus, the resource is not occupied.
A cloud platform, also called a cloud system, cloud environment, or cloud, is a software system for providing cloud services by a cloud provider. The software system may be a software system that provides cloud services in an IaaS mode, a PaaS mode, a SaaS mode, or a FaaS mode.
The public cloud is a cloud platform provided by a public cloud provider of a third party for vast individuals or enterprises. In a public cloud, hardware, software, and other structures are owned and managed by a third party's public cloud provider.
Private cloud is a dedicated cloud platform provided for an enterprise or organization. The private cloud may be operated internally by the corresponding enterprise or organization. Private clouds are mainly enterprise user oriented, also known as enterprise clouds.
The mixed cloud refers to a cloud platform formed by different cloud platforms. The hybrid cloud includes at least two cloud platforms, also referred to as multi-cloud platforms or clouds. Optionally, the hybrid cloud merges a public cloud and a private cloud. Some enterprise users prefer to have data stored in a private cloud for security reasons, but also desire to have access to the computing resources of a public cloud. In this case, a hybrid cloud including a public cloud and a private cloud is increasingly adopted. The public cloud and the private cloud are mixed and matched by the mixed cloud so as to obtain a good use effect.
An Application Programming Interface (API), also called program interface, is an "interface between a program and an operating system" provided by an operating system of a cloud platform to a user, such as a programmer. The interface may be used by a user (e.g., a developer) at the time of programming. Through the application program interface, a user can access resources in the cloud platform and obtain corresponding services. An application program interface is a collection of definitions, functions, procedures and/or protocols. For example, an application program interface of a cloud platform includes one or more system calls (system calls), each of which is a program that can perform a specific function.
An API Gateway (API Gateway, APIG) is specifically a Gateway that provides API hosting services. The API gateway may manage the API for unified authentication, metering, distribution, and/or flow control.
The resource may be a hardware resource or a software resource that provides a service. Resources may also be divided into computing resources, storage resources, and network resources according to function. For example, the computing resources may include processor resources, such as Central Processing Unit (CPU) resources. The memory resources may include memory (memory) resources, external memory resources, and the like. The memory resource may be an internal memory, and the external memory resource may be a hard disk, an optical disk, a flash disk, and the like.
Elastic scaling (auto scaling) is a service that automatically adjusts its business resources through policies according to the business needs of users. For example, when the traffic of the user is large, the service resource may be automatically increased through a policy, such as increasing the application instance corresponding to the service. The application instance refers to an instance (instance) created according to an application. An instance may be considered an application in a run state. An instance may be generated by launching (or opening) an application.
The cloud service provider may provide services to the user through the cloud platform. In order to guarantee the service quality and save resources, an elastic scaling method is proposed in the industry. Specifically, specific indexes of the application, such as a CPU utilization rate and a memory utilization rate, are monitored, and when an index value of the specific index reaches a preset threshold value, the number of application instances is adjusted according to a set elastic scaling strategy.
However, the elastic scaling method is mainly suitable for applications deployed on a single cloud platform. For applications deployed in a multi-cloud platform, i.e., a multi-cloud environment, manual intervention is often required, and rapid elastic expansion and contraction are difficult to realize, so that resource waste or service quality is difficult to guarantee.
In view of this, the present application provides an elastic expansion method in a cloudy environment. The method may be performed by a cross-cloud elastic scaling controller. Specifically, the cross-cloud elastic scaling controller may first model resources provided by a plurality of cloud platforms to obtain resource models of the plurality of cloud platforms. The resource model can realize mapping conversion of the same type of resources among a plurality of cloud platforms, for example, mapping conversion of storage resources between a cloud platform a and a cloud platform B. Then, when the monitoring index value of the application instance deployed on the multiple cloud platforms meets a preset condition, for example, the monitoring index value reaches a set threshold, the cloud-crossing elastic scaling controller may adjust the application instance according to the resource models of the multiple cloud platforms and the elastic scaling policy.
Therefore, under the environment of a plurality of cloud platforms, automatic and application examples stretch out and draw back across the cloud elasticity without manual intervention, the elastic stretching efficiency is improved, and resource waste or service quality difficulty caused by manual intervention is avoided. The whole expansion process does not need manual participation, the elastic expansion of the application example across the cloud platform in the minute level or even the second level can be realized, the resource utilization rate is improved, and the service quality is guaranteed.
In order to make the technical scheme of the present application clearer and easier to understand, an application scenario of the elastic scaling method in a cloudy environment provided by the embodiment of the present application is described below with reference to the accompanying drawings.
Referring to fig. 1, an application scenario of the elastic scaling method in a cloudy environment is shown, and as shown in fig. 1, the scenario includes an elastic scaling controller 102 and a plurality of cloud platforms 104. The plurality of cloud platforms 104 may be respectively denoted as cloud platform 1 through cloud platform n, where n is greater than 1.
At least one of the platforms 1 to n is deployed with an instance of the application a. Wherein at least one cloud platform can deploy one or more instances of application a. In one example, the cloud platform 1 is deployed with a plurality of instances of the application a, specifically, instances 11 to 1m, and the cloud platform n is deployed with a plurality of instances of the application a, specifically, instances n1 to nm. Wherein m is a positive integer.
The elastic expansion controller 102 is connected to the plurality of cloud platforms 104, for example, through a communication path. The elastic expansion controller 102 models the resources provided by the plurality of cloud platforms 104, and obtains resource models of the plurality of cloud platforms 104. When the monitoring index values of the application instances deployed on the multiple cloud platforms 104 satisfy a preset condition, the elastic scaling controller 102 adjusts the application instances according to the resource models of the multiple cloud platforms and the elastic scaling strategy. Therefore, automatic elastic expansion under a cloudy environment is realized.
As shown in fig. 2, the elastic expansion and contraction controller 102 may be deployed in a cloud environment, specifically, one or more computing devices (e.g., a central server) on the cloud environment. The controller 102 may also be deployed in an edge environment, and in particular, on one or more computing devices (edge computing devices) in the edge environment, which may be servers, computing boxes, and the like. The cloud environment indicates a central cluster of computing devices owned by a cloud service provider for providing computing, storage, and communication resources; the edge environment indicates a cluster of edge computing devices geographically close to an end device (i.e., a peer device) for providing computing, storage, and communication resources.
The aforementioned elastic expansion controller 102 may also be disposed on the end device. The end equipment comprises a physical machine (such as a terminal). The terminal includes, but is not limited to, a desktop computer, a notebook computer, a tablet computer, or a smart phone. The elastic expansion controller 102 may also be deployed in a virtual machine (virtual machine) or a container (container) on the above-described physical machine. In some implementations, the resilient scaling controller 102 may also be deployed in a cluster in multiple copies, allowing for load balancing and reliability.
Further, as shown in fig. 3, the elastic expansion controller 102 may include a plurality of portions (e.g., include a plurality of functional modules). Based on this, various portions of the resilient scaling controller 102 may also be deployed in a distributed manner in different environments. For example, a portion of the elastic telescoping controller 102 may be deployed on three environments, a cloud environment, a fringe environment, an end device, or any two of them, respectively.
The functional modules inside the elastic expansion controller 102 may be divided in various ways, and the present application does not limit the functional modules. Fig. 4 illustrates an exemplary division, and as shown in fig. 4, the elastic expansion and contraction controller 102 includes an index monitoring module 1022 and an elastic expansion and contraction control module 1024. In some implementations, the elastic scaling controller 102 may also include one or more of an elastic scaling policy management module 1026, a cloudy cost insight module 1028, and a cloudy performance evaluation module 1029.
The index monitoring module 1022 is configured to read a monitoring index value corresponding to an application-related monitoring index through an API opened by a monitoring system of each cloud platform. When the monitoring index value meets a preset condition, for example, the monitoring index value reaches a preset threshold, the elastic scaling control module 1024 may adjust the application instance according to the resource models of the plurality of cloud platforms 104 and the elastic scaling policy.
In some aspects, the elastic scaling policy management module 1026 is configured to manage elastic scaling policies for use by the elastic scaling control module 1024. Specifically, the elastic scaling policy management module 1026 is provided with a cross-cloud elastic scaling policy management component, and a user can configure an elastic scaling policy based on application of various performance or other user-defined indicators through the cross-cloud elastic scaling policy management component. The elastic scaling policy may include one or more of an affinity (affinity) policy, an anti-affinity (anti-affinity) policy, a performance-first policy, or a cost-first policy.
For the elastic scaling policy, the embodiments of the present application only describe an affinity policy, an anti-affinity policy, a cost priority policy, a performance priority policy, and the like. In some implementations, the elastic telescoping controller 102 may also standardize, plug-in the elastic telescoping policy management module 1026. If more similar strategies need to be supported subsequently, a third-party developer can develop and select the corresponding plug-in according to the requirement.
The affinity policy specifically refers to configuring the application instances in similar regions (including the same region) of the same cloud platform, and even similar nodes (including the same node) in the same region. The anti-affinity policy specifically refers to avoiding configuring application instances in a similar region or in a similar node in the same region. When the application has higher requirement on response efficiency, the application instance can be configured in a similar region of the same cloud platform. When the application has higher requirement on the service reliability, the application instances can be prevented from being configured in the similar regions of the same cloud platform.
The multi-cloud cost insights module 1028 is configured to evaluate the cost of resources on multiple cloud platforms. For a public cloud, the multi-cloud cost insights module 1028 may obtain quotes of various specifications of various resources by accessing a resource quotation system of the public cloud, thereby obtaining costs of various specifications of various resources on the public cloud. For a private cloud, the multi-cloud cost insights module 1028 may analyze the one-time fixed investment cost and the periodic maintenance cost of the private cloud according to the configuration information of the private cloud, such as the hardware configuration, the operating system configuration, and the like, and determine the cost of various specifications of various resources based on the cost, for example, the usage cost of the unit metering unit of the resource such as various computing, storage, network, and the like in unit time.
The multi-cloud performance evaluation module 1029 is configured to evaluate the performance of resources on each cloud platform. Specifically, the multi-cloud performance evaluation module 1029 may perform a performance test, such as a run-out test, on the same configuration resource of each specification of different cloud platforms to obtain performance values, such as a computation rate, a network rate, and a disk input/output (I/O) rate.
When the cost priority policy is adopted for elastic expansion, the elastic expansion control module 1024 may further obtain the costs of various specifications of various resources on the multiple cloud platforms 104 from the multi-cloud cost insights module 1028, and then adjust the application instance based on the resource models of the multiple cloud platforms 104, the costs of various specifications of various resources on the multiple cloud platforms 104, and the cost priority policy.
When the performance priority policy is adopted for elastic expansion, the elastic expansion control module 1024 may further obtain the performance of each specification of each type of resource on the multiple cloud platforms 104 from the multi-cloud performance evaluation module 1029, and then adjust the application instance based on the resource models of the multiple cloud platforms 104, the performance of each specification of each type of resource on the multiple cloud platforms 104, and the performance priority policy.
Next, from the perspective of the elastic expansion controller, an elastic expansion method in a cloudy environment according to an embodiment of the present application is described.
Referring to fig. 5, a flowchart of an elastic scaling method in a cloudy environment is shown, where the method includes:
s502: the elastic expansion controller 102 models resources provided by a plurality of cloud platforms to obtain resource models of the plurality of cloud platforms.
The resource model can realize mapping conversion of the same type of resources among a plurality of cloud platforms, for example, mapping conversion of storage resources between a cloud platform a and a cloud platform B. In some implementations, the resources may be characterized by resource parameters. The resource parameter may specifically be a parameter describing a resource configuration situation. For example, for a computing resource, the resource parameter may include the number of cores, such as 4 cores for a CPU. For another example, for a storage resource, the resource parameter may include a memory capacity, such as 2GB (gigabytes) of memory.
The format of the resource parameter (schema, also called specification) is called resource parameter format for short. For example, the resource parameter format corresponding to a certain resource may be: the resource parameters are the calculation resource specification parameters such as the number of CPUs or the number of GPUs. In particular implementations, the elastic scaling controller 102 may unify the resource parameter format to achieve a unified modeling of the resource.
In some implementations, the elastic scaling controller 102 may obtain a standard resource parameter format, and then establish a correspondence between a resource parameter format supported by at least one cloud platform (for example, each cloud platform) of the plurality of cloud platforms 104 and the standard resource parameter format, so as to obtain a resource model of the plurality of cloud platforms 104, where the resource model may also be referred to as an escape model. The standard resource parameter format in the resource model is used as a reference value, the resource parameter format is transferred, and the mapping conversion relation of the same type of resources among a plurality of cloud platforms can be obtained.
The standard resource parameter format may be a resource parameter format supported by one of the cloud platforms 104. Of course, the standard resource parameter format may also be obtained by processing a resource parameter format supported by at least one cloud platform of the multiple cloud platforms 104.
S504: when the monitoring index values of the application instances deployed on the multiple cloud platforms 104 satisfy a preset condition, the elastic scaling controller 102 adjusts the application instances according to the resource models of the multiple cloud platforms and the elastic scaling strategy.
The elastic expansion controller 102 may monitor the indexes of the application instances deployed on the plurality of cloud platforms 104 to obtain a monitoring index value. Specifically, the elastic expansion controller 102 may read the monitoring index value from the monitoring system on the cloud platform 104. The monitoring system may include a plurality of index values of the monitoring index, and the elastic expansion controller 102 obtains the corresponding monitoring index value according to the actual requirement. In some examples, the monitoring index value obtained by the elastic expansion controller 102 may be a CPU utilization rate, a memory utilization rate, or the like.
When the monitoring index value satisfies the preset condition, the elastic scaling controller 102 may adjust the application instance according to the resource models of the plurality of cloud platforms 104 and the elastic scaling policy. The preset conditions can be set according to actual requirements. For example, the preset condition may be that the monitoring index value is larger than a threshold corresponding to the index, or that the monitoring index value is smaller than a threshold corresponding to the index, or the like. The threshold corresponding to the index may be set according to an empirical value, which is not limited in the embodiment of the present application.
In some implementations, when the elastic scaling controller 102 adjusts the application instance, the application instance may be adjusted according to the resource model of the multiple cloud platforms and any one or more of the affinity policy, the anti-affinity policy, the cost priority policy, and the performance priority policy.
When the application has a high requirement on the response rate, the elastic scaling controller 102 may adjust the application instance according to the resource models and the affinity policy of the plurality of cloud platforms. For example, the elastic scaling controller 102 may deploy an application instance with a higher demand for response rate to a similar region of the same cloud platform, such as to a similar node of the same region.
When the application has high requirements for reliability and availability, the elastic scaling controller 102 may adjust the application instance according to the resource models of the plurality of cloud platforms 104 and the anti-affinity policy. For example, the elastic scaling controller 102 may avoid deploying an application instance with high requirements for reliability and availability in a similar region of the same cloud platform, i.e., the elastic scaling controller 102 may deploy it on different cloud platforms 104.
When the user gives priority to the cost, the elastic expansion controller 102 may adjust the application instance according to the resource models of the plurality of cloud platforms 104 and the cost priority policy when adjusting the application instance. For example, the elastic scaling controller 102 may select the least expensive cloud platform 104 to deploy the application instance.
When the user gives priority to the performance, the elastic scaling controller 102 may adjust the application instance according to the resource models of the plurality of cloud platforms 104 and the performance priority policy when adjusting the application instance. For example, the elastic scaling controller 102 may select the cloud platform 104 with the best performance to deploy the application instance.
In some implementations, when the elastic scaling controller 102 adjusts the application instances, the number of the application instances may be adjusted according to the resource models of the plurality of cloud platforms and the elastic scaling policy. For example, the elastic scaling controller 102 may add new application instances according to resource models of multiple cloud platforms and elastic scaling policies. For another example, the elastic scaling controller 102 may delete an application instance according to the resource models of the plurality of cloud platforms and the elastic scaling policy.
In other implementations, when the elastic scaling controller 102 adjusts the application instance, the configuration of the application instance may be adjusted according to the resource models of the multiple cloud platforms and the elastic scaling policy. For example, the elastic scaling controller 102 may increase the configuration of the application instance from 1 core 1G (indicating that the processor is a single core and the memory is 1G) to 4 cores 2G (indicating that the processor is a 4 core and the memory is 2G).
The following describes the adjustment process in detail by taking the number of application examples as an example.
Specifically, the elastic scaling controller 102 may determine a target cloud platform according to the resource models of the plurality of cloud platforms and the elastic scaling policy, and then create a new application instance through an API of the target cloud platform. The elastic scaling strategy is assumed in this example to be a cost-first strategy. When the monitoring index value is greater than the corresponding threshold value, the elastic expansion controller 102 needs to add an application instance.
Therefore, the elastic expansion controller 102 may determine how many metering units of resources of the cloud platform correspond to one metering unit of standard resources through a corresponding relationship between a resource parameter format supported by each cloud platform in the resource model and a standard resource parameter format, and then determine, based on the costs of various specifications of various resources, the cloud platform with the lowest total cost of the deployed application instance as the target cloud platform. The elastic scaling controller 102 calls the API to create a new application instance on the target cloud platform, thereby ensuring the quality of service through a large number of application instances.
Further, after the application instances are newly added, when the monitoring index values of the application instances deployed on the multiple cloud platforms do not satisfy the preset condition, the elastic scaling controller 102 may also delete the new application instances through the API, thereby avoiding resource waste.
It should be noted that, when an application instance is newly added or deleted, the elastic scaling controller 102 may also select an appropriate cloud platform, that is, a target cloud platform, and create or delete the application instance through the hybrid cloud API gateway.
The elastic expansion method in a cloudy environment provided by the embodiment of the present application is described in detail with reference to fig. 1 to 5, and the apparatus and the device provided by the embodiment of the present application are described with reference to the accompanying drawings.
Referring to fig. 6, a schematic structural diagram of an elastic expansion device in a cloudy environment is shown, where the device 600 includes:
a modeling unit 602, configured to model resources provided by multiple cloud platforms, and obtain resource models of the multiple cloud platforms;
an adjusting unit 604, configured to adjust the application instance according to the resource models of the multiple cloud platforms and the elastic scaling policy when the monitoring index value of the application instance deployed on the multiple cloud platforms satisfies a preset condition.
In some possible implementations, the adjusting unit 604 is specifically configured to:
and adjusting the application examples according to the resource models of the cloud platforms and any one or more of an affinity strategy, an anti-affinity strategy, a cost priority strategy and a performance priority strategy.
In some possible implementations, the adjusting unit 604 is specifically configured to:
adjusting the number of the application instances according to the resource models of the cloud platforms and an elastic expansion strategy; or adjusting the configuration of the application instance according to the resource models of the cloud platforms and the elastic scaling strategy.
In some possible implementations, the adjusting unit 604 is specifically configured to:
determining a target cloud platform according to the resource models of the cloud platforms and an elastic expansion strategy;
creating a new application instance through an Application Programming Interface (API) of the target cloud platform.
In some possible implementations, the adjusting unit 604 is further configured to:
after a new application instance is created, when the monitoring index values of the application instances deployed on the plurality of cloud platforms do not meet preset conditions, deleting the new application instance through the API.
In some possible implementations, the plurality of cloud platforms includes a plurality of public cloud platforms, or a plurality of private cloud platforms, or a hybrid cloud platform formed by at least one public cloud platform and at least one private cloud platform.
The elastic expansion device 600 in the cloudy environment according to the embodiment of the present application may correspond to performing the method described in the embodiment of the present application, and the above and other operations and/or functions of each module/unit of the elastic expansion device 600 in the cloudy environment are respectively for implementing corresponding processes of each method in the embodiment shown in fig. 5, and are not described herein again for brevity.
The embodiment of the application also provides a device 700. The device 700 may be a peer-side device such as a laptop computer or a desktop computer, or may be a computer cluster in a cloud environment or an edge environment. The elastic expansion controller 102 is disposed in the apparatus 700, and the apparatus 700 is specifically used for realizing the functions of the elastic expansion device 600 in a cloudy environment in the embodiment shown in fig. 6.
Fig. 7 provides a schematic diagram of a structure of a device 700, and as shown in fig. 7, the device 700 includes a bus 701, a processor 702, a communication interface 703, and a memory 704. The processor 702, memory 704, and communication interface 703 communicate over a bus 701.
The bus 701 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 7, but this is not intended to represent only one bus or type of bus.
The processor 702 may be a Central Processing Unit (CPU). One or more of a Graphic Processing Unit (GPU), a Micro Processor (MP), a Digital Signal Processor (DSP), and the like.
The communication interface 703 is used for communication with the outside. For example, the cost of obtaining various specifications of various types of resources on the plurality of cloud platforms 104, or the performance of various specifications of various types of resources on the plurality of cloud platforms 104, etc. may be obtained.
The memory 704 may include volatile memory (volatile memory), such as Random Access Memory (RAM). The memory 704 may also include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard drive (HDD) or a Solid State Drive (SSD).
The memory 704 stores executable code that the processor 702 executes to perform the aforementioned elastic scaling method in a cloudy environment.
Specifically, in the case of implementing the embodiment shown in fig. 6, and in the case that the units of the elastic telescopic device 600 in the cloudy environment described in the embodiment of fig. 6 are implemented by software, software or program codes required for executing the functions of the modeling unit 602 and the elastic telescopic control unit 604 in fig. 6 are stored in the memory 704. The communication module functions are implemented through the communication interface 703.
The communication interface 703 receives resource parameter formats of the multiple cloud platforms 104, and transmits the resource parameter formats to the processor 702 through the bus 701, and the processor 702 executes program codes corresponding to units stored in the memory 704, such as program codes corresponding to the modeling unit 602 and the elastic scaling control unit 604, to perform modeling on resources provided by the multiple cloud platforms according to the resource parameter formats and the standard resource parameter formats of the multiple cloud platforms 104, to obtain resource models of the multiple cloud platforms, and when a monitoring index value of an application instance deployed on the multiple cloud platforms meets a preset condition, adjusts the application instance according to the resource models and the elastic scaling policies of the multiple cloud platforms.
In some implementations, the processor 702 is specifically configured to execute the program code corresponding to the elastic expansion control unit 604, so as to perform the following method steps:
and adjusting the application examples according to the resource models of the cloud platforms and any one or more of an affinity strategy, an anti-affinity strategy, a cost priority strategy and a performance priority strategy.
In some implementations, the processor 702 is specifically configured to execute the program code corresponding to the elastic expansion control unit 604, so as to perform the following method steps:
adjusting the number of the application instances according to the resource models of the cloud platforms and an elastic expansion strategy; alternatively, the first and second electrodes may be,
and adjusting the configuration of the application instance according to the resource models of the cloud platforms and the elastic scaling strategy.
In some implementations, the processor 702 is specifically configured to execute the program code corresponding to the elastic expansion control unit 604, so as to perform the following method steps:
determining a target cloud platform according to the resource models of the cloud platforms and an elastic expansion strategy;
creating a new application instance through an Application Programming Interface (API) of the target cloud platform.
In some implementations, the processor 702 is specifically configured to execute the program code corresponding to the elastic expansion control unit 604, so as to perform the following method steps:
after a new application instance is created, when the monitoring index values of the application instances deployed on the plurality of cloud platforms do not meet preset conditions, deleting the new application instance through the API.
An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes instructions that instruct a computer to execute the above elastic expansion and contraction method applied to the elastic expansion and contraction device 600 in a cloudy environment.
An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes instructions that instruct a computer to execute the above elastic expansion and contraction method applied to the elastic expansion and contraction device 600 in a cloudy environment.
The embodiment of the application also provides a computer program product, and when the computer program product is executed by a computer, the computer executes any one of the methods of the elastic scaling method in the multi-cloud environment. The computer program product may be a software installation package, and in case that any one of the aforementioned elastic scaling methods in a cloudy environment needs to be used, the computer program product may be downloaded and executed on a computer.
It should be noted that the above-described embodiments of the apparatus are merely schematic, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiments of the apparatus provided in the present application, the connection relationship between the modules indicates that there is a communication connection therebetween, and may be implemented as one or more communication buses or signal lines.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus necessary general-purpose hardware, and certainly can also be implemented by special-purpose hardware including special-purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose components and the like. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions may be various, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, the implementation of a software program is more preferable. Based on such understanding, the technical solutions of the present application may be substantially embodied in the form of a software product, which is stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, an exercise device, or a network device) to execute the method according to the embodiments of the present application.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.
The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, training device, or data center to another website site, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a training device, a data center, etc., that incorporates one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
Claims (14)
1. An elastic expansion method in a cloudy environment, the method comprising:
modeling resources provided by a plurality of cloud platforms to obtain resource models of the cloud platforms;
and when the monitoring index values of the application examples deployed on the cloud platforms meet preset conditions, adjusting the application examples according to the resource models and the elastic scaling strategies of the cloud platforms.
2. The method of claim 1, wherein the adjusting the application instance according to the resource model of the plurality of cloud platforms and the elastic scaling policy comprises:
and adjusting the application examples according to the resource models of the cloud platforms and any one or more of an affinity strategy, an anti-affinity strategy, a cost priority strategy and a performance priority strategy.
3. The method of claim 1 or 2, wherein the adapting the application instance according to the resource model of the plurality of cloud platforms and the elastic scaling policy comprises:
adjusting the number of the application instances according to the resource models of the cloud platforms and an elastic expansion strategy; alternatively, the first and second electrodes may be,
and adjusting the configuration of the application instance according to the resource models of the cloud platforms and the elastic scaling strategy.
4. The method of any of claims 1 to 3, wherein said adapting the application instance according to the resource model of the plurality of cloud platforms and the elastic scaling policy comprises:
determining a target cloud platform according to the resource models of the cloud platforms and an elastic expansion strategy;
creating a new application instance through an Application Programming Interface (API) of the target cloud platform.
5. The method of claim 4, wherein after creating the new application instance, the method further comprises:
and when the monitoring index values of the application instances deployed on the plurality of cloud platforms do not meet the preset conditions, deleting the new application instances through the API.
6. The method of any one of claims 1 to 5, wherein the plurality of cloud platforms comprises a plurality of public cloud platforms, or a plurality of private cloud platforms, or a hybrid cloud platform formed by at least one public cloud platform and at least one private cloud platform.
7. An elastic telescopic device in a cloudy environment, the device comprising:
the modeling unit is used for modeling resources provided by a plurality of cloud platforms to obtain resource models of the cloud platforms;
and the adjusting unit is used for adjusting the application examples according to the resource models and the elastic scaling strategies of the cloud platforms when the monitoring index values of the application examples deployed on the cloud platforms meet preset conditions.
8. The apparatus according to claim 7, wherein the adjusting unit is specifically configured to:
and adjusting the application examples according to the resource models of the cloud platforms and any one or more of an affinity strategy, an anti-affinity strategy, a cost priority strategy and a performance priority strategy.
9. The apparatus according to claim 7 or 8, wherein the adjusting unit is specifically configured to:
adjusting the number of the application instances according to the resource models of the cloud platforms and an elastic expansion strategy; alternatively, the first and second electrodes may be,
and adjusting the configuration of the application instance according to the resource models of the cloud platforms and the elastic scaling strategy.
10. The apparatus according to any one of claims 7 to 9, wherein the adjusting unit is specifically configured to:
determining a target cloud platform according to the resource models of the cloud platforms and an elastic expansion strategy;
creating a new application instance through an Application Programming Interface (API) of the target cloud platform.
11. The apparatus of claim 10, wherein the adjustment unit is further configured to:
after a new application instance is created, when the monitoring index values of the application instances deployed on the plurality of cloud platforms do not meet preset conditions, deleting the new application instance through the API.
12. The apparatus of any of claims 7 to 11, wherein the plurality of cloud platforms comprises a plurality of public cloud platforms, or a plurality of private cloud platforms, or a hybrid cloud platform formed by at least one public cloud platform and at least one private cloud platform.
13. An apparatus, comprising a processor and a memory;
the processor is to execute instructions stored in the memory to cause the device to perform the method of any of claims 1 to 6.
14. A computer-readable storage medium comprising instructions that direct a device to perform the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010495551.5A CN113760516A (en) | 2020-06-03 | 2020-06-03 | Elastic expansion method, device, equipment and medium in multi-cloud environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010495551.5A CN113760516A (en) | 2020-06-03 | 2020-06-03 | Elastic expansion method, device, equipment and medium in multi-cloud environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113760516A true CN113760516A (en) | 2021-12-07 |
Family
ID=78783257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010495551.5A Pending CN113760516A (en) | 2020-06-03 | 2020-06-03 | Elastic expansion method, device, equipment and medium in multi-cloud environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113760516A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114513344A (en) * | 2022-01-26 | 2022-05-17 | 鼎捷软件股份有限公司 | Integration system and method between cloud applications |
| CN115208891A (en) * | 2022-07-15 | 2022-10-18 | 济南浪潮数据技术有限公司 | Hybrid cloud elastic expansion method, device, equipment and storage medium |
| WO2023106980A1 (en) * | 2021-12-10 | 2023-06-15 | Telefonaktiebolaget Lm Ericsson (Publ) | Scaling arrangement and method performed therein |
-
2020
- 2020-06-03 CN CN202010495551.5A patent/CN113760516A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023106980A1 (en) * | 2021-12-10 | 2023-06-15 | Telefonaktiebolaget Lm Ericsson (Publ) | Scaling arrangement and method performed therein |
| CN114513344A (en) * | 2022-01-26 | 2022-05-17 | 鼎捷软件股份有限公司 | Integration system and method between cloud applications |
| TWI795207B (en) * | 2022-01-26 | 2023-03-01 | 大陸商鼎捷軟件股份有限公司 | Integrated system between multi-cloud applications and method thereof |
| CN114513344B (en) * | 2022-01-26 | 2024-05-24 | 鼎捷软件股份有限公司 | Integration system and method between cloud applications |
| CN115208891A (en) * | 2022-07-15 | 2022-10-18 | 济南浪潮数据技术有限公司 | Hybrid cloud elastic expansion method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106537338B (en) | Self-expanding clouds | |
| US11714686B2 (en) | Resource oversubscription based on utilization patterns in computing systems | |
| US11698782B2 (en) | Determining customized software recommendations for network devices | |
| WO2019062304A1 (en) | Method, device and system for managing computing resources of block chain node | |
| CN113760516A (en) | Elastic expansion method, device, equipment and medium in multi-cloud environment | |
| US9430257B2 (en) | Scheduling virtual machines using user-defined rules | |
| US20200314168A1 (en) | Distributed code execution involving a serverless computing infrastructure | |
| US20180253246A1 (en) | Method and system for memory allocation in a disaggregated memory architecture | |
| US10523580B2 (en) | Automatic cloud provisioning based on related internet news and social network trends | |
| US10594800B2 (en) | Platform runtime abstraction | |
| CN109873714B (en) | Cloud computing node configuration updating method and terminal equipment | |
| US10540452B1 (en) | Automated translation of applications | |
| US20210406053A1 (en) | Rightsizing virtual machine deployments in a cloud computing environment | |
| CN114706690B (en) | Method and system for sharing GPU (graphics processing Unit) by Kubernetes container | |
| CN108293047B (en) | System and method for accessing resources by a user across multiple distributed computing networks | |
| CN111078398A (en) | GPU (graphics processing Unit) distribution method, equipment and storage medium | |
| US10884845B2 (en) | Increasing processing capacity of processor cores during initial program load processing | |
| CN110795202A (en) | Resource allocation method and device of virtualized cluster resource management system | |
| CN113590285A (en) | Method, system and equipment for dynamically setting thread pool parameters | |
| CN111712795A (en) | Method, apparatus, computer program product and readable medium for evaluating application deployment | |
| US20200042331A1 (en) | Dynamic-link library usage based on memory size | |
| CN115794396A (en) | Resource allocation method, system and electronic equipment | |
| CN115328608A (en) | Kubernetes container vertical expansion adjusting method and device | |
| CN115033551A (en) | Database migration method and device, electronic equipment and storage medium | |
| CN113138772B (en) | Construction method and device of data processing platform, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220207 Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Applicant after: Huawei Cloud Computing Technology Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Applicant before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |