CN112182100B - Cloud management platform state data synchronization method and system - Google Patents
Cloud management platform state data synchronization method and system Download PDFInfo
- Publication number
- CN112182100B CN112182100B CN202010999792.3A CN202010999792A CN112182100B CN 112182100 B CN112182100 B CN 112182100B CN 202010999792 A CN202010999792 A CN 202010999792A CN 112182100 B CN112182100 B CN 112182100B
- Authority
- CN
- China
- Prior art keywords
- resource
- state
- activity
- cloud
- synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5011—Pool
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5021—Priority
Abstract
The invention discloses a method and a system for synchronizing state data of a cloud management platform, which configure activity convergence factors for various operations of various resources influencing state changes in cloud computing; when the resource has an operation of influencing state change, setting the activity degree as a maximum value and dividing the activity degree into active areas, carrying out activity degree reduction on the resource according to corresponding activity degree convergence factors, and dividing the resource into static areas when the activity degree is 0; the state synchronization thread pool periodically generates a fixed number of state synchronization threads, the state synchronization threads are distributed to the active area and the static area according to a preset proportion, each thread obtains the latest state for one resource in the corresponding area, and the latest state is synchronously updated in a database of the cloud management platform. The invention sets activity and activity convergence factors for various resources, and divides the resources into active areas or static areas according to the activity, so that the resource state with high activity is synchronous and frequent, the resource with low activity is synchronous updated at low frequency, and the efficiency and the cost are considered.
Description
Technical Field
The invention belongs to the technical field of cloud computing, and particularly relates to a method and a system for synchronizing state data of a cloud management platform.
Background
With reference to fig. 1, a cloud computing management platform (cloud management platform for short) may manage and control various cloud computing resources (e.g., virtual machines, cloud hard disks, etc.) in a public cloud (e.g., AWS, aristoloc, tencent cloud, etc.) and a private cloud (e.g., openstack-based cloud computing system). Generally, a cloud management platform is designed with a database thereof for storing information data of various underlying cloud computing resources. In order to present the accurate state of the resource to the user, the cloud management platform needs to keep the resource state (e.g., in creation, in operation, in derivation, in deletion, deleted, etc.) in its database consistent with the resource state in the managed private cloud and public cloud, that is, the upper layer resource state needs to be consistent with the lower layer resource state.
After the cloud management platform operates the resources, the states of the underlying resources in the private cloud and the public cloud are converted through the state machine; for example, after an Openstack virtual machine is created, the state of the virtual machine may go through intermediate states such as "in scheduling", "being derived", "allocating network", "block device mapping", and finally enter a "running" steady state. No matter the cloud management platform is a private cloud or a public cloud, the change of the states cannot actively inform the cloud management platform, so that the cloud management platform is required to realize a synchronization mechanism to acquire the latest real state of the resource in time. When the state recorded in the database of the self is inconsistent with the real state of the resource, the state data in the database of the self needs to be updated. That is, when the cloud management platform finds that the data maintained by the upper layer is inconsistent with the bottom layer, synchronization is required.
The currently common synchronization mechanism is that the cloud management platform periodically polls a managed private cloud or public cloud for each resource, acquires the state of the resource, and synchronously updates the state to the database of the cloud management platform. However, such an approach has the following disadvantages:
1) When the resource enters a stable state, the state can not be changed any more when the resource is not operated, and at the moment, if the resource is still inquired and the state is updated conventionally, the meaning is not great and the system resource is consumed in an idle way; especially when the system resource is insufficient, the state of the resource (such as the resource just created) with frequently changed state can not be updated in time.
2) When the number of resources is large, if each resource is inquired one by one, the time consumption of polling is long, and the resource updating rate is slow; and if the query is carried out concurrently, great request load pressure is caused to the private cloud or the public cloud.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides a cloud management platform state data synchronization method and system, aiming at synchronizing resource states according to the active conditions of various cloud computing resources, so that the technical problems of system resource waste, long time consumption, high request load pressure and the like of the traditional periodic polling method are solved.
In order to achieve the above object, according to an aspect of the present invention, there is provided a cloud management platform status data synchronization method, including:
configuring activity convergence factors for various operations of the cloud computing various resources, which influence the state change;
when any resource has an operation of influencing state change, setting the activity of the resource to be the maximum value and dividing the resource into an 'active zone', then carrying out activity deduction on the resource according to a corresponding activity convergence factor, and dividing the resource into a 'static zone' when the activity is 0;
the method comprises the steps that a state synchronization thread pool periodically generates a fixed number of state synchronization threads, the state synchronization threads are distributed to an active area and a static area according to a preset proportion, and the number of the state synchronization threads distributed to the active area is more than that of the static area;
and each state synchronization thread calls a state query interface of a public cloud or a private cloud aiming at one resource in the corresponding area, acquires the latest state of the resource and synchronously updates the database of the cloud management platform.
Preferably, when any resource has an operation that affects a state change, the method further includes:
determining the associated resources and corresponding operations of the resources by inquiring an associated resource rule table, setting the activity of the associated resources to be a maximum value, and dividing the associated resources into 'active areas';
and deducting the activity of the associated resources according to the corresponding activity convergence factors, and dividing the associated resources into 'static areas' when the activity of the associated resources is 0.
Preferably, for any resource in the "quiet zone", when the resource has an operation affecting the state change, or when the resource associated with the resource has an operation affecting the state change, the resource is transferred from the "quiet zone" back to the "active zone", and the activity is reset to the maximum value.
Preferably, in the 'active zone', the allocated state synchronization thread processes the resource with higher activity preferentially; in the "quiet zone," the allocated state synchronization thread randomly selects resources for processing.
Preferably, when the number of resources in the "active area" exceeds a preset threshold, the status synchronization thread pool adjusts the status synchronization thread allocation proportion, so that the number of status synchronization threads allocated to the "active area" is increased, and the number of status synchronization threads allocated to the "quiet area" is decreased.
Preferably, for any resource in the "active zone", when the state synchronization thread processes the state synchronization of the resource, if the resource state on the underlying private cloud or public cloud is found to enter the steady state, the activity of the resource is directly set to 0, and the resource is put into the "quiescent zone" in advance.
Preferably, for any resource in the "active region", after the state synchronization thread processes the resource and completes the state synchronization update, the cloud management platform sets an updated flag for the resource, and releases the updated flag after a first preset time;
for any resource in the 'static area', when the state synchronization thread processes the resource and completes the state synchronization update, the cloud management platform sets an updated mark for the resource and releases the updated mark after a second preset time;
and the first preset time is less than the second preset time, and the state synchronization thread with the updated mark cannot be distributed for processing again.
According to another aspect of the present invention, a cloud management platform status data synchronization system is provided, which includes a parameter configuration module, a first communication module, an activity management module, a thread management scheduling module, a status update module, and a second communication module;
the parameter configuration module is used for configuring an activity convergence factor for the operation of changing various influence states of various resources by an administrator of the cloud management platform;
the first communication module is used for communicating with other modules of the cloud management platform, informing the first communication module when other modules of the cloud management platform generate operation influencing state change, and informing the activity management module of the activity of the processing resources by the first communication module;
the activity management module is used for managing the increase and decrease of the activity of the resource and realizing the conversion of the resource between an active area and a static area according to the activity;
the thread management scheduling module is used for constructing a state synchronization thread pool, controlling the state synchronization thread pool to periodically generate a fixed number of state synchronization threads, distributing the state synchronization threads to an active area and a static area according to a preset proportion, and enabling the number of the state synchronization threads distributed to the active area to be more than that of the static area;
the state updating module is used by the state synchronization thread and is used for calling a state query interface of a public cloud or a private cloud, acquiring the latest state of the resource and updating the latest state of the resource in a database of the cloud management platform synchronously;
the second communication module is used for communicating with the bottom resource platform and is connected with the state updating module, so that the state synchronization thread can obtain the latest state of the resource from the bottom resource platform through the second communication module.
Preferably, the system further comprises an associated resource management module, configured to identify an associated resource of the resource and a corresponding operation through an associated resource rule table, and deliver the associated resource to the activity management module for activity processing.
Preferably, the system further comprises a mark management module, configured to set an updated mark for the resource whose status is updated synchronously, release the updated mark after a first preset time for the resource in the "active zone", and release the updated mark after a second preset time for the resource in the "static zone"; the first preset time is less than the second preset time.
Generally, compared with the prior art, the technical scheme of the invention has the following beneficial effects: in the cloud management platform state data synchronization method and system provided by the invention, an active area and a static area are divided for resources, and state data synchronization strategies in different areas are different; through setting activity and activity convergence factors for various resources managed by the cloud management platform, the resources are divided into active areas or static areas according to different activities, different state data synchronization strategies are further executed, the resource state synchronization of high activity is more frequent, the resources of low activity are kept for the synchronization updating of lower frequency, the efficiency and the cost can be considered, and the state data synchronization of various resources of the cloud management platform can be reasonably realized.
Drawings
Fig. 1 is a schematic view of a usage scenario of cloud management platform state data synchronization according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a method for synchronizing status data of a cloud management platform according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an activity convergence factor table maintained by a cloud management platform according to an embodiment of the present invention;
FIG. 4 is a schematic resource partition diagram of a cloud management platform according to an embodiment of the present invention;
fig. 5 is a representation intention of an associated resource rule maintained by a cloud management platform according to an embodiment of the present invention;
fig. 6 is a structural diagram of a cloud management platform state data synchronization system according to an embodiment of the present invention;
fig. 7 is a schematic flowchart of a method for synchronizing status data when a virtual machine is deleted according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention. In addition, the technical features involved in the respective embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1
In order to solve the technical problems of system resource waste, long time consumption, large request load pressure and the like when the traditional periodic polling method carries out state data synchronization, an embodiment of the invention provides a cloud management platform state data synchronization method, which mainly comprises the following steps as shown in fig. 2:
step S201, configuring an activity convergence factor for the operation of changing various influence states of the cloud computing resources.
The various resources of cloud computing include, but are not limited to, a virtual machine, a cloud hard disk (or called a storage volume), a subnet (or called a virtual switch), a router, load balancing, and the like, and the operation range affecting state change is different according to different resource types. Taking a virtual machine as a typical resource in the cloud management platform as an example, the operation affecting the state change of the virtual machine includes creating, migrating, starting, stopping, restarting, suspending, restoring, snapshotting, deleting, and the like. According to the scheme, the activity attribute is added on the cloud computing resource, and the administrator configures 'activity convergence factors' on the cloud management platform aiming at the operation of different influence state changes of different resources respectively, so that the activity of the resources is gradually reduced along with the time until the activity is 0. The 'activity convergence factor' determines the descending rate of the activity of the resource, and represents the time consumption estimation of the state of the resource from an intermediate state (or a transition state) to a steady state after certain operation, the size of the factor is related to the type of the resource, the type of the operation, the type of a platform of a private cloud or a public cloud, the performance of the platform and the like, and an administrator can flexibly configure the resource according to the actual environment.
Taking a virtual machine as an example, the state of the virtual machine during the operation of the virtual machine is established to pass through transition states such as scheduling, deriving, network distribution, block device mapping and the like, and finally enters a steady state in operation, and each stage takes long time, so that the corresponding activity convergence factor can be configured to be 2 points per second; the time required for the virtual machine state to converge to a stopped steady state is short, and thus the corresponding "liveness convergence factor" may be configured to be 10 points per second. After the administrator configures the 'activity convergence factor' of each operation of each resource, the cloud management platform maintains an activity convergence factor table, and records the activity convergence factor gamma of each operation of each resource; taking an Openstack private cloud platform in a certain deployment environment as an example, an activity convergence factor table maintained by a cloud management platform is shown in fig. 3.
Further, the "activity convergence factor" is usually strongly associated with some of the attributes of the resource, so in a preferred embodiment, the administrator can set the "activity convergence factor" as a variable depending on some attribute of the resource. For example, when a snapshot operation is performed on a virtual machine, the time required for the virtual machine to converge from a transient state to a steady state in the snapshot depends on the total capacity size of all system disks and data disks in the virtual machine. The "liveness convergence factor" of the virtual machine snapshot may thus be set in relation to the total capacity of system disks and data disks in the virtual machine: when the total capacity is larger, the configuration of the 'activity convergence factor' is smaller, which indicates that the activity is reduced more slowly; otherwise, the configuration of the activity convergence factor is larger, which indicates that the activity is more rapidly reduced.
Step S202, when any resource has an operation of influencing state change, setting the activity of the resource to be the maximum value and dividing the resource into an 'active zone', then carrying out activity deduction on the resource according to a corresponding activity convergence factor, and dividing the resource into a 'static zone' when the activity is 0;
through the size of the activity, the cloud management platform can logically divide the resources into two areas, wherein the resources with the activity greater than 0 are divided into an "active area" and the resources with the activity of 0 are divided into a "static area", as shown in fig. 4. When an operation affecting the resource state change occurs, the resource is placed in an "active zone" and the activity level is set to the maximum value (here, 100 is taken as an example); then the activity of the resource is gradually reduced to 0 along with the time, the reduction rate depends on the corresponding 'activity convergence factor', and the resource automatically moves into a 'static area' when the attenuation is 0. Taking fig. 4 as an example, there are two resources in the current "quiet zone": the corresponding liveness of the virtual machine 18 and the cloud hard disk 16 is 0; there are four resources within the "active region": when the virtual machine 01 is created, the virtual machine 02 is started, the cloud hard disk 02 is created, and the virtual machine 03 is deleted, the activity of corresponding resources is set to be the maximum value of 100, an 'active area' is placed, and activity deduction is carried out according to the corresponding activity convergence factor.
Furthermore, each resource also has one or more associated resources, and when some operation affecting the state change occurs to the resource, the associated resource also has the operation affecting the state change. Taking a virtual machine as an example, the associated resources of the virtual machine comprise a cloud hard disk, a network card and a snapshot which are hung on the virtual machine; when the virtual machine is deleted, the cloud hard disk, the network card and the snapshot bound to the virtual machine are also deleted, so that the state is also changed. In a preferred embodiment, the cloud management platform sets an "associated resource rule table," as shown in fig. 5, for finding an associated resource that a resource may affect under a certain operation, and the operation triggered by the associated resource. Therefore, when any resource has an operation that affects the state change, the activity management and the partition operation should be performed on the resource associated with the resource synchronously, specifically: determining the associated resources and corresponding operations of the resources by inquiring an associated resource rule table, setting the activity of the associated resources to be a maximum value, and dividing the associated resources into 'active areas'; and then, deducting the activity of the associated resources according to the corresponding activity convergence factor, and dividing the associated resources into 'static areas' when the activity of the associated resources is 0.
When any resource in the 'quiescent zone' has an operation of influencing the state change or when the resource related to the resource has an operation of influencing the state change, the resource is transferred back to the 'active zone' from the 'quiescent zone', and the activity is reset to the maximum value.
As known from the subsequent steps, the resource status update frequency in the "active zone" is higher than that in the "quiescent zone". The administrator sets the 'activity convergence factor' to make the resource fall into the 'static area' after a certain time and reduce the updating frequency, thus preventing the problem that the resource state is stuck in a transition state due to various abnormal reasons and is always in the 'active area' to cause a large amount of invalid updates.
Step S203, the state synchronization thread pool periodically generates a fixed number of state synchronization threads, and allocates the state synchronization threads to the "active area" and the "quiescent area" according to a preset ratio, and the number of the state synchronization threads allocated to the "active area" is greater than that of the "quiescent area".
A 'state synchronization thread pool' is arranged in the cloud management platform, and periodically generates a fixed number of state synchronization threads and distributes the state synchronization threads to an 'active area' and a 'static area' according to a preset proportion. For example, 10 state synchronization threads are generated every 2 seconds and are allocated to an "active zone" and a "quiescent zone" in a fixed proportion of 4. The total number of the generated state synchronization threads is related to the performance of the cloud management platform, and can be set and adjusted according to actual conditions; for example, when the cloud management platform is distributed, resources are many, and the total number of threads to be generated is relatively large; when the cloud management platform is a single-machine system, the resources are less, and the total number of threads to be generated is smaller. The number of threads allocated to the "active zone" and the "quiet zone" is determined by the total number of threads and the allocation proportion, the initial value of the allocation proportion can be set according to the proportion of the number of resources in the initial "active zone" and the "quiet zone", and the initial value can be flexibly adjusted according to the change conditions of the number of resources in the "active zone" and the "quiet zone".
And each state synchronization thread calls a state query interface of a public cloud or a private cloud aiming at one resource in the corresponding area, acquires the latest state of the resource, and synchronously updates the database of the cloud management platform, thereby completing the state data synchronization of the cloud management platform. And each state synchronization thread can be naturally ended after finishing the query and synchronous update of the corresponding resource state, and because the functions in the thread are completely executed at the moment, the life cycle can be automatically ended, and the problem of thread overstocking does not exist.
Because the state synchronization thread is generated at a constant speed, even if the usage amount of the cloud management platform user is increased rapidly, the suddenly increased query pressure on the private cloud and the public cloud can not be caused. The number of the state synchronization threads distributed to the active area is more than that of the quiescent area, so that the resource state updating frequency of the active area is higher than that of the quiescent area, and system resources are reasonably utilized. The reason for letting resources in the "quiet zone" update the state at a slow frequency instead of not at all is that: the private cloud and the public cloud are usually operated by other management platforms, and the state of the resource is changed due to the third-party operation, so that the state of the resource in the 'static area' is updated at a slow frequency, and the cloud management platform can still obtain the latest state of the resource after a certain time.
Further, in a preferred embodiment, in an "active zone", the allocated state synchronization thread preferentially processes resources with higher activity; in the "quiet zone," the allocated state synchronization thread randomly selects resources for processing. When the number of resources in the 'active zone' is increased and exceeds a preset threshold value, the state synchronization thread pool can adjust the allocation proportion of the state synchronization threads, so that the number of the state synchronization threads allocated to the 'active zone' is increased, and the number of the state synchronization threads allocated to the 'static zone' is reduced, thereby preferentially ensuring the state updating efficiency of the 'active zone' resources and reasonably utilizing the system resources.
Further, in a preferred embodiment, for any resource in the "active area", when the state synchronization thread processes the state synchronization of the resource, if it is found that the resource state on the underlying private cloud or public cloud has entered the steady state, the activity of the resource is directly set to 0, and the resource is placed in the "quiescent area" in advance, so that the subsequent state update request frequency of the resource is reduced, and the request load pressure is reduced.
Further, considering that the state of the resource whose state has been updated does not generally change in a short period of time, in a preferred embodiment, for any resource in the "active region," when the state synchronization thread processes the resource and completes one state synchronization update, the cloud management platform sets an updated flag for the resource and releases the updated flag after a first preset time; for any resource in the 'quiescent zone', when the state synchronization thread processes the resource and completes one state synchronization update, the cloud management platform sets an updated mark for the resource and releases the updated mark after a second preset time. The first preset time is less than the second preset time, and the state synchronization thread which is not distributed in a short period and has the updated mark is processed again, so that unnecessary invalid updating is reduced. Referring to fig. 4, "update identification: yes indicates that the resource has an updated flag, update identification: no "indicates that the resource does not have an updated flag, i.e., the flag is removed. For example, if the first preset time is 5 seconds and the second preset time is 10 minutes, the resource in the "active region" having the flag will not be processed again by the state synchronization thread within 5 seconds; the resource in the "quiet zone" that owns this flag will not be processed again by the state synchronization thread within 10 minutes.
In summary, in the cloud management platform state data synchronization method provided in the embodiment of the present invention, by setting the activity and the activity convergence factor for various resources managed by the cloud management platform, and then dividing the resources into the active area and the static area according to different activities, and the number of the state synchronization threads allocated to the active area is higher than that of the static area, different state data synchronization policies for the resources in the active area and the static area are implemented, so that the state synchronization of the high activity resources with rapidly changing states is more frequent, and the low activity resources with slowly changing states retain lower frequency synchronization update, which not only satisfies the real-time acquisition of state changes, but also reduces the system load, thereby taking efficiency and cost into account, and reasonably implementing the state data synchronization of various resources of the cloud management platform.
In addition, the state synchronization thread pool sends state synchronization threads to the active area and the static area at a constant speed, so that the requests of the private cloud and the public cloud on the bottom layer are called stably and smoothly, and the problem that a large number of tasks initiate a large number of state synchronization queries when the request peak value comes is solved; the resources in the updated state cannot be updated again in a short period, so that invalid update requests are reduced, and repeated updating of the same resource is avoided; by setting the 'activity convergence factor', the resource is caused to fall into a 'static area' after a certain time to reduce the updating frequency, thereby preventing the resource state from being stuck in a certain transition state due to various abnormal conditions and causing the problem of being always positioned in an 'active area' repeated high-frequency updating state; by setting the 'static area', the resource in the static area is enabled to be in a state updating state at a slow speed instead of not being in a state updating state at all, so that the resource state change caused by the third-party management platform in which the private cloud is in butt joint with the public cloud can be finally synchronized to the cloud management platform.
Example 2
On the basis of the foregoing embodiment 1, an embodiment of the present invention further provides a cloud management platform state data synchronization system, which can be used to implement the cloud management platform state data synchronization method in embodiment 1. As shown in fig. 6, the state data synchronization system provided in the embodiment of the present invention mainly includes: the system comprises a parameter configuration module, a first communication module, an activity management module, an associated resource management module, a thread management scheduling module, a state updating module, a mark management module and a second communication module. The functions of the modules are as follows:
the parameter configuration module serves an administrator of the cloud management platform, and is used for the administrator of the cloud management platform to respectively configure activity convergence factors for various operations of influencing state changes of various resources of cloud computing and maintain an activity convergence factor table. The parameter configuration module is connected with a user configuration interface, and an administrator configures the parameter configuration module through the user configuration interface.
The first communication module is used for communicating with other modules of the cloud management platform, and the communication mode can be an HTTP interface, a message queue and the like. When other modules of the cloud management platform perform operations such as resource creation, change or deletion which affect state change, the first communication module is notified, and then the first communication module notifies the activity management module of processing the activity of the resource and also notifies the associated resource management module of identifying the associated resource.
The activity management module is used for managing the increase and decrease of the activity of the resources and realizing the conversion of the resources between an active area and a static area according to the activity. When the resource has an operation of influencing state change, the activity management module sets the activity of the resource to be the maximum value and divides the activity into 'active areas', then determines a corresponding activity convergence factor by inquiring an activity convergence factor table maintained by the parameter configuration module, performs activity deduction on the resource according to the corresponding activity convergence factor, and divides the resource into 'static areas' when the activity is 0.
The associated resource management module is used for identifying the associated resources and corresponding operations of the resources by inquiring the associated resource rule table and handing the associated resources to the activity degree management module for activity degree processing. For example, when a virtual machine is deleted, the associated resource management module identifies by querying an associated resource rule table, and the associated resources of the virtual machine are a cloud hard disk, a network card and a virtual machine snapshot which are mounted on the virtual machine and need to be deleted in a linkage manner; therefore, the associated resource management module delivers the cloud hard disk, the network card and the virtual machine snapshot associated with the virtual machine to the activity management module for activity management.
The thread management scheduling module is used for constructing a state synchronization thread pool, controlling the state synchronization thread pool to periodically generate a fixed number of state synchronization threads, distributing the state synchronization threads to an active area and a static area according to a preset proportion, and enabling the number of the state synchronization threads distributed to the active area to be more than that of the static area; for example, 10 state synchronization threads are generated every 2 seconds, and are issued into the "active area" and the "quiescent area" in a ratio of 4. In addition, the thread management scheduling module can also be used for controlling the state synchronization thread to preferentially process resources with higher activity in an 'active region' and filtering out resources with 'updated marks'; and according to the resource quantity in the 'active zone', dynamically adjusting the proportion of the thread quantity allocated to the 'active zone' and the 'quiescent zone', so that the state synchronization thread quantity allocated to the 'active zone' is increased, and the state synchronization thread quantity allocated to the 'quiescent zone' is reduced.
The state updating module is used by the state synchronization thread and used for calling a state query interface of a public cloud or a private cloud through the second communication module, acquiring the latest state of the resource and updating the latest state of the resource synchronously in a database of the cloud management platform.
The mark management module is used for setting an updated mark for the resource of which the state synchronous update is completed, removing the updated mark after a first preset time for the resource in the 'active zone', and removing the updated mark after a second preset time for the resource in the 'static zone'; the first preset time is less than the second preset time.
The second communication module is used for communicating with the bottom resource platform, and the communication mode can be an HTTP interface, a message queue and the like; the second communication module is connected with the state updating module, and the state synchronization thread can acquire the latest state of the resource from the bottom resource platforms such as Openstack, kubernetes, ariiyun, tencent cloud and the like through the second communication module. When the state synchronization thread processes the state synchronization of a certain resource in an 'active region', if the second communication module learns that the resource state on the bottom resource platform enters a steady state, the state updating module informs the activity management module to process the activity of the resource; the activity management module can directly set the activity of the resource to be 0 and put the resource into a 'static area' in advance.
Through the cloud management platform state data synchronization system provided by the embodiment of the invention, the activity and activity convergence factors can be set for various resources managed by the cloud management platform, then the resources are divided into the active area and the static area according to different activities, and the number of the state synchronization processes distributed by the active area is higher than that of the static area, so that different state data synchronization strategies are executed on the resources in the active area and the static area, the state synchronization of the high-activity resources with rapidly changed states is more frequent, the low-activity resources with slowly changed states keep the synchronous update of lower frequencies, the real-time acquisition of state change is met, the system load is reduced, the efficiency and the cost are considered, and the state data synchronization of various resources of the cloud management platform is reasonably realized.
Example 3
On the basis of the foregoing embodiment 1 and embodiment 2, the embodiment of the present invention further introduces a cloud management platform state data synchronization method by taking a deletion operation of a typical resource "virtual machine" in an Openstack private cloud platform as an example, as shown in fig. 7, the specific operation steps are as follows:
and step S101, configuring an activity convergence factor for the operation of the virtual machine and the associated resources in a parameter configuration module by an administrator through a user configuration interface, and recording the activity convergence factor in an activity convergence factor table.
With reference to fig. 5, when a virtual machine is deleted, the affected associated resources include a cloud hard disk, a network card, and a virtual machine snapshot that are mounted on the virtual machine, and the operations triggered by each associated resource are deleted, and with reference to fig. 3, the corresponding activity convergence factors are configured as follows: deleting the virtual machine: 5 points per second; cloud hard disk deletion: 5 points per second; deleting the network card: 20 dots per second; deleting the snapshot of the virtual machine: 10 dots per second; these data records are then entered into the liveness convergence factor table.
And S102, when the virtual machine is deleted, other modules of the cloud management platform inform the first communication module, and the first communication module informs the activity management module and the associated resource management module of processing. And the other modules of the cloud management platform inform the first communication module in an HTTP mode or a message queue mode.
And step S103, setting the activity of the virtual machine as the maximum value by the activity management module, putting the virtual machine into an 'active zone', and performing activity deduction according to the corresponding activity convergence factor.
Taking the maximum value as 100 as an example, after receiving the notification of the first communication module, the activity management module sets the activity of the virtual machine to 100 and puts the virtual machine into an "active zone", and if the virtual machine is already in a "static zone", the activity management module transfers the virtual machine to the "active zone"; then, determining that the activity convergence factor corresponding to the deletion of the virtual machine is 5 points per second by inquiring the activity convergence factor table, and therefore, performing activity deduction on the virtual machine along with the time lapse according to 5 points per second; when the liveness decay is 0, the liveness management module transfers the virtual machine to a 'quiet zone'.
And step S104, the associated resource management module identifies the associated resources and corresponding operations related to the deletion of the virtual machine by inquiring the associated resource rule table, and delivers the identified resources and operations to the activity management module for processing.
As can be seen from the associated resource rule table shown in fig. 5, deleting the associated resources and corresponding operations involved in the virtual machine includes: and deleting a cloud hard disk on the virtual machine, a network card on the virtual machine and a snapshot of the virtual machine, so that the associated resource management module hands the resources and the operation to the activity management module for processing.
Step S105, the activity management module sets the activity of each associated resource of the virtual machine as the maximum value, puts each associated resource into an 'active zone', and performs activity deduction according to the corresponding activity convergence factor.
Taking the maximum value as 100 as an example, after receiving the notification of the associated resource association module, the activity management module sets the activity of the cloud hard disk associated with the virtual machine to 100 and puts the cloud hard disk into an "active area", and if the cloud hard disk is already in a "static area", the cloud hard disk is transferred to the "active area"; then, determining that the activity convergence factor corresponding to cloud hard disk deletion is 5 points per second by inquiring the activity convergence factor table, and performing activity deduction on the cloud hard disk along with the time lapse according to the activity convergence factor of 5 points per second; when the liveness decays to 0, the cloud hard disk is transferred to a "quiet zone". For the network card and the virtual machine snapshot associated with the virtual machine, the processing is also performed according to the above method, which is not described herein again.
Step S106, the thread management scheduling module periodically generates a fixed number of state synchronization threads, and distributes the state synchronization threads to an active area and a static area according to the proportion of a to b, wherein a is larger than b. Here, 10 state-synchronized threads are generated every 2 seconds and allocated to the "active zone" and the "quiet zone" in a ratio of 4.
Step S107, determine whether the virtual machine or its associated resource is still in the "active zone". If yes, jumping to step S108; if not, it jumps to step S112.
Step S108, the thread management scheduling module controls the state synchronization thread allocated to the active area to preferentially select the virtual machine or the related resource thereof for state synchronization processing.
When the virtual machine or the associated cloud hard disk, network card and virtual machine snapshot are located in the "active area", due to the high activity of the virtual machine or the associated cloud hard disk, network card and virtual machine snapshot, the thread management scheduling module controls the state synchronization thread allocated to the "active area" to preferentially select the resource with the high activity to perform state synchronization processing. Certainly, as the liveness of the virtual machine or the associated cloud hard disk, network card, or virtual machine snapshot decreases, the priority of the virtual machine or the associated cloud hard disk, network card, or virtual machine snapshot processed by the state synchronization thread decreases, and the update frequency decreases accordingly.
Step S109, if the virtual machine or its associated resource (i.e. associated cloud hard disk, network card or virtual machine snapshot) is selected by the state synchronization thread, the state updating module invokes an interface for inquiring details of the resource through the second communication module, and updates the state of the corresponding resource in the cloud management platform database.
Step S110, the mark management module marks an updated mark on the resource which is selected by the state synchronization thread to finish the state updating, and removes the mark after a first preset time.
Taking 5 seconds as an example of the first preset time, when the virtual machine or the related cloud hard disk, network card and virtual machine snapshot is selected by the state synchronization thread and the state update is completed, the mark management module marks an updated mark on the resource, and the mark is automatically removed after 5 seconds. When the mark is not released, the thread management scheduling module does not schedule the state synchronization thread to process the resource.
Step S111, if it is found in the state synchronization process that the virtual machine or its associated resource has entered the steady state (i.e. has been deleted), setting the activity of the resource to 0 in advance, and placing the resource into a "quiet zone".
In step S112, the thread management scheduling module controls the state synchronization thread allocated to the "quiet zone" to randomly select resources in the zone for performing state synchronization processing.
When the virtual machine or the associated cloud hard disk, network card and virtual machine snapshot enters a 'static area', the thread management scheduling module controls the state synchronization thread allocated to the 'static area' to randomly select resources in the area for processing. When the virtual machine or the associated cloud hard disk, network card and virtual machine snapshot enters a 'static area', the probability that the virtual machine or the associated cloud hard disk, network card and virtual machine snapshot is selected by the state synchronization thread is low.
Step S113, if the virtual machine or its associated resource (i.e. associated cloud hard disk, network card or virtual machine snapshot) is randomly selected by the state synchronization thread, the state updating module invokes an interface for inquiring details of the resource through the second communication module, so as to update the state of the corresponding resource in the cloud management platform database.
Step S114, the mark management module marks an updated mark on the resource which is selected by the state synchronization thread to finish the state updating, and removes the mark after a second preset time.
Taking 10 minutes as an example of the second preset time, when the virtual machine or the related cloud hard disk, network card and virtual machine snapshot is selected by the state synchronization thread and the state updating is completed, the mark management module marks an updated mark on the resource, and the mark is automatically removed after 10 minutes. When the mark is not released, the thread management scheduling module does not schedule the state synchronization thread to process the resource.
It should be noted that, this embodiment is only an example of the most common resource virtual machine in the cloud computing platform, and for other resources including, but not limited to, a cloud hard disk, a VPC (virtual private cloud), a subnet, a router, load balancing, and the like, the state synchronization thereof may also adopt the above method, and all of them are included in the protection scope of the present invention, and the specific process is not described herein again.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A cloud management platform state data synchronization method is characterized by comprising the following steps:
configuring an activity convergence factor for the operation of various influence state changes of various resources of cloud computing;
when any resource has an operation of influencing the state change, setting the activity of the resource to be the maximum value and dividing the resource into an 'active region', then reducing the activity of the resource according to the corresponding activity convergence factor, and dividing the resource into a 'static region' when the activity is 0, wherein the activity convergence factor corresponding to each resource is set according to the time-consuming time length of the state change operation, and the activity of each resource reduces the value corresponding to the activity convergence factor every preset period;
the method comprises the steps that a state synchronization thread pool periodically generates a fixed number of state synchronization threads, the state synchronization threads are distributed to an active area and a static area according to a preset proportion, the number of the state synchronization threads distributed to the active area is more than that of the static area, for any resource in the active area, after the state synchronization threads process the resource and complete state synchronization updating, a cloud management platform sets an updated mark for the resource and releases the updated mark after first preset time, for any resource in the static area, after the state synchronization threads process the resource and complete state synchronization updating, the cloud management platform sets an updated mark for the resource and releases the updated mark after second preset time, wherein the first preset time is less than the second preset time, and the state synchronization threads with the updated mark cannot be distributed to process again;
and each state synchronization thread calls a state query interface of a public cloud or a private cloud for polling aiming at one resource in the corresponding area to acquire the latest state of the resource and synchronously updates the database of the cloud management platform, wherein the state synchronization thread completes a resource state query task of the allocated area, completes the resource synchronization task when the resource state is changed, and automatically ends after the task is completed.
2. The cloud-managed platform state data synchronization method of claim 1, wherein when an operation occurs to any resource that affects a state change, the method further comprises:
determining the associated resources and corresponding operations of the resources by inquiring an associated resource rule table, setting the activity of the associated resources to be a maximum value, and dividing the associated resources into 'active zones';
and deducting the activity of the associated resources according to the corresponding activity convergence factors, and dividing the associated resources into 'quiet zones' when the activity of the associated resources is 0.
3. The cloud management platform state data synchronization method of claim 2, wherein for any resource in the "quiet zone", when the resource has an operation that affects the state change, or when the resource associated with the resource has an operation that affects the state change, the resource is transferred from the "quiet zone" back to the "active zone", and the activity is reset to the maximum value.
4. The cloud management platform state data synchronization method of claim 1, wherein in an "active zone", the allocated state synchronization thread preferentially processes resources with higher liveness; in the "quiet zone," the assigned state synchronization thread randomly selects resources to process.
5. The cloud management platform state data synchronization method according to claim 1, wherein when the number of resources in the "active zone" exceeds a preset threshold, the state synchronization thread pool adjusts the state synchronization thread allocation proportion, so that the number of state synchronization threads allocated to the "active zone" is increased, and the number of state synchronization threads allocated to the "quiet zone" is decreased.
6. The method for synchronizing the state data of the cloud management platform according to claim 1, wherein for any resource in the "active zone", when the state synchronization thread processes the state synchronization of the resource, if the resource state on the underlying private cloud or public cloud is found to have entered a steady state, the activity of the resource is directly set to 0, and the resource is put into the "quiescent zone" in advance.
7. A cloud management platform state data synchronization system is characterized by comprising a parameter configuration module, a first communication module, an activity management module, a thread management scheduling module, a state updating module and a second communication module;
the parameter configuration module is used for configuring activity convergence factors for various operation influencing state changes of various resources of cloud computing by an administrator of the cloud management platform;
the first communication module is used for communicating with other modules of the cloud management platform, informing the first communication module when other modules of the cloud management platform generate operation influencing state change, and informing the activity management module of the activity of the processing resources by the first communication module;
the activity management module is used for managing the increase and decrease of the activity of the resource according to the method provided by any one of claims 1 to 6 and realizing the conversion of the resource between an 'active area' and a 'static area' according to the size of the activity;
the thread management scheduling module is used for constructing a state synchronization thread pool, controlling the state synchronization thread pool to periodically generate a fixed number of state synchronization threads, and allocating the state synchronization threads to an active area and a static area according to a preset proportion, wherein the number of the state synchronization threads allocated to the active area is more than that of the static area;
the state updating module is used by the state synchronization thread and is used for calling a state query interface of a public cloud or a private cloud, acquiring the latest state of the resource and updating the latest state of the resource in a database of the cloud management platform synchronously;
the second communication module is used for communicating with a bottom resource platform and is connected with the state updating module, so that the state synchronization thread can obtain the latest state of the resource from the bottom resource platform through the second communication module.
8. The cloud management platform state data synchronization system of claim 7, further comprising an associated resource management module for identifying associated resources of a resource and corresponding operations through an associated resource rule table, and handing the associated resources to the activity management module for activity processing.
9. The cloud management platform state data synchronization system of claim 7, further comprising a flag management module for setting an updated flag for a resource that completes a state synchronization update, wherein the updated flag is released after a first preset time for a resource in an "active zone" and the updated flag is released after a second preset time for a resource in a "quiet zone"; the first preset time is less than the second preset time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010999792.3A CN112182100B (en) | 2020-09-22 | 2020-09-22 | Cloud management platform state data synchronization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010999792.3A CN112182100B (en) | 2020-09-22 | 2020-09-22 | Cloud management platform state data synchronization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112182100A CN112182100A (en) | 2021-01-05 |
| CN112182100B true CN112182100B (en) | 2022-11-08 |
Family
ID=73956541
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010999792.3A Active CN112182100B (en) | 2020-09-22 | 2020-09-22 | Cloud management platform state data synchronization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112182100B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225722A (en) * | 2021-04-20 | 2022-10-21 | 中兴通讯股份有限公司 | Computing resource notification method and device, storage medium and electronic device |
| CN115878310A (en) * | 2021-09-30 | 2023-03-31 | 中兴通讯股份有限公司 | Virtual resource information synchronization method, orchestrator, electronic device, and readable medium |
| CN114418771B (en) * | 2022-03-28 | 2022-08-09 | 深圳希施玛数据科技有限公司 | Data transmission method and related device |
| CN115801833A (en) * | 2022-11-16 | 2023-03-14 | 浙江九州云信息科技有限公司 | Enterprise-level public cloud resource management method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101155018A (en) * | 2006-09-28 | 2008-04-02 | 华为技术有限公司 | Data synchronization method and its implementing device and system |
| CN102202072A (en) * | 2010-03-23 | 2011-09-28 | 盛霆信息技术(上海)有限公司 | Unidirectional synchronization method of internet website data |
| CN106933534A (en) * | 2015-12-31 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization and device |
| CN108282501A (en) * | 2017-01-05 | 2018-07-13 | 阿里巴巴集团控股有限公司 | A kind of Cloud Server resource information synchronous method, device and system |
-
2020
- 2020-09-22 CN CN202010999792.3A patent/CN112182100B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101155018A (en) * | 2006-09-28 | 2008-04-02 | 华为技术有限公司 | Data synchronization method and its implementing device and system |
| CN102202072A (en) * | 2010-03-23 | 2011-09-28 | 盛霆信息技术(上海)有限公司 | Unidirectional synchronization method of internet website data |
| CN106933534A (en) * | 2015-12-31 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization and device |
| CN108282501A (en) * | 2017-01-05 | 2018-07-13 | 阿里巴巴集团控股有限公司 | A kind of Cloud Server resource information synchronous method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112182100A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112182100B (en) | Cloud management platform state data synchronization method and system | |
| CN112867050B (en) | UPF network element management method and system | |
| CN109783218B (en) | Kubernetes container cluster-based time-associated container scheduling method | |
| CN111385114B (en) | VNF service instantiation method and device | |
| CN107547596B (en) | Cloud platform control method and device based on Docker | |
| US20130160014A1 (en) | Computer system and control method for computer system | |
| CN109968352B (en) | Robot control method, robot and device with storage function | |
| CN103593229B (en) | Integrated and United Dispatching framework and the dispatching method of isomery cloud operating system | |
| US20160239322A1 (en) | Computer system and control method for computer system | |
| CN101227416A (en) | Method for distributing link bandwidth in communication network | |
| CN111427670A (en) | Task scheduling method and system | |
| US20210216353A1 (en) | Host Upgrade Method and Device | |
| US20110106934A1 (en) | Method and apparatus for controlling flow of management tasks to management system databases | |
| CN103366022A (en) | Information processing system and processing method for use therewith | |
| CN112437129B (en) | Cluster management method and cluster management device | |
| CN102694873A (en) | Address pool distribution system and method | |
| CN112214288B (en) | Pod scheduling method, device, equipment and medium based on Kubernetes cluster | |
| CN113032102A (en) | Resource rescheduling method, device, equipment and medium | |
| CN114389955A (en) | Embedded platform heterogeneous resource pooling management method | |
| CN108073408A (en) | The method for updating system and device of self-aided terminal | |
| CN106549786B (en) | Configuration method and device for service provisioning resources | |
| CN113849295A (en) | Model training method and device and computer readable storage medium | |
| CN107547622B (en) | Resource adjusting method and device | |
| CN114706663A (en) | Computing resource scheduling method, medium and computing device | |
| CN114579298A (en) | Resource management method, resource manager, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |