CN116627658A - Yun Bianduan load balancing method and system - Google Patents
Yun Bianduan load balancing method and system Download PDFInfo
- Publication number
- CN116627658A CN116627658A CN202310778478.6A CN202310778478A CN116627658A CN 116627658 A CN116627658 A CN 116627658A CN 202310778478 A CN202310778478 A CN 202310778478A CN 116627658 A CN116627658 A CN 116627658A
- Authority
- CN
- China
- Prior art keywords
- node
- data
- cloud
- standby
- migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000005012 migration Effects 0.000 claims abstract description 89
- 238000013508 migration Methods 0.000 claims abstract description 89
- 230000010076 replication Effects 0.000 claims abstract description 65
- 230000007246 mechanism Effects 0.000 claims abstract description 27
- 230000000977 initiatory effect Effects 0.000 claims abstract description 7
- 238000003860 storage Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 7
- 230000004048 modification Effects 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 3
- 230000001419 dependent effect Effects 0.000 claims description 3
- 230000002085 persistent effect Effects 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 238000007726 management method Methods 0.000 abstract description 23
- 238000013523 data management Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 6
- 210000004899 c-terminal region Anatomy 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a cloud side load balancing method and a cloud side load balancing system, which belong to the technical field of database data management, and the technical problem to be solved by the invention is uneven load pressure of cloud nodes, and the adopted technical scheme is as follows: and (3) establishing a relation: initiating data uploading by the side end, and establishing a data uploading relationship with the cloud end, namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters; selecting a main node and a standby node; and (3) the main node migration: when the side end migrates the own main node, the side end firstly changes the own system table to establish a metadata copying relation with the new main node, and disconnects the metadata copying relation with the original main node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node; and (5) preparing node migration.
Description
Technical Field
The invention relates to the technical field of database data management, in particular to a cloud edge load balancing method and a cloud edge load balancing system.
Background
In practical production and application, there are many products of cloud-edge architecture, and data collected by the edge is uploaded to the cloud for long-term storage or data analysis by the cloud. In the process of uploading data, data replication management needs to be performed on the data, such as which cloud node the data of the local database is uploaded to, and then, which cloud nodes are selected to backup the data, and the like.
Generally, user data selects which cloud node to upload to, and the data copy management system decides which cloud node is selected as a backup node, so as to provide an optimal scheme under the current situation. The user does not care about the distribution condition of own data in the cloud node. However, in some special scenarios, users with different requirements desire to be able to control the uploading and backing up of data to a specific cloud node. If the system is preferably concerned about an index of low network delay so that all user data uploading backup nodes are selected to the same area, it is desirable for a user who is concerned about data integrity to manually select nodes of other areas as part of backup or uploading data nodes. For example, when some nodes are frequently used as data uploading nodes under the system calculation rule, the network traffic pressure of the nodes is very high, but the pressure of other nodes is very low, and a part of users want to manually change the cloud target nodes so as to reduce the network pressure; for example, when a node particularly accords with a priority rule selected by the system, the priority is extremely high, most users can upload data to the node, so that the storage capacity of the node is far greater than that of other nodes, and the users do not want to change the target node when considering the subsequent uploading data, so that the other nodes need to be selected as the target nodes independently, and the like.
As a system administrator, the whole cloud node is expected to work as evenly as possible, the situation that the pressure of one node is particularly high and the load of some nodes is very small is not expected to occur, so that the system can stably operate, the service life of software and hardware is prolonged, and the like.
Therefore, how to autonomously change the cloud target nodes for uploading and backing up data when the cloud nodes have uneven load, so that cloud cluster load balancing is a technical problem to be solved urgently at present.
Disclosure of Invention
The technical task of the invention is to provide a cloud side load balancing method and a cloud side load balancing system, which are used for solving the problem of balancing the cloud cluster load by independently changing cloud target nodes for uploading and backing up data when the cloud nodes are uneven in load.
The cloud side load balancing method is initiated by the side, and cloud cluster load balancing is achieved through a migration master node and a migration standby node; the method comprises the following steps:
and (3) establishing a relation: initiating data uploading by an Edge (E end Edge), and establishing a data uploading relationship with a Cloud (C end Cloud), namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters;
selecting a main node and a standby node: after the side initiates the establishment of the metadata replication relationship, the side and the cloud end jointly generate data replication management data, an optimal node is calculated according to the network delay, the cloud node capacity and the user quantity and is used as a main node, and a suboptimal node is used as a standby node;
and (3) the main node migration: when the side end migrates the own main node, the side end firstly changes the own system table to establish a metadata copying relation with the new main node, and disconnects the metadata copying relation with the original main node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node;
and (5) standby node migration: when the side end migrates the own standby node, the side end modifies the system table, updates the own standby node, synchronizes to the cloud end through data playback, disconnects the main node from the old standby node, establishes the connection between the main node and the new standby node, and realizes the migration of the standby node.
Preferably, the standby node migration is specifically as follows:
when the cloud node to be migrated is a standby node of the edge, the edge executes an SQL sentence, and the SQL specifies the information of the old standby node to be migrated and the information of the target node;
executing the SQL modified side own system table information; the system table information comprises node information to be migrated and target node information to be migrated;
because the metadata copying relation between the edge end and the main node is not disconnected, the channel can enable a data playback mechanism to be effective; the changed system table information of the edge is synchronized to the cloud through a data playback mechanism;
when the cloud host node detects that the corresponding side end modifies the own standby node, disconnecting the own standby node from the old cloud standby node, and establishing connection with the new standby node;
and synchronizing user data to the new standby node by the background to realize the migration of the standby node.
Preferably, the master node migration is specifically as follows:
when the cloud node to be migrated is a master node of the edge, the edge executes an SQL sentence, and the SQL specifies the information of the old master node to be migrated and the information of the target node;
because the migration master node needs to disconnect the metadata copy relationship between the edge and the master node, so that a data playback mechanism is not available, the metadata copy relationship between the edge and the new master node needs to be reestablished, the connection between the old master node and the new master node is disconnected, and when the old master node senses that the connection between the old master node and the edge is disconnected, the connection between the old master node and the standby node is automatically disconnected;
after the metadata replication relationship is established, the new master node and the backup node establish a master-backup replication relationship;
according to the old master node information recorded in the system table, a corresponding node is found, data migration connection is established, and migration of historical data is completed, a new master node has all data uploaded by an edge end user and serves as the new master node for uploading data by the edge end, and then a master-slave connection between cloud ends is established with the slave nodes, so that migration of the master node is realized.
Preferably, the data playback mechanism is a mechanism for synchronizing two cluster data replication management data of a side and a cloud; metadata replication relationship is established between two clusters of the side and the cloud end, so that data replication at a table level and a database level is realized, and the method specifically comprises the following steps:
modifying the system table of the user at the side to realize modification of the data copy management data, and enabling the modified system table information to pass through the cloud end through a data playback mechanism, synchronously changing the cloud system table information and further updating the cloud data copy management data;
and similarly, the cloud operation is performed to achieve synchronization to the edge.
More preferably, a database cluster and a plurality of services are deployed in the cloud for the persistent storage of data and the prediction analysis of cloud computers;
the side end is provided with a single-node database and a plurality of dependent services for collecting data and locally storing;
the main node is a cloud node, the edge end selects the cloud node for uploading data, and the data mountain at the edge reaches the cloud main node and is backed up to the standby node;
the standby node refers to a node of the cloud, namely, a node of the main node backing up data in the cloud.
The cloud side load balancing system comprises an establishing unit, a selecting unit, a main node migration unit and a standby node migration unit;
the establishing unit is used for initiating data uploading at an Edge (E end Edge), and establishing a data uploading relationship with a Cloud (C end Cloud), namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters;
the selecting unit is used for generating data replication management data together by the side end and the cloud end after the side end initiates the establishment of the metadata replication relation, and calculating according to the network delay, the cloud node capacity and the number of users to obtain an optimal node serving as a main node and a suboptimal node serving as a standby node;
the master node migration unit is used for firstly changing a system table of the side end to establish a metadata replication relationship with a new master node when the side end migrates the master node of the side end, and disconnecting the metadata replication relationship with the original master node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node;
the standby node migration unit is used for modifying the system table by the side end when the side end migrates the standby node, updating the standby node, synchronizing to the cloud end through data playback, disconnecting the main node from the old standby node, establishing the connection between the main node and the new standby node, and realizing the migration of the standby node.
Preferably, the standby node migration unit comprises a SQl statement execution module I, an information modification module, an information synchronization module, a new connection establishment module and a standby node migration module;
the SQl statement execution module is used for executing an SQL statement by the edge when the cloud node to be migrated is the standby node of the edge, wherein the SQL statement designates the information of the old standby node to be migrated and the information of the target node;
the information changing module is used for executing the SQL to change the system table information of the side; the system table information comprises node information to be migrated and target node information to be migrated;
the information synchronization module is used for synchronizing the changed system table information of the edge to the cloud through a data playback mechanism;
the new connection establishment module is used for disconnecting the cloud host node from the old cloud standby node and establishing connection with the new standby node when the cloud host node detects that the corresponding side end modifies the standby node;
and the standby node migration module is used for synchronizing user data to the new standby node through the background so as to realize the migration of the standby node.
More preferably, the master node migration unit comprises a SQl statement execution module II, a connection reestablishing module, a replication relationship establishing module and a master node migration module;
the SQl statement execution module II is used for executing an SQL statement by the edge when the cloud node to be migrated is the main node of the edge, wherein the SQL statement designates the information of the old main node to be migrated and the information of the target node;
the connection reestablishing module is used for reestablishing the metadata copying relation between the side end and the new main node, disconnecting the connection between the side end and the old main node, and automatically disconnecting the connection between the side end and the standby node after the old main node senses the disconnection of the side end;
the replication relationship establishing module is used for waiting for the establishment of the metadata replication relationship, and establishing a master-slave replication relationship between the new master node and the slave node;
the master node migration module is used for finding out corresponding nodes according to old master node information recorded in the system table, establishing data migration connection and completing migration of historical data, a new master node has all data uploaded by an edge end user, serves as the new master node for uploading data by the edge end, and establishes master-slave connection between cloud ends with the slave nodes to realize migration of the master nodes.
An electronic device, comprising: a memory and at least one processor;
wherein the memory has a computer program stored thereon;
the at least one processor executes the computer program stored by the memory, causing the at least one processor to perform the Yun Bianduan load balancing method as described above.
A computer readable storage medium having stored therein a computer program executable by a processor to implement a Yun Bianduan load balancing method as described above.
The cloud edge load balancing method and system have the following advantages:
the cloud end and the edge end establish a data replication relationship, the edge end uploads data to a plurality of nodes of the cloud end, and the edge end can manually change a main node and a standby node of the cloud end so as to relieve the high load condition of a certain node of the cloud end, so that the clusters share the data flow pressure and the load of the whole system is balanced;
according to the cloud end collaborative data management system, manual load balancing in the cloud end collaborative data management system is achieved, and when the cloud nodes are uneven in load, cloud target nodes for uploading and backing up data can be manually changed, so that cloud cluster load balancing is achieved; when the load pressure of a certain node is overlarge, migrating user data on the node and reselecting other cloud nodes for the user as new target nodes; when the load pressure of a certain node is too small, data on one or more other nodes are migrated to the node and taken as a new target node;
the cloud node load pressure non-uniformity problem is solved. Migrating users and data on the high-load nodes to other nodes with small load pressure, so that the cloud cluster is balanced in load;
the E end only is responsible for modifying own system table, the process of disconnecting the main node and the old standby node occurs at the C end, the process of establishing the main standby copy relationship between the main node and the new standby node also occurs at the C end, the E end sends out instructions, and the connection between the nodes and the migration of data all occur at the C end;
fifthly, the E end of the invention needs to initiate a metadata copying relation with a new main node, and then disconnects the old main node from the standby node and the E end from the old main node; and then migrating the data of the old master node to the new master node, and establishing the master-slave copy connection between the new master node and the slave node.
Drawings
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of a cloud end load balancing method;
FIG. 2 is a schematic diagram of a standby node migration;
fig. 3 is a schematic diagram of a master node migration.
Detailed Description
The cloud edge load balancing method and the cloud edge load balancing system are described in detail below with reference to the accompanying drawings and specific embodiments.
Example 1:
as shown in fig. 1, the present embodiment provides a cloud edge load balancing method, which is initiated by an edge, and balances the cloud cluster load by migrating a master node and a migration standby node; the method comprises the following steps:
s1, establishing a relation: initiating data uploading by an Edge (E end Edge), and establishing a data uploading relationship with a Cloud (C end Cloud), namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters;
s2, selecting a main node and a standby node: after the side initiates the establishment of the metadata replication relationship, the side and the cloud end jointly generate data replication management data, an optimal node is calculated according to the network delay, the cloud node capacity and the user quantity and is used as a main node, and a suboptimal node is used as a standby node;
s3, the master node migrates: when the side end migrates the own main node, the side end firstly changes the own system table to establish a metadata copying relation with the new main node, and disconnects the metadata copying relation with the original main node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node;
s4, standby node migration: when the side end migrates the own standby node, the side end modifies the system table, updates the own standby node, synchronizes to the cloud end through data playback, disconnects the main node from the old standby node, establishes the connection between the main node and the new standby node, and realizes the migration of the standby node.
As shown in fig. 2, the standby node migration in step S4 of this embodiment is specifically as follows:
s401, when a cloud node to be migrated is a standby node of an edge, the edge executes an SQL sentence, and the SQL specifies information of an old standby node to be migrated and information of a target node;
s402, executing the SQL modified side to modify own system table information; the system table information comprises node information to be migrated and target node information to be migrated;
s403, as the metadata copying relation between the edge end and the main node is not disconnected, the channel can enable a data playback mechanism to be effective; the changed system table information of the edge is synchronized to the cloud through a data playback mechanism;
s404, when the cloud host node detects that the corresponding side end modifies the own standby node, disconnecting the own standby node from the old cloud standby node, and establishing connection with the new standby node;
s405, synchronizing user data to the new standby node by the background to realize the migration of the standby node.
As shown in fig. 3, the migration of the master node in step S3 of this embodiment is specifically as follows:
s301, when a cloud node to be migrated is a master node of an edge, the edge executes an SQL sentence, and the SQL specifies information of an old master node to be migrated and information of a target node;
s302, because the migration master node needs to disconnect the metadata copying relation between the side end and the master node, so that a data playback mechanism is not available, the metadata copying relation between the side end and the new master node needs to be reestablished, the connection between the side end and the old master node is disconnected, and when the old master node senses that the connection between the side end and the old master node is disconnected, the connection between the side end and the standby node is automatically disconnected;
s303, after the metadata replication relationship is established, the new master node and the backup node establish a master-backup replication relationship;
s304, finding out corresponding nodes according to old main node information recorded in a system table, establishing data migration connection and completing migration of historical data, enabling a new main node to have all data uploaded by an edge end user, enabling the new main node to serve as the new main node to upload data, and then establishing main and standby connection between cloud ends with standby nodes to realize migration of the main node.
The data playback mechanism in this embodiment is a mechanism for synchronizing two cluster data replication management data of a side and a cloud; metadata replication relationship is established between two clusters of the side and the cloud end, so that data replication at a table level and a database level is realized, and the method specifically comprises the following steps:
modifying the system table of the user at the side to realize modification of the data copy management data, and enabling the modified system table information to pass through the cloud end through a data playback mechanism, synchronously changing the cloud system table information and further updating the cloud data copy management data;
and similarly, the cloud operation is performed to achieve synchronization to the edge. Therefore, the connection established between the two clusters realizes the synchronization of the data replication management data, the interaction between the two clusters can be reduced, and the sql is not required to be sent to the opposite end for execution every time the system table is updated.
Wherein, data replication management: when the data is copied from the E-terminal to the C-terminal, some data for managing the copying behavior of the data, such as which C-terminal node the E-terminal is to copy to, the corresponding IP address and other key information, are also data, which we call the data to be copied, and the data exist in a system table. The data that the corresponding user really needs to upload is called user data, which is simply called data. The data is management data, i.e. user data, if not explicitly stated later in this document.
The Cloud (C-terminal Cloud) in the embodiment is provided with a database cluster and a plurality of services, and is used for the persistent storage of data and the prediction analysis of a Cloud computer;
a single-node database and a plurality of dependent services are deployed at the Edge (E end Edge) for collecting data and locally storing;
the main node is a cloud node, the edge end selects the cloud node for uploading data, and the data mountain at the edge reaches the cloud main node and is backed up to the standby node;
the standby node refers to a node of the cloud, namely a node of the main node backing up data in the cloud;
data copy management data: the data for managing the data copying behavior, such as key information of which C-terminal node the E-terminal is to copy to, the corresponding IP address, etc., is called data copy management data.
And (3) data playback: a synchronization mechanism is used to synchronize E, C data replication management data across.
Example 2:
as shown in fig. 1, this embodiment provides a cloud edge load balancing system, which includes an establishing unit, a selecting unit, a primary node migration unit and a standby node migration unit;
the establishing unit is used for initiating data uploading at an Edge (E end Edge), and establishing a data uploading relationship with a Cloud (C end Cloud), namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters;
the selecting unit is used for generating data replication management data together by the side end and the cloud end after the side end initiates the establishment of the metadata replication relation, and calculating according to the network delay, the cloud node capacity and the number of users to obtain an optimal node serving as a main node and a suboptimal node serving as a standby node;
the master node migration unit is used for firstly changing a system table of the side end to establish a metadata replication relationship with a new master node when the side end migrates the master node of the side end, and disconnecting the metadata replication relationship with the original master node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node;
the standby node migration unit is used for modifying the system table by the side end when the side end migrates the standby node, updating the standby node, synchronizing to the cloud end through data playback, disconnecting the main node from the old standby node, establishing the connection between the main node and the new standby node, and realizing the migration of the standby node.
As shown in fig. 2, the standby node migration unit in this embodiment includes a SQl statement execution module one, an information modification module, an information synchronization module, a new connection establishment module, and a standby node migration module;
the SQl statement execution module is used for executing an SQL statement by the edge when the cloud node to be migrated is the standby node of the edge, wherein the SQL statement designates the information of the old standby node to be migrated and the information of the target node;
the information changing module is used for executing the SQL to change the system table information of the side; the system table information comprises node information to be migrated and target node information to be migrated;
the information synchronization module is used for synchronizing the changed system table information of the edge to the cloud through a data playback mechanism;
the new connection establishment module is used for disconnecting the cloud host node from the old cloud standby node and establishing connection with the new standby node when the cloud host node detects that the corresponding side end modifies the standby node;
and the standby node migration module is used for synchronizing user data to the new standby node through the background so as to realize the migration of the standby node.
As shown in fig. 3, the master node migration unit in this embodiment includes a SQl statement execution module two, a connection reestablishing module, a copy relationship establishing module, and a master node migration module;
the SQl statement execution module II is used for executing an SQL statement by the edge when the cloud node to be migrated is the main node of the edge, wherein the SQL statement designates the information of the old main node to be migrated and the information of the target node;
the connection reestablishing module is used for reestablishing the metadata copying relation between the side end and the new main node, disconnecting the connection between the side end and the old main node, and automatically disconnecting the connection between the side end and the standby node after the old main node senses the disconnection of the side end;
the replication relationship establishing module is used for waiting for the establishment of the metadata replication relationship, and establishing a master-slave replication relationship between the new master node and the slave node;
the master node migration module is used for finding out corresponding nodes according to old master node information recorded in the system table, establishing data migration connection and completing migration of historical data, a new master node has all data uploaded by an edge end user, serves as the new master node for uploading data by the edge end, and establishes master-slave connection between cloud ends with the slave nodes to realize migration of the master nodes.
Example 3:
the embodiment also provides an electronic device, including: a memory and a processor;
wherein the memory stores computer-executable instructions;
and the processor executes the computer-executed instructions stored in the memory, so that the processor executes the cloud side load balancing method in any embodiment of the invention.
The processor may be a Central Processing Unit (CPU), but may also be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may be used to store computer programs and/or modules, and the processor implements various functions of the electronic device by running or executing the computer programs and/or modules stored in the memory, and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the terminal, etc. The memory may also include high-speed random access memory, but may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, memory card only (SMC), secure Digital (SD) card, flash memory card, at least one disk storage period, flash memory device, or other volatile solid state memory device.
Example 4:
the embodiment of the invention also provides a computer readable storage medium, wherein a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the cloud side load balancing method in any embodiment of the invention. Specifically, a system or apparatus provided with a storage medium on which a software program code realizing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or apparatus may be caused to read out and execute the program code stored in the storage medium.
In this case, the program code itself read from the storage medium may realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code form part of the present invention.
Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RYM, DVD-RWs, DVD+RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer by a communication network.
Further, it should be apparent that the functions of any of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform part or all of the actual operations based on the instructions of the program code.
Further, it is understood that the program code read out by the storage medium is written into a memory provided in an expansion board inserted into a computer or into a memory provided in an expansion unit connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion unit is caused to perform part and all of actual operations based on instructions of the program code, thereby realizing the functions of any of the above embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (10)
1. The cloud side load balancing method is characterized in that the method is initiated by a side, and cloud cluster load balancing is achieved in a mode of migrating a main node and a migration standby node; the method comprises the following steps:
and (3) establishing a relation: initiating data uploading by the side end, and establishing a data uploading relationship with the cloud end, namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters;
selecting a main node and a standby node: after the side initiates the establishment of the metadata replication relationship, the side and the cloud end jointly generate data replication management data, an optimal node is calculated according to the network delay, the cloud node capacity and the user quantity and is used as a main node, and a suboptimal node is used as a standby node;
and (3) the main node migration: when the side end migrates the own main node, the side end firstly changes the own system table to establish a metadata copying relation with the new main node, and disconnects the metadata copying relation with the original main node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node;
and (5) standby node migration: when the side end migrates the own standby node, the side end modifies the system table, updates the own standby node, synchronizes to the cloud end through data playback, disconnects the main node from the old standby node, establishes the connection between the main node and the new standby node, and realizes the migration of the standby node.
2. The Yun Bianduan load balancing method of claim 1, wherein the standby node migration is specifically as follows:
when the cloud node to be migrated is a standby node of the edge, the edge executes an SQL sentence, and the SQL specifies the information of the old standby node to be migrated and the information of the target node;
executing the SQL modified side own system table information; the system table information comprises node information to be migrated and target node information to be migrated;
the changed system table information of the edge is synchronized to the cloud through a data playback mechanism;
when the cloud host node detects that the corresponding side end modifies the own standby node, disconnecting the own standby node from the old cloud standby node, and establishing connection with the new standby node;
and synchronizing user data to the new standby node by the background to realize the migration of the standby node.
3. The Yun Bianduan load balancing method of claim 1, wherein the master node migration is specifically as follows:
when the cloud node to be migrated is a master node of the edge, the edge executes an SQL sentence, and the SQL specifies the information of the old master node to be migrated and the information of the target node;
reestablishing a metadata replication relationship between the edge and the new master node, disconnecting the connection between the old master node and the metadata replication relationship, and automatically disconnecting the connection between the old master node and the backup node after the old master node senses that the connection between the old master node and the edge is disconnected;
after the metadata replication relationship is established, the new master node and the backup node establish a master-backup replication relationship;
according to the old master node information recorded in the system table, a corresponding node is found, data migration connection is established, and migration of historical data is completed, a new master node has all data uploaded by an edge end user and serves as the new master node for uploading data by the edge end, and then a master-slave connection between cloud ends is established with the slave nodes, so that migration of the master node is realized.
4. The Yun Bianduan load balancing method according to claim 1, wherein the data playback mechanism is a mechanism for synchronizing two cluster data replication management data of a side and a cloud; metadata replication relationship is established between two clusters of the side and the cloud end, so that data replication at a table level and a database level is realized, and the method specifically comprises the following steps:
modifying the system table of the user at the side to realize modification of the data copy management data, and enabling the modified system table information to pass through the cloud end through a data playback mechanism, synchronously changing the cloud system table information and further updating the cloud data copy management data;
and similarly, the cloud operation is performed to achieve synchronization to the edge.
5. The Yun Bianduan load balancing method according to any one of claims 1-4, wherein a database cluster and a plurality of services are deployed in the cloud for persistent storage of data and cloud computer predictive analysis;
the side end is provided with a single-node database and a plurality of dependent services for collecting data and locally storing;
the main node is a cloud node, the edge end selects the cloud node for uploading data, and the data mountain at the edge reaches the cloud main node and is backed up to the standby node;
the standby node refers to a node of the cloud, namely, a node of the main node backing up data in the cloud.
6. The cloud side load balancing system is characterized by comprising an establishing unit, a selecting unit, a main node migration unit and a standby node migration unit;
the establishing unit is used for initiating data uploading at the side end, and establishing a data uploading relationship with the cloud end, namely a metadata copying relationship; the data playback mechanism relies on the metadata replication relationship to realize synchronization of data replication management data between the edge and cloud clusters;
the selecting unit is used for generating data replication management data together by the side end and the cloud end after the side end initiates the establishment of the metadata replication relation, and calculating according to the network delay, the cloud node capacity and the number of users to obtain an optimal node serving as a main node and a suboptimal node serving as a standby node;
the master node migration unit is used for firstly changing a system table of the side end to establish a metadata replication relationship with a new master node when the side end migrates the master node of the side end, and disconnecting the metadata replication relationship with the original master node; then the connection between the old main node and the standby node is disconnected, the new main node establishes a main-standby copy relationship with the standby node, and establishes a data migration channel with the old main node to migrate the historical data on the old main node;
the standby node migration unit is used for modifying the system table by the side end when the side end migrates the standby node, updating the standby node, synchronizing to the cloud end through data playback, disconnecting the main node from the old standby node, establishing the connection between the main node and the new standby node, and realizing the migration of the standby node.
7. The Yun Bianduan load balancing system according to claim 6, wherein the standby node migration unit comprises a SQl statement execution module one, an information modification module, an information synchronization module, a new connection establishment module, and a standby node migration module;
the SQl statement execution module is used for executing an SQL statement by the edge when the cloud node to be migrated is the standby node of the edge, wherein the SQL statement designates the information of the old standby node to be migrated and the information of the target node;
the information changing module is used for executing the SQL to change the system table information of the side; the system table information comprises node information to be migrated and target node information to be migrated;
the information synchronization module is used for synchronizing the changed system table information of the edge to the cloud through a data playback mechanism;
the new connection establishment module is used for disconnecting the cloud host node from the old cloud standby node and establishing connection with the new standby node when the cloud host node detects that the corresponding side end modifies the standby node;
and the standby node migration module is used for synchronizing user data to the new standby node through the background so as to realize the migration of the standby node.
8. The Yun Bianduan load balancing system according to claim 6 or 7, wherein the master node migration unit comprises a SQl statement execution module two, a connection reestablishing module, a replication relationship establishing module and a master node migration module;
the SQl statement execution module II is used for executing an SQL statement by the edge when the cloud node to be migrated is the main node of the edge, wherein the SQL statement designates the information of the old main node to be migrated and the information of the target node;
the connection reestablishing module is used for reestablishing the metadata copying relation between the side end and the new main node, disconnecting the connection between the side end and the old main node, and automatically disconnecting the connection between the side end and the standby node after the old main node senses the disconnection of the side end;
the replication relationship establishing module is used for waiting for the establishment of the metadata replication relationship, and establishing a master-slave replication relationship between the new master node and the slave node;
the master node migration module is used for finding out corresponding nodes according to old master node information recorded in the system table, establishing data migration connection and completing migration of historical data, a new master node has all data uploaded by an edge end user, serves as the new master node for uploading data by the edge end, and establishes master-slave connection between cloud ends with the slave nodes to realize migration of the master nodes.
9. An electronic device, comprising: a memory and at least one processor;
wherein the memory has a computer program stored thereon;
the at least one processor executing the computer program stored by the memory causes the at least one processor to perform the Yun Bianduan load balancing method of any one of claims 1 to 5.
10. A computer readable storage medium having stored therein a computer program executable by a processor to implement the method of Yun Bianduan load balancing of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310778478.6A CN116627658A (en) | 2023-06-29 | 2023-06-29 | Yun Bianduan load balancing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310778478.6A CN116627658A (en) | 2023-06-29 | 2023-06-29 | Yun Bianduan load balancing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116627658A true CN116627658A (en) | 2023-08-22 |
Family
ID=87613597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310778478.6A Pending CN116627658A (en) | 2023-06-29 | 2023-06-29 | Yun Bianduan load balancing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116627658A (en) |
-
2023
- 2023-06-29 CN CN202310778478.6A patent/CN116627658A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10164894B2 (en) | Buffered subscriber tables for maintaining a consistent network state | |
| US9906598B1 (en) | Distributed data storage controller | |
| US10169163B2 (en) | Managing backup operations from a client system to a primary server and secondary server | |
| US8090917B2 (en) | Managing storage and migration of backup data | |
| CN102594849B (en) | Data backup and recovery method and device, virtual machine snapshot deleting and rollback method and device | |
| US20070061379A1 (en) | Method and apparatus for sequencing transactions globally in a distributed database cluster | |
| US8930364B1 (en) | Intelligent data integration | |
| US9031906B2 (en) | Method of managing data in asymmetric cluster file system | |
| WO2017152736A1 (en) | Hadoop distributed file system hdfs management method, device and system | |
| CN103902410A (en) | Data backup acceleration method for cloud storage system | |
| US10452680B1 (en) | Catch-up replication with log peer | |
| US20120324436A1 (en) | Method of updating versioned software using a shared cache | |
| CN110928728A (en) | Virtual machine copying and switching method and system based on snapshot | |
| CN113010496A (en) | Data migration method, device, equipment and storage medium | |
| CN105593839A (en) | Distributed disaster recovery file sync server system | |
| CN109298977A (en) | A kind of virtual machine business quick recovery method, system, device and storage medium | |
| WO2018010603A1 (en) | Storage mode upgrade method, apparatus and system based on video cloud storage system | |
| CN115982101B (en) | Machine room data migration method and device based on multi-machine room copy placement strategy | |
| WO2020060617A1 (en) | Disturbance-free partitioning and migration of data from one storage account to another storage account | |
| CN116627658A (en) | Yun Bianduan load balancing method and system | |
| CN114415977B (en) | Method for accessing storage pool and distributed storage system | |
| JP6845470B2 (en) | Storage system | |
| CN111522688B (en) | Data backup method and device for distributed system | |
| CN111142921A (en) | Software upgrading method and device | |
| KR101748912B1 (en) | Data storage system in distributed storage environment and cluster management method for upgrading, scaling out and scaling in cluster include in the data storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |