CN118260265A - Cluster migration method, device, computer equipment and storage medium - Google Patents
Cluster migration method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN118260265A CN118260265A CN202310031308.1A CN202310031308A CN118260265A CN 118260265 A CN118260265 A CN 118260265A CN 202310031308 A CN202310031308 A CN 202310031308A CN 118260265 A CN118260265 A CN 118260265A
- Authority
- CN
- China
- Prior art keywords
- cluster
- double
- data
- migration
- function switch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013508 migration Methods 0.000 title claims abstract description 150
- 230000005012 migration Effects 0.000 title claims abstract description 150
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000001105 regulatory effect Effects 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 130
- 239000012634 fragment Substances 0.000 claims description 20
- 230000005856 abnormality Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012217 deletion Methods 0.000 description 8
- 230000037430 deletion Effects 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 230000006854 communication Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000009977 dual effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000007175 bidirectional communication Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a cluster migration method, a cluster migration device, computer equipment and a storage medium, wherein the cluster migration method comprises the following steps: acquiring cluster migration requests for a first cluster and a second cluster; based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring the associated application of the first cluster; migrating a plurality of services to a second cluster when a flow control switch preset in the associated application is in a first state; after the migration of the plurality of services is completed, regulating the flow control switch to a second state, and starting a double-read function switch and a double-write function switch which are preset in the related application; and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster. The embodiment of the application can ensure that the situation of data loss and data missing can not occur in the migration process of the cluster, and realize smooth migration of the cluster under the condition of continuous service.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a cluster migration method, a cluster migration device, a computer device, and a storage medium.
Background
The remote dictionary service (Remote Dictionary Server, redis) is an open-source log-type, key-Value database written in ANSI C language, supported by network, memory-based and persistent, and the Redis cluster is a distributed server cluster composed of a plurality of Redis node clusters. Redis clusters include a sentinel mode Cluster and a Cluster mode Cluster, and because the sentinel mode Cluster is difficult to support online capacity expansion, the existing sentinel mode Cluster needs to be migrated to the Cluster mode Cluster. However, existing methods cannot smoothly migrate the sentinel mode Cluster to the Cluster mode Cluster without stopping service.
Disclosure of Invention
The embodiment of the application provides a cluster migration method, a cluster migration device, computer equipment and a storage medium, which can ensure that the situation of data loss and data missing can not occur in the migration process of a cluster, and realize smooth migration of the cluster under the condition of continuous service.
In a first aspect, the present application provides a cluster migration method, where the cluster migration method includes:
Acquiring cluster migration requests for a first cluster and a second cluster;
Based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services;
migrating the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, wherein the associated application preferentially performs data operation on the first cluster in the first state;
After the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster;
and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
In some embodiments of the application, the determining whether the first cluster successfully migrates to the second cluster comprises:
Acquiring cluster operation logs of the associated application, wherein the cluster operation logs are operation logs of the associated application to the first cluster and the second cluster after the double-read function switch and the double-write function switch are turned on;
Based on the cluster operation log, it is determined whether the first cluster successfully migrated to the second cluster.
In some embodiments of the present application, the cluster operation log includes a data write log and a data read log, and the determining whether the first cluster is successfully migrated to the second cluster based on the cluster operation log includes:
determining whether the second cluster has data writing abnormality based on the data writing log;
Determining, when the second cluster does not have a data writing abnormality, whether target data exists in the first cluster based on the data reading log, the target data being data that exists in the first cluster but does not exist in the second cluster;
and when the target data does not exist in the first cluster, determining that the first cluster is successfully migrated to the second cluster.
In some embodiments of the present application, after the determining whether the target data exists in the first cluster based on the data read log, the method further includes:
And when target data exists in the first cluster, determining that the first cluster is not successfully migrated to the second cluster.
In some embodiments of the present application, after the determining that the first cluster has not successfully migrated to the second cluster, the method further includes:
determining whether a target service exists in the first cluster, wherein the target service is a service which is not migrated to the second cluster in the plurality of services;
migrating the target service to the second cluster when the target service exists;
Synchronizing data of the first cluster to the second cluster.
In some embodiments of the present application, after the migration of the plurality of services is completed, the method further includes:
Starting a double-deleting function switch preset in the associated application, wherein the double-deleting function switch is used for carrying out data double-deleting operation on the first cluster and the second cluster;
after the double read function switch and the double write function switch are closed, the method further comprises:
And closing the double-delete function switch.
In some embodiments of the present application, before the obtaining the cluster migration request for the first cluster and the second cluster, the method further includes:
Creating the second cluster, wherein the second cluster comprises a plurality of fragments, the number of the fragments is larger than a preset number threshold, and the memory of each fragment in the plurality of fragments is smaller than a preset memory threshold.
In a second aspect, the present application provides a cluster migration apparatus, including:
a request acquisition unit configured to acquire a cluster migration request for a first cluster and a second cluster;
the first synchronization unit is used for synchronizing the data of the first cluster to the second cluster based on the cluster migration request, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services;
A service migration unit, configured to migrate, in a first state of a flow control switch preset in the association application, the plurality of services to the second cluster, where in the first state, the association application preferentially performs data operation on the first cluster;
The switch starting unit is used for adjusting the flow control switch to a second state after the migration of the plurality of services is completed, starting a double-read function switch and a double-write function switch which are preset in the associated application, wherein the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster;
and the migration determining unit is used for determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
In some embodiments of the present application, the migration determining unit is specifically configured to:
Acquiring cluster operation logs of the associated application, wherein the cluster operation logs are operation logs of the associated application to the first cluster and the second cluster after the double-read function switch and the double-write function switch are turned on;
Based on the cluster operation log, it is determined whether the first cluster successfully migrated to the second cluster.
In some embodiments of the present application, the cluster operation log includes a data write log and a data read log, and the migration determining unit is specifically further configured to:
determining whether the second cluster has data writing abnormality based on the data writing log;
Determining, when the second cluster does not have a data writing abnormality, whether target data exists in the first cluster based on the data reading log, the target data being data that exists in the first cluster but does not exist in the second cluster;
and when the target data does not exist in the first cluster, determining that the first cluster is successfully migrated to the second cluster.
In some embodiments of the present application, the migration determination unit is specifically further configured to:
And when target data exists in the first cluster, determining that the first cluster is not successfully migrated to the second cluster.
In some embodiments of the present application, the cluster service migration apparatus further includes:
A target determining unit, configured to determine whether a target service exists in the first cluster, where the target service is a service that is not migrated to the second cluster among the plurality of services;
a target migration unit, configured to migrate, when a target service exists, the target service to the second cluster;
And the second synchronization unit is used for synchronizing the data of the first cluster to the second cluster.
In some embodiments of the application, the switch-on unit is further configured to:
And starting a double-deleting function switch preset in the associated application, wherein the double-deleting function switch is used for carrying out data double-deleting operation on the first cluster and the second cluster.
In some embodiments of the application, the migration determination unit is further configured to:
And closing the double-delete function switch.
In some embodiments of the present application, the cluster service migration apparatus further includes:
The cluster creating unit is used for creating the second cluster, the second cluster comprises a plurality of fragments, the number of the fragments is larger than a preset number threshold, and the memory of each fragment in the plurality of fragments is smaller than the preset memory threshold.
In a third aspect, the present application also provides a computer apparatus comprising:
One or more processors;
a memory; and
One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the cluster migration method of any one of the first aspects.
In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program, the computer program being loaded by a processor to perform the steps of the cluster migration method of any one of the first aspects.
The application migrates the clusters based on the flow control switch, the double-read function switch and the double-write function switch which are preset in the related application, and can smoothly migrate the first cluster to the second cluster under the condition of continuous service; after the migration of a plurality of services is completed, a double-write function switch is started, so that the situation that data are lost in the cluster migration process can be avoided; after the migration of the plurality of services is completed, the double-read function switch is started, and when the data reading operation is carried out in the cluster migration process by the associated application, the condition of missing data cannot occur, so that the smooth migration of the clusters under the condition of continuous service is realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of a cluster migration system according to an embodiment of the present application;
FIG. 2 is a flow chart of an embodiment of a cluster migration method provided in an embodiment of the present application;
FIG. 3 is a flow chart of one embodiment of a data operation performed by an associated application provided in an embodiment of the present application;
FIG. 4 is a flowchart of an embodiment of a cluster migration method according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of an embodiment of a cluster migration apparatus according to the present application;
FIG. 6 is a schematic diagram of an embodiment of a computer device provided in an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the drawings are merely for convenience in describing the present application and simplifying the description, and do not indicate or imply that the apparatus or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first", "second", "third" may include one or more of the stated features, either explicitly or implicitly. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the present application, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "exemplary" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
It should be noted that, because the method of the embodiment of the present application is executed in the computer device, the processing objects of each computer device exist in the form of data or information, for example, the duration is substantially the duration information, it can be understood that in the subsequent embodiment, if the size, the number, the position, etc. are all corresponding data, so that the computer device can process the data, which is not described herein in detail.
The embodiment of the application provides a cluster migration method, a cluster migration device, computer equipment and a storage medium, and the method, the device, the computer equipment and the storage medium are respectively described in detail below.
Referring to fig. 1, fig. 1 is a schematic view of a cluster migration system according to an embodiment of the present application, where the cluster migration system may include a computer device 100, and a cluster migration apparatus, such as the computer device in fig. 1, is integrated in the computer device 100.
The computer device 100 in the embodiment of the present application is mainly configured to obtain a cluster migration request for a first cluster and a second cluster; based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services; migrating the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, wherein the associated application preferentially performs data operation on the first cluster in the first state; after the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster; and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster, wherein the situation that data loss and data missing do not occur in the cluster migration process of the computer equipment 100 can be avoided, and smooth migration of the clusters under the condition of continuous service can be realized.
In the embodiment of the present application, the computer device 100 may be an independent server, or may be a server network or a server cluster formed by servers, for example, the computer device 100 described in the embodiment of the present application includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud server formed by a plurality of servers. Wherein the Cloud server is composed of a large number of computers or web servers based on Cloud Computing (Cloud Computing).
It is to be understood that the computer device 100 used in embodiments of the present application may be a device that includes both receive and transmit hardware, i.e., a device having receive and transmit hardware capable of performing bi-directional communications over a bi-directional communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may be one of a mobile phone, a tablet computer, a notebook computer, and the like.
It will be appreciated by those skilled in the art that the application environment shown in fig. 1 is merely an application scenario of the present application, and is not limited to the application scenario of the present application, and other application environments may also include more or fewer computer devices than those shown in fig. 1, for example, only 1 computer device is shown in fig. 1, and it will be appreciated that the cluster migration system may also include one or more other services, which is not limited herein.
In addition, as shown in fig. 1, the cluster migration system may further include a memory 200 for storing data, such as a cluster operation log, e.g., a data write log, a data read log, etc., such as a plurality of services running in the first cluster, associated applications of the first cluster, etc.
It should be noted that, the schematic view of the cluster migration system shown in fig. 1 is only an example, and the cluster migration system and the scene described in the embodiment of the present application are for more clearly describing the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application, and those skilled in the art can know that, with the evolution of the cluster migration system and the appearance of a new service scenario, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.
Firstly, in an embodiment of the present application, a cluster migration method is provided, an execution subject of the cluster migration method is a cluster migration device, the cluster migration device is applied to a computer device, and the cluster migration method includes: acquiring cluster migration requests for a first cluster and a second cluster; based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services; migrating the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, wherein the associated application preferentially performs data operation on the first cluster in the first state; after the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster; and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
Referring to fig. 2, a flow chart of an embodiment of a cluster migration method according to the present application may include the following steps 301 to 306, which are specifically as follows:
301. Cluster migration requests for the first cluster and the second cluster are obtained.
The first Cluster is a Cluster to be migrated, the second Cluster is a target object for Cluster migration, for example, the first Cluster may be a sentinel mode Cluster, and the second Cluster may be a Cluster mode Cluster.
The Cluster migration request is a Cluster migration instruction sent by a user to the computer device to migrate the first Cluster to the second Cluster, where the Cluster migration request includes, but is not limited to, a touch instruction, a mouse instruction, a remote control instruction, a voice instruction, and the like, for example, when the user wants to migrate the whistle mode Cluster to the Cluster mode Cluster, the user directly clicks an icon for Cluster migration on the touch screen, or the user clicks the icon for Cluster migration on the touch screen through a mouse, or the user sends a voice instruction for "migrating the whistle mode Cluster to the Cluster mode Cluster". After a user sends a cluster migration request to the computer device for the first cluster and the second cluster, the computer device obtains the cluster migration request and migrates the first cluster to the second cluster based on the cluster migration request.
302. And synchronizing the data of the first cluster to the second cluster based on the cluster migration request, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services.
The first cluster comprises a large amount of data, when the first cluster is migrated to the second cluster, the data of the first cluster needs to be synchronized to the second cluster, and the data of the first cluster can be synchronized to the second cluster by adopting the existing open-source synchronization tool. For example, when the first Cluster is a sentinel mode Cluster and the second Cluster is a Cluster in a Cluster mode, the data of the sentinel mode Cluster can be synchronized to the Cluster mode Cluster at one time by adopting open source tools such as REDISSHAKE in the air and redis-migratol of a unique sink.
The first cluster is made up of a plurality of servers, each of which is operable with a plurality of services, and the associated application is an application that uses the services operable on the plurality of servers of the first cluster. According to the embodiment, based on the cluster migration request, after the data of the first cluster are synchronized to the second cluster, the associated application of the first cluster is further acquired, so that the first cluster and the second cluster are subjected to cluster migration based on the associated application in the subsequent step.
303. And in the first state, the associated application preferentially performs data operation on the first cluster.
When the first cluster is migrated to the second cluster, a plurality of services running on the first cluster need to be migrated to the second cluster, so as to avoid inconsistent data generated by the first cluster and the second cluster in the process of migrating the plurality of services. In this embodiment, a flow control switch is preset in an associated application of a first cluster, where the flow control switch includes a first state and a second state, and when the flow control switch is in the first state, the associated application may preferentially perform data operation on the first cluster, and when the flow control switch is in the second state, the associated application may preferentially perform data operation on the second cluster.
And synchronizing the data of the first cluster to the second cluster, acquiring the associated application of the first cluster, regulating a flow control switch preset in the associated application to a first state, and then migrating the service of the first cluster to the second cluster when the flow control switch is in the first state. In the service migration process, because the flow control switch is in the first state, the associated application can preferentially perform data operation on the first cluster, namely the sequence of the plurality of service read-write clusters is consistent, and the problem that data generated by the first cluster and the second cluster are inconsistent can not occur.
It should be noted that, in the service migration process, the data synchronization function of the synchronization tool is always in an on state, so that after the associated application performs data operation on the first cluster, the data of the first cluster is synchronized to the second cluster through the synchronization tool, that is, in the service migration process, the data of the first cluster and the data of the second cluster are always in a synchronization state.
304. After the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster.
The association application of the first cluster is also provided with a double-read function switch and a double-write function switch in advance, the double-read function switch is used for carrying out data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for carrying out data double-write operation on the first cluster and the second cluster.
As shown in fig. 3, when the flow control switch is in the first state and the associated application performs the data double-reading operation, data is first read from the first cluster and then read from the second cluster; when the associated application performs the data double-writing operation, the data is written into the first cluster first and then the data is written into the second cluster.
With continued reference to fig. 3, when the flow control switch is in the second state and the associated application performs the data double-reading operation, data is first read from the second cluster and then read from the first cluster; when the associated application performs the data double-writing operation, the data is written into the second cluster first, and then the data is written into the first cluster.
After the migration of the plurality of services of the first cluster is completed, the flow control switch is adjusted from the first state to the second state, and a double-read function switch and a double-write function switch which are preset in the related application are started, when the second cluster is abnormal and needs to roll back data, the data of the first cluster are also full due to double-write bottom-covering operation, so that the situation of losing the data does not occur; when the synchronization tool has a problem or the service of the first cluster is not migrated, the data cannot be lost due to double-read bottom operation, so that the stability of cluster migration is ensured.
305. And determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
After the service of the first cluster is migrated to the second cluster, the first cluster may not be migrated successfully due to various reasons, for example, a problem occurs in the synchronization tool, the service of the first cluster is not migrated, and the like. In order to ensure that the first cluster and the second cluster are migrated under the condition of no stopping, in the embodiment, after the flow control switch is adjusted to a second state and a double-read function switch and a double-write function switch which are preset in the associated application are started, whether the first cluster is successfully migrated to the second cluster is determined, and after the first cluster is determined to be successfully migrated to the second cluster, the second cluster is indicated to be capable of providing services for the associated application instead of the first cluster, and at the moment, the double-read function switch and the double-write function switch are not required to perform bottom-covering operation, and the double-read function switch and the double-write function switch can be turned off.
With continued reference to fig. 3, after the dual-erasure function switch is turned off, the associated application only performs a data write operation and a data read operation on the second cluster because the flow control switch is in the second state.
In a specific embodiment, referring to fig. 4, the determining in step 305 whether the first cluster is successfully migrated to the second cluster may include the following steps 401 to 402, which are specifically as follows:
401. Acquiring cluster operation logs of the associated application, wherein the cluster operation logs are operation logs of the associated application to the first cluster and the second cluster after the double-read function switch and the double-write function switch are turned on;
402. Based on the cluster operation log, it is determined whether the first cluster successfully migrated to the second cluster.
After the double-read function switch and the double-write function switch are turned on, the associated application can perform data double-read operation and data double-write operation on the first cluster and the second cluster, and the cluster operation log is the operation log of the associated application on the first cluster and the second cluster after the double-read function switch and the double-write function switch are turned on. When determining whether the first cluster is successfully migrated to the second cluster, the embodiment firstly acquires a cluster operation log of the associated application, and then determines whether the first cluster is successfully migrated to the second cluster based on the cluster operation log.
In a specific embodiment, the determining whether the first cluster is successfully migrated to the second cluster in step 402 based on the cluster operation log may include the following steps 501-503, which are specifically as follows:
501. Determining whether the second cluster has data writing abnormality based on the data writing log;
502. determining, when the second cluster does not have a data writing abnormality, whether target data exists in the first cluster based on the data reading log, the target data being data that exists in the first cluster but does not exist in the second cluster;
503. And when the target data does not exist in the first cluster, determining that the first cluster is successfully migrated to the second cluster.
The cluster operation log comprises a data writing log and a data reading log, wherein the data writing log is the log of the associated application for performing data double-writing operation on the first cluster and the second cluster, and the data reading log is the log of the associated application for performing data double-reading operation on the first cluster and the second cluster. Wherein the data write log records anomalies of data double write operations, such as a failure of data a to write to the second cluster; the data read log may record anomalies in data double read operations, such as data a failed to read from the second cluster.
The target data is data existing in the first cluster but not in the second cluster, and when the associated application performs data double-reading operation on the target data, the data reading log records that the data is not read from the second cluster because the data does not exist in the second cluster, namely whether the target data exists in the first cluster can be determined through the data reading log.
In the embodiment, when determining whether the first cluster is successfully migrated to the second cluster based on the cluster operation log, firstly, determining whether the second cluster has a daily data writing based on the data writing log, when the second cluster does not have abnormal data writing, determining whether target data exists in the first cluster based on the data reading log, and when the target data does not exist in the first cluster, determining that the first cluster is successfully migrated to the second cluster. Otherwise, when the target data exists in the first cluster, the first cluster is determined to be not successfully migrated to the second cluster.
In a specific embodiment, when the target data exists in the first cluster, after determining that the first cluster is not successfully migrated to the second cluster, the method may include the following steps 504 to 506, which are specifically as follows:
504. Determining whether a target service exists in the first cluster, wherein the target service is a service which is not migrated to the second cluster in the plurality of services;
505. migrating the target service to the second cluster when the target service exists;
506. synchronizing data of the first cluster to the second cluster.
The target service is a service which is not migrated to the second cluster in a plurality of services of the first cluster, and when target data exists in the first cluster, the target service may be a service which has a problem in a synchronization tool or is not migrated in the first cluster. In the embodiment, when target data exists in a first cluster, whether target service exists in the first cluster is firstly determined, when the target service exists in the first cluster, the target service is migrated to a second cluster, and the data of the first cluster is synchronized to the second cluster again by adopting a synchronization tool; and when the target service does not exist in the first cluster, directly adopting a synchronization tool to resynchronize the data of the first cluster to the second cluster.
In a specific embodiment, with continued reference to fig. 4, after the migration of the plurality of services in step 304 is completed, the method may further include the following step 403, which is specifically as follows:
403. And starting a double-deleting function switch preset in the associated application, wherein the double-deleting function switch is used for carrying out data double-deleting operation on the first cluster and the second cluster.
And a double-deleting function switch is also preset in the associated application, and is used for carrying out data double-deleting operation on the first cluster and the second cluster. With continued reference to fig. 3, when the flow control switch is adjusted to be in the first state, and the associated application executes the data double deletion operation, the data deletion operation is performed on the first cluster first, and then the data deletion operation is performed on the second cluster; when the flow control switch is in the second state and the associated application executes the data double deletion operation, the data deletion operation is firstly performed on the second cluster, and then the data deletion operation is performed on the first cluster.
After the migration of the plurality of services is completed, the double-deletion function switch preset in the associated application is also started, and the data size of the cluster can be reduced and whether the data in the cluster are reasonably used or not can be determined through the double-deletion function switch. For example, in the express service, a complete life cycle from ordering to signing can be completed in one week, so that in theory, data in one week should be expired or deleted by related applications along with the completion of the service, if the data size is not reduced after exceeding the service period, whether the memory space of the cluster is reasonably used needs to be checked.
In this embodiment, the dual-erasure function switch may be turned off after the dual-write function switch is turned off, so that the data size of the first cluster is reduced, and the dual-erasure function switch may be turned off after the dual-read function switch and the dual-write function switch determine that the data size of the first data cluster is reduced.
With continued reference to fig. 3, after the dual delete function switch is turned off, the associated application will only perform a data delete operation on the second cluster because the flow control switch is in the second state.
In a specific embodiment, with continued reference to fig. 4, before the obtaining the cluster migration request for the first cluster and the second cluster in step 301, the method may further include the following step 300, specifically as follows:
300. creating the second cluster, wherein the second cluster comprises a plurality of fragments, the number of the fragments is larger than a preset number threshold, and the memory of each fragment in the plurality of fragments is smaller than a preset memory threshold.
In this embodiment, before obtaining the cluster migration requests for the first cluster and the second cluster, the second cluster may be further created, and in order to implement migration of the first cluster, the memory of the second cluster needs to be greater than or equal to the memory of the first cluster. In addition, in order to facilitate capacity expansion in peak hours, the number of the created plurality of slices is greater than a preset number threshold, and the memory of each of the plurality of slices is less than a preset memory threshold.
In order to better implement the cluster migration method in the embodiment of the present application, on the basis of the cluster migration method, the embodiment of the present application further provides a cluster migration device, as shown in fig. 5, where the cluster migration device 600 includes:
A request acquiring unit 601, configured to acquire a cluster migration request for a first cluster and a second cluster;
A first synchronization unit 602, configured to synchronize data of the first cluster to the second cluster based on the cluster migration request, and obtain an associated application of the first cluster, where a plurality of services are running in the first cluster, and the associated application is an application that uses the plurality of services;
a service migration unit 603, configured to migrate the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, where the associated application preferentially performs data operation on the first cluster in the first state;
The switch opening unit 604 is configured to adjust the flow control switch to a second state after the migration of the plurality of services is completed, and open a dual-read function switch and a dual-write function switch that are preset in the associated application, where in the second state, the associated application preferentially performs data operation on the second cluster, the dual-read function switch is configured to perform data dual-read operation on the first cluster and the second cluster, and the dual-write function switch is configured to perform data dual-write operation on the first cluster and the second cluster;
The migration determining unit 605 is configured to determine whether the first cluster is migrated to the second cluster successfully, and close the dual read function switch and the dual write function switch after the first cluster is migrated to the second cluster successfully.
In the embodiment of the application, the clusters are migrated based on the flow control switch, the double-read function switch and the double-write function switch which are preset in the associated application, so that the first cluster can be smoothly migrated to the second cluster under the condition of continuous service; after the migration of a plurality of services is completed, a double-write function switch is started, so that the situation that data are lost in the cluster migration process can be avoided; after the migration of the plurality of services is completed, the double-read function switch is started, and when the data reading operation is carried out in the cluster migration process by the associated application, the condition of missing data cannot occur, so that the smooth migration of the clusters under the condition of continuous service is realized.
In some embodiments of the present application, the migration determining unit 605 is specifically configured to:
Acquiring cluster operation logs of the associated application, wherein the cluster operation logs are operation logs of the associated application to the first cluster and the second cluster after the double-read function switch and the double-write function switch are turned on;
Based on the cluster operation log, it is determined whether the first cluster successfully migrated to the second cluster.
In some embodiments of the present application, the cluster operation log includes a data write log and a data read log, and the migration determining unit 605 is specifically further configured to:
determining whether the second cluster has data writing abnormality based on the data writing log;
Determining, when the second cluster does not have a data writing abnormality, whether target data exists in the first cluster based on the data reading log, the target data being data that exists in the first cluster but does not exist in the second cluster;
and when the target data does not exist in the first cluster, determining that the first cluster is successfully migrated to the second cluster.
In some embodiments of the present application, the migration determining unit 605 is specifically further configured to:
And when target data exists in the first cluster, determining that the first cluster is not successfully migrated to the second cluster.
In some embodiments of the present application, the cluster service migration apparatus 600 further includes:
A target determining unit, configured to determine whether a target service exists in the first cluster, where the target service is a service that is not migrated to the second cluster among the plurality of services;
a target migration unit, configured to migrate, when a target service exists, the target service to the second cluster;
And the second synchronization unit is used for synchronizing the data of the first cluster to the second cluster.
In some embodiments of the present application, the switch opening unit 604 is further configured to:
And starting a double-deleting function switch preset in the associated application, wherein the double-deleting function switch is used for carrying out data double-deleting operation on the first cluster and the second cluster.
In some embodiments of the present application, the migration determining unit 605 is further configured to:
And closing the double-delete function switch.
In some embodiments of the present application, the cluster service migration apparatus 600 further includes:
The cluster creating unit is used for creating the second cluster, the second cluster comprises a plurality of fragments, the number of the fragments is larger than a preset number threshold, and the memory of each fragment in the plurality of fragments is smaller than the preset memory threshold.
The embodiment of the application also provides a computer device, which integrates any one of the cluster migration devices provided by the embodiment of the application, and the computer device comprises:
One or more processors;
a memory; and
One or more applications, wherein the one or more applications are stored in the memory and configured to perform the steps of the cluster migration method described in any of the above cluster migration method embodiments by the processor.
The embodiment of the application also provides computer equipment which integrates any cluster migration device provided by the embodiment of the application. As shown in fig. 6, a schematic structural diagram of a computer device according to an embodiment of the present application is shown, specifically:
The computer device may include one or more processors 701 of a processing core, memory 702 of one or more computer readable storage media, power supply 703, and input unit 704, among other components. Those skilled in the art will appreciate that the computer device structure shown in FIG. 6 is not limiting of the computer device and may include more or fewer components than shown, or may be combined with certain components, or a different arrangement of components. Wherein:
The processor 701 is a control center of the computer device, connects various parts of the entire computer device using various interfaces and lines, and performs various functions of the computer device and processes data by running or executing software programs and/or modules stored in the memory 702, and calling data stored in the memory 702, thereby performing overall monitoring of the computer device. Optionally, processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor primarily handles operating systems, user interfaces, applications, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 701.
The memory 702 may be used to store software programs and modules, and the processor 701 executes various functional applications and data processing by executing the software programs and modules stored in the memory 702. The memory 702 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide access to the memory 702 by the processor 701.
The computer device further comprises a power supply 703 for powering the various components, preferably the power supply 703 is logically connected to the processor 701 by a power management system, whereby the functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 703 may also include one or more of any component, such as a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.
The computer device may further comprise an input unit 704, which input unit 704 may be used for receiving input numerical or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.
Although not shown, the computer device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 701 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 executes the application programs stored in the memory 702, so as to implement various functions, as follows:
Acquiring cluster migration requests for a first cluster and a second cluster;
Based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services;
migrating the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, wherein the associated application preferentially performs data operation on the first cluster in the first state;
After the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster;
and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like. On which a computer program is stored, which is loaded by a processor to perform the steps of any of the cluster migration methods provided by the embodiments of the present application. For example, the loading of the computer program by the processor may perform the steps of:
Acquiring cluster migration requests for a first cluster and a second cluster;
Based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services;
migrating the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, wherein the associated application preferentially performs data operation on the first cluster in the first state;
After the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster;
and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of one embodiment that are not described in detail in the foregoing embodiments may be referred to in the foregoing detailed description of other embodiments, which are not described herein again.
In the implementation, each unit or structure may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit or structure may be referred to the foregoing method embodiments and will not be repeated herein.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
The foregoing describes in detail a cluster migration method, apparatus, computer device and storage medium provided by the embodiments of the present application, and specific examples are applied to illustrate the principles and embodiments of the present application, where the foregoing examples are only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.
Claims (10)
1. A cluster migration method, wherein the cluster migration method comprises:
Acquiring cluster migration requests for a first cluster and a second cluster;
Based on the cluster migration request, synchronizing the data of the first cluster to the second cluster, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services;
migrating the plurality of services to the second cluster when a flow control switch preset in the associated application is in a first state, wherein the associated application preferentially performs data operation on the first cluster in the first state;
After the migration of the plurality of services is completed, the flow control switch is regulated to a second state, a double-read function switch and a double-write function switch which are preset in the associated application are started, the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster;
and determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
2. The cluster migration method of claim 1, wherein the determining whether the first cluster successfully migrates to the second cluster comprises:
Acquiring cluster operation logs of the associated application, wherein the cluster operation logs are operation logs of the associated application to the first cluster and the second cluster after the double-read function switch and the double-write function switch are turned on;
Based on the cluster operation log, it is determined whether the first cluster successfully migrated to the second cluster.
3. The cluster migration method of claim 2, wherein the cluster operation log includes a data write log and a data read log, and wherein the determining whether the first cluster successfully migrates to the second cluster based on the cluster operation log includes:
determining whether the second cluster has data writing abnormality based on the data writing log;
Determining, when the second cluster does not have a data writing abnormality, whether target data exists in the first cluster based on the data reading log, the target data being data that exists in the first cluster but does not exist in the second cluster;
and when the target data does not exist in the first cluster, determining that the first cluster is successfully migrated to the second cluster.
4. The cluster migration method of claim 3, wherein after determining whether target data exists in the first cluster based on the data read log, further comprising:
And when target data exists in the first cluster, determining that the first cluster is not successfully migrated to the second cluster.
5. The method of cluster migration of claim 4, wherein after the determining that the first cluster did not migrate successfully to the second cluster, further comprising:
determining whether a target service exists in the first cluster, wherein the target service is a service which is not migrated to the second cluster in the plurality of services;
migrating the target service to the second cluster when the target service exists;
Synchronizing data of the first cluster to the second cluster.
6. The cluster migration method of claim 1, wherein after the plurality of service migration is completed, further comprising:
Starting a double-deleting function switch preset in the associated application, wherein the double-deleting function switch is used for carrying out data double-deleting operation on the first cluster and the second cluster;
after the double read function switch and the double write function switch are closed, the method further comprises:
And closing the double-delete function switch.
7. The cluster migration method of claim 1, wherein prior to obtaining the cluster migration requests for the first cluster and the second cluster, further comprising:
Creating the second cluster, wherein the second cluster comprises a plurality of fragments, the number of the fragments is larger than a preset number threshold, and the memory of each fragment in the plurality of fragments is smaller than a preset memory threshold.
8. A cluster migration apparatus, wherein the cluster migration apparatus comprises:
a request acquisition unit configured to acquire a cluster migration request for a first cluster and a second cluster;
the first synchronization unit is used for synchronizing the data of the first cluster to the second cluster based on the cluster migration request, and acquiring an associated application of the first cluster, wherein a plurality of services are operated in the first cluster, and the associated application is an application using the plurality of services;
A service migration unit, configured to migrate, in a first state of a flow control switch preset in the association application, the plurality of services to the second cluster, where in the first state, the association application preferentially performs data operation on the first cluster;
The switch starting unit is used for adjusting the flow control switch to a second state after the migration of the plurality of services is completed, starting a double-read function switch and a double-write function switch which are preset in the associated application, wherein the associated application preferentially performs data operation on the second cluster in the second state, the double-read function switch is used for performing data double-read operation on the first cluster and the second cluster, and the double-write function switch is used for performing data double-write operation on the first cluster and the second cluster;
and the migration determining unit is used for determining whether the first cluster is successfully migrated to the second cluster, and closing the double-read function switch and the double-write function switch after the first cluster is successfully migrated to the second cluster.
9. A computer device, the computer device comprising:
One or more processors;
a memory; and
One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the cluster migration method of any one of claims 1 to 7.
10. A computer readable storage medium, having stored thereon a computer program, the computer program being loaded by a processor to perform the steps of the cluster migration method of any one of claims 1 to 7.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211690119 | 2022-12-27 | ||
| CN2022116901197 | 2022-12-27 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118260265A true CN118260265A (en) | 2024-06-28 |
Family
ID=91602947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310031308.1A Pending CN118260265A (en) | 2022-12-27 | 2023-01-10 | Cluster migration method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118260265A (en) |
-
2023
- 2023-01-10 CN CN202310031308.1A patent/CN118260265A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9396216B2 (en) | Repository redundancy implementation of a system which incrementally updates clients with events that occurred via a cloud-enabled platform | |
| RU2544777C2 (en) | Dynamic placement of exact copy data | |
| KR20210002574A (en) | Data backup methods, storage media and computing devices | |
| EP3499786B1 (en) | Data synchronization method, apparatus, storage medium and electronic device | |
| US20140337491A1 (en) | Repository redundancy implementation of a system which incrementally updates clients with events that occurred via a cloud-enabled platform | |
| CN111475483B (en) | Database migration method and device and computing equipment | |
| US20140379656A1 (en) | System and Method for Maintaining a Cluster Setup | |
| CN104737135B (en) | The information processing terminal and synchronisation control means | |
| JP2012520504A (en) | A programming model for synchronizing the browser cache across devices and web services | |
| WO2018192478A1 (en) | Cloud management platform, virtual machine management method, system thereof | |
| EP3812930A1 (en) | Distributed transaction processing method and related device | |
| JP2013517536A (en) | Storage network system and control method thereof | |
| CN115114344A (en) | Transaction processing method and device, computing equipment and storage medium | |
| US11157456B2 (en) | Replication of data in a distributed file system using an arbiter | |
| CN111399753A (en) | Method and device for writing pictures | |
| CN112650629B (en) | Block chain index data recovery method, device, equipment and computer storage medium | |
| CN116578447B (en) | Virtual machine data recovery method and device, computer equipment and storage medium | |
| CN117033082A (en) | Virtual machine backup recovery method and device, computer equipment and storage medium | |
| CN118260265A (en) | Cluster migration method, device, computer equipment and storage medium | |
| CN114490540B (en) | Data storage method, medium, device and computing equipment | |
| CN112925770B (en) | Database capacity expansion method, device, equipment and medium | |
| CN115174890A (en) | Flow playback test method and device, computer equipment and storage medium | |
| CN112732198A (en) | File management method, device, equipment and storage medium | |
| US20240202077A1 (en) | Service cluster instance backup and recovery methods and related devices | |
| CN117632891A (en) | Cluster analysis method and system for distributed system, target equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |