CN115237892B - Data migration method and device - Google Patents
Data migration method and device Download PDFInfo
- Publication number
- CN115237892B CN115237892B CN202211146513.4A CN202211146513A CN115237892B CN 115237892 B CN115237892 B CN 115237892B CN 202211146513 A CN202211146513 A CN 202211146513A CN 115237892 B CN115237892 B CN 115237892B
- Authority
- CN
- China
- Prior art keywords
- database cluster
- communication connection
- application node
- database
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005012 migration Effects 0.000 title claims abstract description 57
- 238000013508 migration Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000004891 communication Methods 0.000 claims abstract description 124
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present application relates to the field of databases, and in particular, to a data migration method and apparatus. The data migration method comprises the following steps: establishing communication connection with a first database cluster, and acquiring original data stored by the first database cluster; establishing communication connection with an application node connected to the first database cluster, wherein the communication connection with the first database cluster is disconnected before the application node establishes communication connection with the second database cluster; and when the application node connected with the second database cluster provides service to the outside, second incremental data are generated in the second database cluster, and first incremental data generated in the first database cluster when the application node connected with the first database cluster provides service to the outside are acquired through communication connection with the first database cluster.
Description
Technical Field
The present application relates to the field of databases, and in particular, to a data migration method and apparatus.
Background
When the data volume generated by the application node is too large and occupies too much database resources, a new database is often required to be built to provide data storage service for the application node. In the process of migrating the application node to the new database, the data generated by the application node in the original database also needs to be migrated to the new database.
In the related art, before the migration of the application node is started, the data in the original database needs to be copied and transferred to the new database. And after the data migration is finished, migrating each application node connected to the original database to the new database. However, in order to ensure service availability, the application nodes do not complete migration simultaneously, and part of the application nodes are still connected to the original database to provide service. Therefore, incremental data generated by part of the application nodes on the original database cannot be smoothly migrated to the new database.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data migration method and apparatus, where a connection channel is established between a first database cluster and a second database cluster, and incremental data generated in part of application nodes in the first database cluster is acquired in real time, so as to solve a problem that the incremental data cannot be migrated smoothly.
In a first aspect, an embodiment of the present invention provides a data migration method, where the method is applied to a second database cluster, and the method includes:
establishing communication connection with a first database cluster, and acquiring original data stored by the first database cluster;
establishing communication connection with an application node connected to the first database cluster, wherein the communication connection with the first database cluster is disconnected before the application node establishes communication connection with the second database cluster;
and when the application node connected with the second database cluster provides service to the outside, generating second incremental data in the second database cluster, and acquiring first incremental data generated in the first database cluster when the application node connected with the first database cluster provides service to the outside through communication connection with the first database cluster.
Optionally, the establishing a communication connection with the first database cluster includes:
the first database cluster comprises a first main node and a plurality of first slave nodes in communication connection with the first main node;
the second database cluster comprises a second main node and a plurality of second slave nodes in communication connection with the second main node;
establishing a communication connection with the first database cluster through the second master node;
and synchronizing the acquired original data, the acquired first incremental data and the acquired second incremental data to second slave nodes which are in communication connection with the second master node through the second master node.
Optionally, the establishing a communication connection with an application node connected to the first database cluster includes:
and sequentially establishing communication connection with each application node connected to the first database cluster until the first database cluster is not connected with any application node any more.
Optionally, the establishing a communication connection with an application node connected to the first database cluster further includes:
and establishing communication connection with each application node through the sentinel assembly.
Optionally, before the application node establishes a communication connection with the second database cluster, disconnecting the communication connection with the first database cluster includes:
and restarting the application node, and establishing communication connection between the application node and the second database cluster after restarting.
Optionally, the obtaining, through the communication connection with the first database cluster, first incremental data generated in the first database cluster when the application node establishing connection with the first database cluster provides a service to the outside includes:
when an application node which is connected with the first database cluster and is not connected with the second database cluster provides services to the outside, first incremental data are generated in the first database cluster;
and acquiring first incremental data in the first database cluster at regular time through the communication connection between the second database cluster and the first database cluster.
Optionally, the method includes:
after the second database cluster acquires all the original data and the first incremental data, the communication connection with the first database cluster is disconnected;
at least one application node is always connected to the first database cluster or the second database cluster and provides services to the outside between the disconnection of the communication connection with the first database cluster from the establishment of the communication connection with the first database cluster.
In a second aspect, an embodiment of the present invention provides a data migration apparatus, including:
the first connection module is used for establishing communication connection with a first database cluster and acquiring original data stored by the first database cluster;
the second connection module is used for establishing communication connection with an application node connected to the first database cluster, wherein the application node is disconnected from the first database cluster before establishing communication connection with a second database cluster;
and the acquisition module is used for generating second incremental data in the second database cluster when the application node connected with the second database cluster provides service to the outside, and acquiring first incremental data generated in the first database cluster when the application node connected with the first database cluster provides service to the outside through communication connection with the first database cluster.
In a third aspect, an embodiment of the present invention provides a data migration apparatus, including:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor being capable of performing the method of any of the first aspects when invoked by the program instructions.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and when the program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute the method according to any one of the first aspects.
Through the scheme, the connection between the first database and the second database is established without an external third-party tool, incremental data generated during the restart switching of the application nodes are synchronized into the second database cluster, and smooth migration of the data is realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a data migration system according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of another data migration system according to an embodiment of the present invention;
fig. 3 is a flowchart of a data migration method according to an embodiment of the present invention;
FIG. 4 is a flow chart of another data migration method provided by the embodiments of the present invention;
FIG. 5 is a schematic structural diagram of a data migration apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For better understanding of the technical solutions of the present invention, the following detailed descriptions of the embodiments of the present invention are provided with reference to the accompanying drawings.
It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
When the resources of the database cluster are insufficient, it is often necessary to migrate each application node connected to the database cluster to a new database cluster, and provide services for each application node by the new database cluster. Before executing migration of the terminal node, data generated by each application node on the original database cluster needs to be migrated in advance, so that data synchronization between the new database cluster and the original database cluster is realized.
In the process of migrating each application node, the application nodes partially connected to the original database cluster can generate incremental data on the original database cluster, and the partial incremental data cannot be smoothly migrated to the new database cluster.
In order to migrate all original data and incremental data in an original database cluster to a new database cluster, an embodiment of the present invention first provides a data migration system. Fig. 1 shows a data migration system according to an embodiment of the present invention. The data migration system comprises a first database cluster 101, a plurality of application nodes 102 needing to be migrated, and a second database cluster 103.
The first database cluster 101 is a database cluster that initially establishes a communication connection with each application node and provides data storage services for each application node. The first database cluster 101 is configured to store original data generated by each application node before migration is performed, and first incremental data generated when an application node that establishes a connection with the first database cluster provides a service to the outside during migration of each application node. Wherein the first database cluster 101 may be a Redis database cluster.
In the data migration process, the first database cluster 101 migrates all the original data and the first incremental data generated by each application node to the second database cluster 103 through the communication connection relationship between the first database cluster and the second database cluster 103.
A number of application nodes 102, initially connected to a first database cluster 101, need to be retransferred and connected to a second database cluster 103. Before the migration of each application node 102 is started, all original data generated by the externally provided service is stored in the first database cluster 101. During the migration process of each application node, the first incremental data generated by the application nodes still connected to the first database cluster 101 are still stored in the first database cluster 101, and the second incremental data generated by each application node migrated to the second database cluster 103 are directly stored in the migrated second database cluster 103.
The second database cluster 103 establishes a communication connection with each application node 102 after the migration of each application node 102 is completed, and provides a database cluster of a data storage service for each application node 102. The second database cluster 103 is used for storing second incremental data generated by each application node 102 after the migration is completed, and the original data and the first incremental data acquired from the first database cluster 101. Wherein the second database cluster 103 may be a Redis database cluster.
In the data migration process, the second database cluster 103 actively establishes a communication connection relationship with the first database cluster 101, and is used for acquiring the original data stored in the first database cluster 101 when data migration is started, and acquiring the application nodes 102 still connected to the first database cluster 101 at regular time and first incremental data generated on the first database cluster 101 in the process of migrating each application node 102.
Fig. 2 is a schematic diagram showing a specific structure of the first database cluster 101 and the second database cluster 103 in the data migration system shown in fig. 1 according to the embodiment of the present invention.
As shown in fig. 2, the first database cluster 101 is composed of a first master node 201, a plurality of first slave nodes 202 communicatively connected to the first master node, and a first sentinel element 203. The first master node and the plurality of first slave nodes may be distributed over a plurality of terminal devices.
Specifically, the first master node 201 is configured to establish a communication connection with a second master node according to an instruction of the second master node in the second database cluster 103, so as to implement the communication connection between the first database cluster 101 and the second database cluster 103, and complete data migration. Meanwhile, the first master node 201 is also configured to synchronize the stored original data and the first incremental data to other first slave nodes 202 with which communication connections are established.
The first sentinel component 203 is configured to establish a communication connection between the first database cluster 101 and each application node 102. Meanwhile, the first sentinel component 203 is further configured to periodically acquire the operating states of the first master node 201 and each first slave node 202 in the first database cluster 101, initiate a vote when the state of the first master node 201 is abnormal, and select a new first master node from the plurality of first slave nodes 202.
As shown in fig. 2, the second database cluster 103 is composed of a second master node 204, a plurality of second slave nodes 205 communicatively connected to the second master node, and a second sentinel component 206. The second master node and the plurality of second slave nodes may be distributed over a plurality of terminal devices.
Specifically, the second master node 204 is configured to actively establish a communication connection with the first master node 201 in the first database cluster 101 according to the operation instruction, so as to implement a communication connection between the first database cluster 101 and the second database cluster 103, and complete data migration. In the data migration process, the second master node 204 actively acquires the original data stored in the first database cluster 101 according to the operation instruction, and periodically acquires the first incremental data stored in the first database cluster 101. Meanwhile, the second master node 204 is further configured to synchronize the acquired original data, the first incremental data, and the second incremental data to other second slave nodes 205 that establish communication connections therewith.
The second sentinel component 206 is configured to establish a communication connection between the second database cluster 103 and each application node 102. Meanwhile, the second sentinel component 206 is further configured to periodically obtain the operating states of the second master node 204 and each second slave node 205 in the second database cluster 103, initiate a vote when the state of the second master node 204 is abnormal, and select a new second master node from the plurality of second slave nodes 205.
With reference to the data migration system shown in fig. 1 and fig. 2, an embodiment of the present invention provides a flowchart of a data migration method, where a second database cluster shown in fig. 1 and fig. 2 executes the data migration method, and as shown in fig. 3, the method includes the specific steps of:
and 301, establishing communication connection with the first database cluster, and acquiring the original data stored in the first database cluster.
Specifically, the second host node initiates a connection instruction, and establishes a communication connection with the first host node by calling a communication interface of the first host node. Because the second master node is in communication connection with other second slave nodes in the second database cluster, and the first master node is in communication connection with other first slave nodes in the first database cluster, the communication connection between the first database cluster and the second database cluster is established through the connection between the second master node and the first master node.
The method comprises the steps of obtaining original data generated by each terminal device on a first database cluster before each terminal device starts to execute migration.
302, a communication connection is established with an application node connected to a first database cluster.
Specifically, after the connection between each application node and the second database cluster is established, the communication connection between each application node and the first database cluster is disconnected. And the second database cluster sequentially establishes communication connection with each application node connected to the first database cluster until all the application nodes connected to the first database cluster are migrated to the second database cluster, and the first database cluster is not linked with any application node.
And each application node can be disconnected from the first database cluster only by executing the restarting operation, and is migrated to the second database cluster to establish communication connection with the second database cluster. When the application node is restarted, communication connection is established with the second database cluster after the restart is completed according to the start file modified in advance.
Because all the application nodes cannot be migrated from the first database cluster to the second database cluster at the same time, after part of the application nodes are migrated, part of the application nodes are still connected to the first database cluster, and data generated by external services is stored in the first database cluster, that is, first incremental data is generated.
303, when the application node connected to the second database cluster provides service to the outside, generating second incremental data in the second database cluster, and acquiring, through the communication connection with the first database cluster, the first incremental data generated in the first database cluster when the application node connected to the first database cluster provides service to the outside.
Specifically, when the application node that has migrated to the cluster on the second database cluster provides a service to the outside, the generated second incremental data is directly stored in the second database cluster, and is not required to be acquired from the first database cluster through a communication connection between the first database clusters.
And the first incremental data generated when the application node still connected to the first database cluster provides services to the outside is still stored in the first database cluster. The second database cluster needs to acquire the first incremental data from the first database cluster at regular time through the communication connection relation established with the first database cluster until the application node is no longer in communication connection with the first database cluster, namely when the application node is no longer generating the first incremental data on the first database cluster, the second database cluster determines that the data migration is completely completed.
The second database cluster acquires original data, first incremental data and second incremental data from a first main node of the first database cluster through a second main node, and synchronizes the acquired original data through the second main node to each second slave node in communication connection with the second main node, so that data synchronization in the database cluster is realized.
And after the second database cluster acquires all the original data and the first incremental data from the first database cluster, disconnecting the communication connection with the first database cluster.
And at least one application node is always connected with the first database cluster or the second database cluster and provides service to the outside between the establishment of the communication connection with the first database cluster by the second database cluster and the disconnection of the communication connection with the first database cluster.
According to the embodiment of the invention, the connection between the first database cluster and the second database cluster is established without an external third-party tool, the incremental data generated during the restart switching of the application nodes are synchronized into the second database cluster, the smooth migration of the data is realized, and the service provided by each application node to the outside is not interrupted.
Referring to fig. 4, in a specific embodiment of the data migration method provided in the embodiment of the present invention, in this embodiment, the number of the application nodes is three, the three application nodes may belong to the same application, the first database cluster includes one first master node and two first slave nodes, and the second database cluster includes one second master node and two second slave nodes. The second database cluster executes the data migration method, as shown in fig. 4, the method specifically includes the steps of:
401, establishing a communication connection with the first database cluster, and acquiring the original data already stored by the first database cluster.
Specifically, a communication interface of the first master node is called through the second master node, and a communication connection between the second master node and the first master node is established, so that a communication connection between the second database cluster and the first database cluster is established.
And acquiring original data which are generated and stored on the first database cluster before the first application node, the second application node and the third application node start to migrate from the first database cluster through a communication connection relation with the first database cluster.
402, establishing communication connections with a first application node, a second application node, and a third application node connected to a first database cluster.
Specifically, a communication connection is first established with the first application node. And the first application node executes restart operation, disconnects the communication connection relation with the first database cluster and establishes the communication connection relation with the second database cluster. And when the communication connection relation is established between the first application node and the second database cluster after the first application node is restarted, the first application node can provide services to the outside again, and second incremental data generated after the first application node is stored in the second database cluster. In the process, the second application node and the third application node which are still connected to the first database cluster always provide services to the outside, and the generated first incremental data is stored in the first database cluster.
Second, a communication connection is established with a second application node. And the second application node executes the restarting operation, disconnects the communication connection relation with the first database cluster and establishes the communication connection relation with the second database cluster. And when the communication connection relation is established between the second application node and the second database cluster after the second application node is restarted, the second application node can provide service to the outside again, and second incremental data generated after the second application node is stored in the second database cluster. In the process, the first application node and the third application node always provide services to the outside, second incremental data generated by the first application node is stored in the second database cluster, and first incremental data generated by the third application node is stored in the first database cluster.
And finally, establishing communication connection with the third application node. And the third application node executes the restarting operation, disconnects the communication connection relation with the first database cluster and establishes the communication connection relation with the second database cluster. And when the communication connection relation is established between the third application node and the second database cluster after the third application node is restarted, the third application node can provide service to the outside again, and the second incremental data generated after the third application node is stored in the second database cluster. In the process, the first application node and the second application node which are migrated to the second database cluster always provide services to the outside, and the generated second incremental data is stored in the second database cluster.
And when the first application node, the second application node and the third application node are all migrated to the second database cluster, the first database cluster is not in communication connection with any application node any more, and no application node generates the first incremental data any more, determining that the migration of the application node is completed.
403, when the application node connected to the second database cluster provides the service to the outside, generating second incremental data in the second database cluster, and acquiring, through the communication connection with the first database cluster, the first incremental data generated in the first database cluster by the application node connected to the first database cluster when the application node provides the service to the outside.
Specifically, in the migration process of the first application node, the second application node and the third application node, the first incremental data generated by each application node on the first database cluster is acquired at regular time through the communication connection between the second database cluster and the first database cluster. And when the preset time is up, acquiring first incremental data generated by each application node through a communication connection relation established between the first incremental data and the first main node.
For example, when the preset time is reached, the first application node completes the migration, the second application node and the third application node are still connected to the first database cluster, and the first incremental data is generated on the first database cluster. And at the moment, acquiring first incremental data of the second application node and the third application node through the communication connection relationship established between the first application node and the first host node. The first application node at this point generates second incremental data directly on the second database cluster.
Optionally, the second database cluster may configure the preset time through the second master node.
Optionally, the second database cluster may also directly and actively acquire the first incremental data on the first database cluster through the second master node, without waiting for the preset time.
Data synchronization is completed within the second database cluster 404.
Specifically, the second master node synchronizes the acquired original data, the first incremental data and the second incremental data to the other two second slave nodes in the second database cluster, so as to complete data synchronization in the second database cluster.
According to the embodiment of the invention, the connection between the first database cluster and the second database cluster is established without an external third-party tool, the incremental data generated during the restart switching of the application nodes are synchronized into the second database cluster, the smooth migration of the data is realized, and the service provided by each application node to the outside is not interrupted.
Corresponding to the data migration method, the embodiment of the invention also provides a data migration device. Fig. 5 is a schematic structural diagram of a data migration apparatus according to an embodiment of the present invention. As shown in fig. 5, the apparatus may include: a first connection module 501, a second connection module 502 and an acquisition module 503.
The first connection module 501 establishes communication connection with a first database cluster, and acquires original data stored by the first database cluster;
a second connection module 502, which establishes a communication connection with an application node connected to the first database cluster, wherein the communication connection with the first database cluster is disconnected before the application node establishes a communication connection with the second database cluster;
the obtaining module 503 is configured to generate second incremental data in the second database cluster when the application node connected to the second database cluster provides a service to the outside, and obtain first incremental data generated in the first database cluster when the application node connected to the first database cluster provides a service to the outside through the communication connection with the first database cluster.
The data migration apparatus provided in the embodiment shown in fig. 5 may be used to implement the technical solutions of the method embodiments shown in this specification, and reference may be further made to the relevant descriptions in the method embodiments for implementing the principles and technical effects.
Fig. 6 is a schematic structural diagram of an embodiment of an electronic device in the present specification. As shown in fig. 6, the electronic device may include at least one processor; and at least one memory communicatively coupled to the processing unit, wherein: the memory stores program instructions executable by the processing unit, and the processor calls the program instructions to execute the data migration method provided by the embodiment.
The electronic device may be a device capable of performing an intelligent conversation with a user, and the specific form of the electronic device is not limited in the embodiments of the present specification. It is understood that the electronic device herein is a machine as mentioned in the method embodiments.
FIG. 6 illustrates a block diagram of an exemplary electronic device suitable for use in implementing embodiments of the present specification. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the range of use of the embodiments of the present specification.
As shown in fig. 6, the electronic device is in the form of a general purpose computing device. Components of the electronic device may include, but are not limited to: one or more processors 610, a communication interface 620, a memory 630, and a communication bus 640 that connects the various system components (including the memory 630, the communication interface 620, and the processors 610).
Electronic devices typically include a variety of computer system readable media. Such media may be any available media that is accessible by an electronic device and includes both volatile and nonvolatile media, removable and non-removable media.
A program/utility having a set (at least one) of program modules, including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment, may be stored in memory 630. The program modules generally perform the functions and/or methodologies of the embodiments described herein.
The processor 610 executes various functional applications and data processing by executing programs stored in the memory 630, for example, implementing the data migration method provided by the embodiments illustrated in this specification.
The embodiments of the present specification provide a non-transitory computer readable storage medium storing computer instructions for causing the computer to execute the data migration method provided by the embodiments shown in the present specification.
The non-transitory computer readable storage medium described above may take any combination of one or more computer readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM) or flash Memory, an optical fiber, a portable compact disc Read Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present description may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this specification, "plurality" means at least two, e.g., two, three, etc., unless explicitly specified otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present description in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present description.
The word "if," as used herein, may be interpreted as "at \8230; \8230when" or "when 8230; \823030when" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
It should be noted that the terminal referred to in the embodiments of the present disclosure may include, but is not limited to, a Personal Computer (Personal Computer; hereinafter, referred to as PC), a Personal Digital Assistant (Personal Digital Assistant; hereinafter, referred to as PDA), a wireless handheld device, a Tablet Computer (Tablet Computer), a mobile phone, an MP3 player, an MP4 player, and the like.
In the embodiments provided in the present specification, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present description may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or in the form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a Processor (Processor) to execute some steps of the methods described in the embodiments of the present disclosure.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.
Claims (9)
1. A method of data migration, the method being applied to a second cluster of databases, the method comprising:
establishing communication connection with a first database cluster, and acquiring original data stored by the first database cluster;
establishing communication connection with an application node connected to the first database cluster, wherein the communication connection with the first database cluster is disconnected before the application node establishes communication connection with the second database cluster;
when the application node connected with the second database cluster provides service to the outside, second incremental data are generated in the second database cluster, and first incremental data generated in the first database cluster when the application node connected with the first database cluster provides service to the outside are obtained through communication connection with the first database cluster;
the acquiring, through the communication connection with the first database cluster, first incremental data generated in the first database cluster by an application node establishing a connection with the first database cluster when providing a service to the outside includes:
when an application node which is connected with the first database cluster and is not connected with the second database cluster provides service to the outside, first incremental data are generated in the first database cluster;
and acquiring first incremental data in the first database cluster at regular time through the communication connection between the second database cluster and the first database cluster.
2. The method of claim 1, wherein establishing a communication connection with a first database cluster comprises:
the first database cluster comprises a first main node and a plurality of first slave nodes in communication connection with the first main node;
the second database cluster comprises a second main node and a plurality of second slave nodes in communication connection with the second main node;
establishing a communication connection with the first database cluster through the second master node;
and synchronizing the acquired original data, the acquired first incremental data and the acquired second incremental data to second slave nodes which are in communication connection with the second master node through the second master node.
3. The method of claim 1, wherein establishing a communication connection with an application node connected to the first database cluster comprises:
and sequentially establishing communication connection with each application node connected to the first database cluster until the first database cluster is not connected with any application node any more.
4. The method of claim 1, wherein establishing a communication connection with an application node connected to the first database cluster further comprises:
and establishing communication connection with each application node through the sentinel assembly.
5. The method of claim 1, wherein disconnecting communication with the first database cluster before the application node establishes communication with the second database cluster comprises:
and restarting the application node, and establishing communication connection between the application node and the second database cluster after restarting.
6. The method of claim 1, comprising:
after the second database cluster acquires all the original data and the first incremental data, disconnecting the communication connection with the first database cluster;
at least one application node is always connected to the first database cluster or the second database cluster and provides service to the outside between the disconnection of the communication connection with the first database cluster from the establishment of the communication connection with the first database cluster.
7. A data migration apparatus, comprising:
the first connection module is used for establishing communication connection with a first database cluster and acquiring original data stored by the first database cluster;
the second connection module is used for establishing communication connection with an application node connected to the first database cluster, wherein the application node is disconnected from the first database cluster before establishing communication connection with a second database cluster;
the acquisition module is used for generating second incremental data in the second database cluster when the application node connected with the second database cluster provides services to the outside, and acquiring first incremental data generated in the first database cluster when the application node connected with the first database cluster provides services to the outside through communication connection with the first database cluster;
the acquiring, through a communication connection with a first database cluster, first incremental data generated in the first database cluster when an application node establishing a connection with the first database cluster provides a service to the outside includes:
when an application node which is connected with the first database cluster and is not connected with the second database cluster provides services to the outside, first incremental data are generated in the first database cluster;
and acquiring first incremental data in the first database cluster at fixed time through the communication connection between the second database cluster and the first database cluster.
8. A data migration apparatus, comprising:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1 to 6.
9. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211146513.4A CN115237892B (en) | 2022-09-20 | 2022-09-20 | Data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211146513.4A CN115237892B (en) | 2022-09-20 | 2022-09-20 | Data migration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115237892A CN115237892A (en) | 2022-10-25 |
| CN115237892B true CN115237892B (en) | 2022-12-16 |
Family
ID=83680425
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211146513.4A Active CN115237892B (en) | 2022-09-20 | 2022-09-20 | Data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115237892B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106339387B (en) * | 2015-07-08 | 2019-11-22 | 阿里巴巴集团控股有限公司 | Increase the method for data synchronization and device of server in a kind of data-base cluster newly |
| CN108132949B (en) * | 2016-12-01 | 2021-02-12 | 腾讯科技（深圳）有限公司 | Data migration method and device in database cluster |
| CN111324596B (en) * | 2020-03-06 | 2021-06-11 | 腾讯科技（深圳）有限公司 | Data migration method and device for database cluster and electronic equipment |
| CN112286905A (en) * | 2020-10-15 | 2021-01-29 | 北京沃东天骏信息技术有限公司 | Data migration method and device, storage medium and electronic equipment |
| CN113157670B (en) * | 2021-05-10 | 2024-02-09 | 中国工商银行股份有限公司 | Data migration method and system of database |
| CN114942838A (en) * | 2022-05-26 | 2022-08-26 | 中信建投证券股份有限公司 | Data access method, device, equipment and storage medium |
-
2022
- 2022-09-20 CN CN202211146513.4A patent/CN115237892B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115237892A (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110381042B (en) | Graceful restart service method, device, medium and electronic equipment | |
| CN113835992B (en) | Memory leakage processing method and device, electronic equipment and computer storage medium | |
| WO2024077885A1 (en) | Management method, apparatus and device for container cluster, and non-volatile readable storage medium | |
| CN107526623B (en) | Data processing method and device | |
| CN110928715A (en) | Method, device, medium and electronic equipment for prompting error description information | |
| CN114328097A (en) | File monitoring method and device, electronic equipment and storage medium | |
| CN114222003A (en) | Service calling method, system, device, equipment and storage medium | |
| CN110704050B (en) | Module initializing method and device, electronic equipment and computer readable storage medium | |
| CN115237892B (en) | Data migration method and device | |
| CN110888773B (en) | Method, device, medium and electronic equipment for acquiring thread identification | |
| CN110659024B (en) | Graphics resource conversion method and device, electronic equipment and storage medium | |
| CN112181724A (en) | Big data disaster tolerance method and device and electronic equipment | |
| CN111240801A (en) | Method, device, medium and electronic equipment for generating heap memory snapshot file | |
| CN113391860A (en) | Service request processing method and device, electronic equipment and computer storage medium | |
| CN115373998A (en) | Application program optimization method, device, equipment and medium | |
| CN112084254A (en) | Data synchronization method and system | |
| CN116048757A (en) | Task processing method, device, electronic equipment and storage medium | |
| CN115167874A (en) | Automatic driving software mirror image deployment method and device, electronic equipment and readable medium | |
| CN114741175A (en) | Task execution method and device, central node and downstream node equipment | |
| CN111538717B (en) | Data processing method, device, electronic equipment and computer readable medium | |
| CN111930391B (en) | Application data processing method and device, server and user terminal | |
| CN110502891B (en) | Method, device, medium and electronic equipment for acquiring process memory leakage | |
| CN113760487A (en) | Service processing method and device | |
| CN111176679A (en) | Application program updating method and device | |
| CN117369953B (en) | Mirror synchronization method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |