CN112256676A

CN112256676A - Method, device, equipment and medium for migrating database

Info

Publication number: CN112256676A
Application number: CN202011254703.9A
Authority: CN
Inventors: 徐晓明
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-11-11
Filing date: 2020-11-11
Publication date: 2021-01-22

Abstract

The method for migrating the database comprises the steps of obtaining full backup data after the latest backup of the database to be migrated, carrying out data synchronization between the database to be migrated and a target database according to the full backup data, after the data synchronization is determined to be completed, transmitting a received data access message to the target database in an agent mode, and transmitting an address modification notification message to each service device, so that each service device transmits a service data message to the target database. Therefore, the data access message sent to the database to be migrated is forwarded to the target database by the proxy in a proxy forwarding mode, so that the problem of database double-writing is avoided, data safety is guaranteed, data loss is reduced, and smooth migration of the database is realized.

Description

Method, device, equipment and medium for migrating database

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a medium for database migration.

Background

With the development of computer technology and internet technology, the application of databases is becoming more and more popular. In the application in the database, due to the old database and the like, the arbitration of the database and the data migration of the database are also inevitable.

In the prior art, two access addresses are usually provided, so that a user can switch between a target database and a database to be migrated according to services.

However, this may result in double writing of the target database and the database to be migrated, causing data corruption.

Disclosure of Invention

The embodiment of the application provides a method, a device, equipment and a medium for database migration, which are used for avoiding database double-writing and reducing data loss when the database is migrated.

In one aspect, a method for database migration is provided, which is applied to a database to be migrated, and includes:

acquiring latest full backup data, wherein the full backup data is acquired after periodic data backup of a database to be migrated;

according to the full backup data, carrying out data synchronization between the database to be migrated and the target database;

if the data synchronization is determined to be completed, forwarding the received data access message agent to a target database;

and sending an address modification notification message to each service device, so that each service device sends a service data message to the target database.

In one aspect, a method for database migration is provided, which is applied to a target database, and includes:

carrying out data synchronization according to the received data to be migrated of the database to be migrated;

receiving a data access message forwarded by an agent after the database to be migrated determines that data synchronization is completed, wherein the data access message is a message sent to the database to be migrated;

returning an address modification notification message to the service equipment which sends the data access message;

and receiving a service data message sent by the service equipment.

In one aspect, an apparatus for database migration is provided, including:

the system comprises an acquisition unit, a storage unit and a migration unit, wherein the acquisition unit is used for acquiring latest full backup data which is acquired after periodic data backup of a database to be migrated;

the synchronization unit is used for carrying out data synchronization between the database to be migrated and the target database according to the latest full backup data;

the forwarding unit is used for forwarding the received data access message to the target database in an agent mode if the data synchronization is determined to be completed;

and the sending unit is used for sending the address modification notification message to each service device so that each service device sends the service data message to the target database.

Preferably, the synchronization unit is configured to:

sending the full backup data to a target database;

if the full backup data is determined to be synchronously completed, acquiring a transaction log and a global transaction identifier corresponding to the full backup data, wherein the transaction log comprises operation records of each database and the corresponding global transaction identifier;

determining a database operation record corresponding to unsynchronized data recorded in a transaction log according to a global transaction identifier corresponding to the full amount of backup data;

and performing data synchronization updating on the target database according to the database operation record corresponding to the unsynchronized data.

Preferably, the synchronization unit is configured to:

acquiring target address information and target authorization information of a target database;

and sending the full backup data to a target database according to the target address information and the target authorization information.

Preferably, the forwarding unit is configured to:

transparently forwarding the data access message received by the appointed network port of the database to be migrated to a target database, so that the target database performs message verification according to the source address information contained in the data access message; alternatively, the first and second electrodes may be,

and based on the appointed network port, receiving a data access message which is sent by the service equipment and contains the authorized access information, and forwarding the received data access message to the target database, so that the target database performs message verification according to the authorized access information contained in the data access message.

Preferably, the sending unit is configured to:

and if the data synchronization is determined to be completed, stopping the database service process.

In one aspect, an apparatus for database migration is provided, including:

the synchronization unit is used for carrying out data synchronization according to the received data to be migrated of the database to be migrated;

the forwarding unit is used for receiving a data access message forwarded by an agent after the database to be migrated determines that the data synchronization is completed, wherein the data access message is a message sent to the database to be migrated;

a return unit, configured to return an address modification notification message to a service device that sends a data access message;

and the receiving unit is used for receiving the service data message sent by the service equipment.

In one aspect, a control device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of any of the above-described database migration methods.

In one aspect, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of any of the above-mentioned methods of database migration.

In one aspect, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in any of the various alternative implementations of database migration described above.

In the method, the device, the equipment and the medium for database migration provided by the embodiment of the application, the full backup data of the latest backup of the database to be migrated is acquired, data synchronization between the database to be migrated and the target database is performed according to the full backup data, and after the data synchronization is determined to be completed, the received data access message is proxied to the target database, and an address modification notification message is sent to each service equipment, so that each service equipment sends a service data message to the target database. Therefore, the data access message sent to the database to be migrated is forwarded to the target database by the proxy in a proxy forwarding mode, so that the problem of database double-writing is avoided, data safety is guaranteed, data loss is reduced, and smooth migration of the database is realized.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a block diagram illustrating an architecture of a database migration system according to an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating an implementation of a method for database migration according to an embodiment of the present disclosure;

fig. 3 is a schematic view of an application scenario of database migration according to an embodiment of the present application;

FIG. 4 is a flowchart illustrating a detailed implementation of a method for database migration according to an embodiment of the present disclosure;

FIG. 5 is a first schematic structural diagram of an apparatus for database migration according to an embodiment of the present disclosure;

FIG. 6 is a second schematic structural diagram of an apparatus for database migration according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a control device in an embodiment of the present application.

Detailed Description

In order to make the purpose, technical solution and beneficial effects of the present application more clear and more obvious, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

First, some terms referred to in the embodiments of the present application will be described to facilitate understanding by those skilled in the art.

The terminal equipment: may be a mobile terminal, a fixed terminal, or a portable terminal such as a mobile handset, station, unit, device, multimedia computer, multimedia tablet, internet node, communicator, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, personal communication system device, personal navigation device, personal digital assistant, audio/video player, digital camera/camcorder, positioning device, television receiver, radio broadcast receiver, electronic book device, gaming device, or any combination thereof, including the accessories and peripherals of these devices, or any combination thereof. It is also contemplated that the terminal device can support any type of interface to the user (e.g., wearable device), and the like.

A server: the cloud server can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and can also be a cloud server for providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, big data and artificial intelligence platform and the like.

A database: in short, it can be regarded as the place where the electronic file is stored in the electronic file cabinet, and the user can add, query, update, delete and the like to the data in the file. A database is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.

A database management system: the computer software system designed for managing the database generally has basic functions of storage, interception, safety guarantee, backup and the like. The database management system may be categorized according to the database models it supports, such as relational, extensible markup language, or according to the types of computers supported, such as server clusters, mobile phones; or classified according to the Query Language used, e.g., Structured Query Language (SQL), XQuery; or by performance impulse emphasis, e.g., maximum size, maximum operating speed; or other classification schemes. Regardless of the manner of classification used, some database management systems are capable of supporting multiple query languages across categories, for example, simultaneously.

Cloud storage: the distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system which integrates a large number of storage devices (storage devices are also referred to as storage nodes) of different types in a network through application software or application interfaces to cooperatively work through functions of cluster application, grid technology, distributed storage file systems and the like, and provides data storage and service access functions to the outside.

At present, a storage method of a storage system is as follows: logical volumes are created, and when created, each logical volume is allocated physical storage space, which may be the disk composition of a certain storage device or of several storage devices. The application program stores data on a certain logical volume, namely, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as data identification, the file system writes each object into a physical storage space of the logical volume respectively, and the file system records storage location information of each object, so that when the application program requests to access the data, the file system can enable the application program to access the data according to the storage location information of each object.

The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided in advance into stripes according to a set of capacity measures of objects stored in a logical volume (the measures usually have a large margin with respect to the capacity of the actual objects to be stored) and Redundant Array of Independent Disks (RAID), and one logical volume can be understood as one stripe, thereby allocating physical storage space to the logical volume.

Cloud DataBase (CDB), a relational DataBase Cloud service.

MySQL: is a relational database management system. The command can be received and corresponding operation can be made, and the command can contain a plurality of operations such as deleting files, obtaining file contents and the like, and the written command is an SQL statement.

The distributed storage system (HBase) is a highly reliable, high-performance, column-oriented, scalable distributed storage system.

Nginx: the system is a free, open-source and high-performance hypertext Transfer Protocol (HTTP) server and a reverse proxy server, and is also a Mail Access Protocol (Internet Mail Access Protocol, IMAP), Post Office Protocol Version (Post Office Protocol Version 3, POP3) and Simple Mail Transfer Protocol (SMTP) proxy server; that is, Nginx itself may host a website to perform HTTP service processing, and may also be used as a reverse proxy server.

HAProxy provides high availability, load balancing, and proxy based on TCP and HTTP applications, supporting virtual hosts, which is a free, fast, and reliable solution. HAProxy is particularly well suited for those web sites that are very heavily loaded, which in turn typically require session maintenance or seven-layer processing. HAProxy runs on current hardware, can fully support tens of thousands of concurrent connections, and can protect your web server from being exposed on the network.

Global Transaction Identification (GTID): for locating and tracking each transaction as it is submitted on the master library or applied by the slave library, with global uniqueness and increasing trend.

Tencent GateWay (TGW): the system is a set of system for realizing multi-network unified access, forwarding network requests of external networks and supporting automatic load balancing. The TGW provides free service for developers, the application based on the HTTP can be directly accessed, and the application based on other private protocols can be quickly accessed only by carrying out little modification. The TGW automatically carries out domain name resolution, and after the application accesses the TGW, the domain name mode can be used for providing external services and telecommunication/Unicom/mobile three-network access. In addition, the TGW supports backend weighted load balancing, and applications do not need to pay attention to load balancing.

Database arbitration: the machine room where the database is located is retired.

Smooth migration of the database: when the database is cut down and needs to be migrated, the database is prevented from double writing and data failure occurs.

The design concept of the embodiment of the present application is described below.

With the development of computer technology and internet technology, the application of databases has been deeply reached in life and work, and various database management systems, websites and the like cannot be supported by the databases. In the application process of the database, there are problems of device aging, upgrading and the like, which require database arbitration and database migration.

For example, if a database to be migrated fails, data needs to be migrated to a new database. For another example, as the demand for data volume increases, the capacity of the database to be migrated is smaller, and data needs to be migrated to a larger capacity database.

In the conventional art, the following techniques are generally employed: in the data migration process, the access address of the database to be migrated and the access address of the target database are provided for the service equipment, so that the service equipment can autonomously switch the databases according to service requirements.

However, in the data migration process, since a large number of access sources exist in the database to be migrated, it is difficult to switch all the access sources of the database to be migrated to the target database at one time, and it is also impossible to ensure that all the service devices modify access addresses at the same time, which may cause double-write of the database, and further cause data loss.

Therefore, a technical scheme for database migration that can avoid database double-writing and reduce data loss is needed.

In view of the above analysis and consideration, the embodiment of the present application provides a technical scheme for database migration, where in the scheme, full backup data after a latest backup of a database to be migrated is obtained, data synchronization between the database to be migrated and a target database is performed according to the full backup data, after the data synchronization is completed, a data access message sent to the database to be migrated is proximately forwarded to the target database, and an address modification notification message is sent to each service device, so that each service device sends a service data message to the target database.

To further illustrate the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the accompanying drawings and the detailed description. Although the embodiments of the present application provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by the embodiments of the present application. The method can be executed in sequence or in parallel according to the method shown in the embodiment or the figure when the method is executed in an actual processing procedure or a device.

The terms "first," "second," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Referring to fig. 1, a schematic diagram of an architecture of a database migration system is shown, which includes a database to be migrated 101, a target database 102, and one or more service devices 103.

The database to be migrated 101 and the target database 102 may be the same database or different databases. The types of the target database 102 and the database to be migrated 101 include, but are not limited to, a relational database. The to-be-migrated database 101 and the target database 102 may be deployed in a control device, where the control device may be a server, may also be a terminal device, may also be deployed in a cloud environment, that is, a cloud database, and specifically may be deployed in a control device or a virtual machine located in the cloud environment.

For example, the to-be-migrated database 101 and the target database 102 may be MySQL databases or HBase databases.

Wherein, the database 101 to be migrated is installed with a proxy forwarding application. Alternatively, the proxy forwarding application may be open source TCP software Nginx or HAProxy. The database to be migrated 101 may employ a Linux operating system.

In practical application, a new database device is deployed, and if it is determined that the new database device is operating normally, the new database device is used as the target database 102.

Wherein, the service device 103: the system may be a terminal device or a server, and is configured to interact with the database to be migrated 101 according to the access address of the database to be migrated 101, and may also interact with the target database 102 according to the access address of the target database 102.

In an application scenario, the method includes the steps of obtaining full backup data of a database to be migrated after the latest backup, determining the data to be migrated according to the full backup data, stopping a database service process after synchronizing the data to be migrated in a database to be migrated 101 to a target database 102, sending data access messages sent by business devices 103 to the database to be migrated 101 to the target database 102 in an agent forwarding mode through an agent forwarding application program, and returning address modification notification messages to the business devices 103 through the database to be migrated 101 and/or the target database 102. After receiving the address modification notification message, each service device 103 sends a subsequent service data message to the target database 102.

The service device may be an access source determined according to the received historical access data, or an access source determined according to the data access message.

In practical application, because the service legacy history is long, all access sources cannot be combed, that is, all the service devices 103 cannot be determined, it is difficult to send address modification notification messages to all the service devices 103 at one time, and thus, database switching of all the service devices 103 cannot be realized at one time, and because the service also has the gray scale, the sequence and the like of a release version, database switching cannot be guaranteed to be performed simultaneously on all the service devices 103, which easily causes database double writing, and further causes data loss. In the embodiment of the present application, after the database service process is stopped, the data access message sent by each service device 103 is still received, the data access message is proxy-forwarded to the target database 102 in a proxy forwarding manner, and the service device 103 is also notified to perform database switching at the same time, so that by using the proxy forwarding manner, the database switching of all the service devices 103 is not required to be implemented at one time, and database double writing is not caused, and the service device 103 requiring database switching can be continuously determined according to each received data access message, until all the relevant service devices 103 are determined to implement database switching, the operation of the to-be-migrated database 101 is stopped, and database cutting-down is completed.

Referring to fig. 2, a flowchart of an implementation of a database migration method according to the present application is shown. The method comprises the following specific processes:

step 201: and the database to be migrated acquires the latest full backup data, and performs data synchronization between the database to be migrated and the target database according to the latest full backup data.

Specifically, when step 201 is executed, the following steps may be adopted:

s2011: and the database to be migrated acquires the full backup data after the latest backup.

Specifically, after the database to be migrated determines that a database migration instruction of the user is received, the latest full-amount backup data is acquired.

Wherein, the full backup data is obtained after the periodic data backup. The database migration instruction may be issued by a user, or may be issued by other devices, for example, a target database, where the latest full-volume backup data includes a global transaction identifier of the full-volume backup data in the last backup operation, that is, a GTID corresponding to an operation record of the backup operation to obtain the latest full-volume backup data.

The time stamp of the operation record of each transaction and the execution sequence of each transaction can be obtained through the GTID.

Before S2011 is executed, the database to be migrated may backup the stored data in any one or a combination of the following two manners.

The first mode is as follows: and periodically carrying out full backup on the data stored in the database to be migrated to obtain a plurality of full backup data.

The full backup refers to performing complete backup on the whole database to be migrated.

Thus, there will usually be a plurality of full backup data, and the migration database can obtain the latest full backup data according to the data backup time.

The second way is: and performing incremental backup on the data stored in the database to be migrated periodically to obtain a plurality of incremental backup data.

The incremental backup refers to that only the data modified after the last backup is backed up each time.

In practical applications, the interval duration of the periodic backup may be set according to practical application scenarios, for example, one day, and is not limited herein.

Therefore, the database to be migrated can obtain the latest full backup data according to the plurality of incremental backups obtained by the periodic incremental backup.

In one embodiment, first, a full backup is performed on data in a database to be migrated to obtain initial full backup data, and then, incremental backups are periodically performed on the data in the database to be migrated to obtain a plurality of incremental backup data.

In practical applications, to avoid the occupation of system resources, only one backup is usually set at the same time point.

Therefore, the latest full backup data can be obtained according to the full backup data obtained by history and the plurality of incremental backup data.

S2012: and the database to be migrated sends the full backup data to the target database.

Specifically, the database to be migrated acquires target address information and target authorization information of the target database, and sends the full amount of backup data to the target database according to the target address information and the target authorization information.

The target address information is a network address used for performing network communication with the target database, for example, the target address information may be a target Internet Protocol (IP) address and a target port of the target database, and for example, the target address information may be a target domain name and a target port of the target database. The target authorization information is used for authorization verification of the message, and for example, the target authorization information may include a target account number and a target password.

In one embodiment, the database to be migrated executes the following code to send the full amount of backup data to the target database:

mysql-h target IP address-p target port-u target account-p target password < backup.

Thus, the full amount of backup data can be copied to the target database according to the target address information and the target authorization information.

S2013: and if the synchronization of the full backup data is determined to be completed, the database to be migrated acquires the transaction log and the global transaction identifier corresponding to the full backup data.

Before executing S2013, the target database receives the full backup data sent by the database to be migrated, and after the full backup data are determined to be synchronized, the GTID corresponding to the full backup data contained in the full backup data is acquired, and the source address information and the source authorization information of the database to be migrated are acquired. And the target database sends a data synchronization completion response message containing the GTID to the database to be migrated, and establishes a copy communication channel between the database to be migrated and the target database according to the source address information and the source authorization information of the migration database.

After receiving a data synchronization completion response message sent by a target database, the database to be migrated determines that the synchronization of the full backup data is completed, acquires a GTID corresponding to the full backup data contained in the data synchronization completion response message, establishes a copy communication channel between the database to be migrated and the target database after passing the verification of source authorization information sent by the migration database, and acquires a transaction log containing an operation record of the database to be migrated.

The transaction log comprises operation records of the database to be migrated and corresponding timestamps and GTIDs.

In one embodiment, the GTID corresponding to the full backup data, namely 42dcaaf8-168f-11ea-bb4b-047d7bd5d7e7:1-3646040323, can be set on the target database in the following way: a resetmaster;

SET@@GLOBAL.GTID_PURGED＝'42dcaaf8-168f-11ea-bb4b-047d7bd5d7e7:1-3646040323'。

in one embodiment, the replication communication channel between the database to be migrated and the target database may be established in the following manner.

CHANGE MASTER TO MASTER _ HOST ═ Source IP Address ', MASTER _ PORT ═ Source Port, MASTER _ USER ═ Source Account ', MASTER _ PASSWORD ═ Source PASSWORD ', MASTER _ AUTO _ POSITION ═ 1.

The source address information comprises a source IP address and a source port, and the source authorization information comprises a source account and a source password. The source authorization information is used for message authentication.

S2014: and the database to be migrated determines a database operation record corresponding to the unsynchronized data recorded in the transaction log according to the global transaction identifier corresponding to the full backup data, and performs data synchronization update on the target database according to the database operation record corresponding to the unsynchronized data.

Therefore, after the synchronization of the full backup data is completed, the backup generation time of the latest full backup data can be determined through the GTID corresponding to the full backup data, the operation record of the database to be migrated after the backup generation time is further acquired, and corresponding updating operation is sequentially executed on the data in the target database according to the acquired operation record.

The operation record can be deletion, writing, modification and the like of data.

Therefore, the data synchronization updating is carried out on the target database according to the operation record of the database to be migrated after the latest full backup operation.

And further, detecting the synchronization state between the database to be migrated and the target database until the data between the database to be migrated and the target database are completely synchronized.

In one embodiment, the show slow status \ G is executed to look at the second _ while _ Master value, which when 0, indicates that the data of the target database and the database to be migrated have been synchronized.

Therefore, the latest full backup data can be synchronized to the target database through the data synchronization channel between the database to be migrated and the target database, and the data synchronization update is performed on the target database through the transaction log and the GTID corresponding to the full backup data, so that the data synchronization between the database to be migrated and the target database is realized.

Further, if it is determined that the data synchronization is completed, the technical library to be migrated stops the database service process, that is, stops the services related to data storage, such as data insertion, deletion, and modification.

Step 202: and if the data synchronization is determined to be completed, the database to be migrated transmits the received data access message to the target database in an agent mode.

Specifically, when step 202 is executed, the following steps may be adopted:

s2021: and starting proxy forwarding by the database to be migrated.

Specifically, a proxy forwarding application program, such as open source code TCP software Nginx or HAProxy, is installed in the database to be migrated. And the database to be migrated starts proxy forwarding through the proxy forwarding application program. Proxy forwarding includes transparent proxy forwarding and opaque proxy forwarding.

The transparent proxy forwarding refers to that after the data access message sent by the service equipment to the database to be migrated is proxied to the target database, the target database analyzes and obtains the address information of the service equipment, namely, the source access address.

The opaque proxy forwarding refers to that after the data access message agent, which is sent to the database to be migrated by the service device, is forwarded to the target database, the target database analyzes and obtains the address information of the database to be migrated, namely, the proxy address. The data access message forwarded by the proxy may be denied access by the target database due to improper data origin.

In one embodiment, transparent proxy forwarding may be initiated in the following manner:

# cat/proc/sys/net/ipv4/ip _ forward// see if transparent proxy forwarding is enabled; if the output value is 1, the starting is indicated, otherwise, the starting is not performed.

# echo 1>/proc/sys/net/ipv4/ip _ forward// initiate transparent proxy forwarding.

Therefore, proxy forwarding can be started, so that the received data access message can be proxied and forwarded to the target database in a proxy forwarding mode in the subsequent step.

S2022: and after the database to be migrated determines that the proxy forwarding deployment is successful, the received data access message is proxied and forwarded to the target database.

Specifically, when S2022 is executed, the following two ways may be adopted:

the first mode is as follows: and transparently forwarding the data access message received by the appointed network port of the database to be migrated to the target database by the database to be migrated through the proxy forwarding application program by adopting a transparent proxy forwarding mode, so that the target database performs message verification according to the source address information contained in the data access message.

Specifically, the database to be migrated monitors a local designated network port, for example, an 80 port, of the database to be migrated through an open source code TCP software, a proxy forwarding application such as Nginx or HAProxy, and when it is determined that the designated network port receives a data access message sent by other service equipment, the proxy forwarding the data access message to the target database. And after the target database receives the data access message forwarded by the agent, the data access message is subjected to message analysis to obtain a source access address contained in the data access message, and after the source access address is determined to be legal, the subsequent data processing operation is performed on the data access message.

Therefore, when the message is transparently forwarded, the target database can verify the validity of the message according to the source access address.

The second way is: and receiving a data access message which is sent by the service equipment and contains authorized access information by the database to be migrated based on the designated network port by adopting an opaque proxy forwarding mode, and forwarding the received data access message to the target database, so that the target database performs message verification according to the authorized access information contained in the data access message.

Specifically, each service device sends a data access message containing authorized access information to the database to be migrated. Monitoring a local appointed network port, such as an 80 port, of the database to be migrated by the proxy forwarding application program in the database to be migrated, and when the appointed network port is determined to receive the data access message sent by other service equipment, proxy forwarding the data access message to a target database. And after receiving the data access message forwarded by the agent, the target database analyzes the data access message to obtain authorized access information contained in the data access message, and after determining that the authorized access information is legal, the target database performs subsequent data processing operation on the data access message.

Therefore, when the opaque proxy is adopted for forwarding, the validity of the data access message can be verified through the authorized access information.

Step 203: and the database to be migrated sends an address modification notification message to each service device.

Specifically, the database to be migrated may determine, according to the historical access data, the service device that has historically accessed the database to be migrated, and send an address modification notification message to the determined service device.

Wherein, the address modification notification message contains the target address information of the target database.

Further, the target database may also send an address modification notification message to the service device sending the data access message after receiving the data access message.

And after receiving the address modification notification message, the service equipment acquires target address information contained in the address modification notification message and sends subsequent service data messages to a target database according to the target address information.

Fig. 3 is a schematic diagram illustrating an application scenario of database migration. The system comprises a TGW1, a TGW2, a plurality of business devices, a database to be migrated and a target database. Wherein, the database to be migrated is provided with a proxy forwarding application program.

Referring to fig. 4, a detailed implementation flowchart of a method for database migration is shown. With reference to the application scenario in fig. 3, the specific flow of the method is as follows:

step 400: and the database to be migrated acquires the full backup data after the latest backup.

Step 401: and the database to be migrated performs data synchronization between the database to be migrated and the target database according to the latest full backup data.

Step 402: and after the target database determines that the data synchronization is completed, sending a data synchronization completion response message to the database to be migrated.

Step 403: and the database to be migrated determines that the data synchronization is completed according to the received data synchronization completion response message.

Step 404: and after the data synchronization is determined to be completed by the database to be migrated, forwarding the received data access message to the target database by the agent.

In one embodiment, the business device sends a data access message to the database to be migrated via TGW 1. After the data synchronization of the database to be migrated is determined to be completed, monitoring the designated network port through the proxy forwarding application program, and when monitoring the data access message received through the TGW1, forwarding the received data access message to the target database in a proxy mode.

Further, after the database to be migrated determines that the data synchronization is completed and the proxy forwarding is successful, the database service process is stopped.

Step 405: and the database to be migrated sends an address modification notification message to each service device, so that each service device sends a service data message to the target database.

Specifically, the database to be migrated determines that proxy forwarding deployment is successful, determines a plurality of service devices accessing the database to be migrated, and sends an address modification notification message to the determined service devices.

Step 406: the target database receives the data access message forwarded by the agent.

Step 407: and the target database determines the source access address of the service equipment according to the received data access message.

Step 408: and the target database returns an address modification notification message to the corresponding service equipment according to the source access address.

Specifically, after determining that the proxy forwarding deployment is successful, the to-be-migrated database and/or the target database sends an address modification notification message to each service device.

Alternatively, the destination database may not send the address modification notification message.

Step 409: and the service equipment modifies the notification message according to the received address and sends the service data message to the target database.

Specifically, the target database receives a service data message sent by the service device through the TGW 2.

Further, after it is determined that all the related service devices realize database switching, the operation of the database devices to be migrated is stopped, and database arbitration and withdrawal are completed.

In practical application, because the service has a long history of leaving behind, all access sources cannot be combed, that is, all service devices cannot be determined, it is difficult to send address modification notification messages to all service devices at one time, and database switching of all service devices cannot be realized at one time, and because the service also has the gray scale, the sequence and the like of a release version, database switching cannot be guaranteed to be performed simultaneously by all service devices, so that double writing of a database is caused, and further data loss is caused. In the embodiment of the application, after the database service process is stopped, the data access message sent by each service device is still received, the data access message is proximately forwarded to the target database in a proxy forwarding mode, and the service device is informed of database switching at the same time, so that the proxy forwarding mode is adopted, database switching of all service devices is not required to be realized at one time, database double writing is not caused, the service device needing database switching can be continuously determined according to the received data access message within a period of migration time, and the operation of the database to be migrated is stopped until all relevant service devices are determined to realize database switching, and database withdrawal is completed.

Based on the same inventive concept, the embodiment of the present application further provides a device for database migration, and because the principle of the device and the apparatus for solving the problem is similar to that of a method for database migration, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.

Fig. 5 is a schematic structural diagram of a database migration apparatus according to an embodiment of the present application. An apparatus for database migration comprising:

an obtaining unit 500, configured to obtain latest full backup data, where the full backup data is obtained after a periodic data backup of a database to be migrated;

a synchronization unit 501, configured to perform data synchronization between the database to be migrated and the target database according to the latest full backup data;

a forwarding unit 502, configured to forward the received data access message to the target database by proxy if it is determined that data synchronization is completed;

a sending unit 503, configured to send an address modification notification message to each service device, so that each service device sends a service data message to the target database.

Preferably, the synchronization unit 501 is configured to:

sending the full backup data to a target database;

Preferably, the synchronization unit 501 is configured to:

Preferably, the forwarding unit 502 is configured to:

Preferably, the sending unit 503 is configured to:

Fig. 6 is a schematic structural diagram of a database migration apparatus according to an embodiment of the present application. An apparatus for database migration comprising:

a synchronization unit 601, configured to perform data synchronization according to the received data to be migrated of the database to be migrated;

a forwarding unit 602, configured to receive a data access message forwarded by an agent after the database to be migrated determines that data synchronization is completed, where the data access message is a message sent to the database to be migrated;

a returning unit 603, configured to return an address modification notification message to a service device that sends a data access message;

a receiving unit 604, configured to receive a service data message sent by a service device.

Fig. 7 shows a schematic configuration of a control device 7000. Referring to fig. 7, the control apparatus 7000 includes: a processor 7010, a memory 7020, a power supply 7030, a display unit 7040, and an input unit 7050.

The processor 7010 is a control center of the control apparatus 7000, connects the respective components by various interfaces and lines, and executes various functions of the control apparatus 7000 by running or executing software programs and/or data stored in the memory 7020, thereby monitoring the control apparatus 7000 as a whole.

In an embodiment of the present application, the processor 7010, when calling a computer program stored in the memory 7020, performs the method for database migration as provided by the embodiment shown in fig. 2.

Optionally, the processor 7010 may include one or more processing units; preferably, the processor 7010 may integrate an application processor, which handles primarily the operating system, user interfaces, applications, etc., and a modem processor, which handles primarily wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 7010. In some embodiments, the processor, memory, and/or memory may be implemented on a single chip, or in some embodiments, they may be implemented separately on separate chips.

The memory 7020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, and the like; the stored data area may store data created from the use of the control device 7000 and the like. In addition, the memory 7020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The control device 7000 also includes a power supply 7030 (e.g., a battery) for powering the various components, which may be logically coupled to the processor 7010 via a power management system that may be used to manage charging, discharging, and power consumption.

Display unit 7040 may be configured to display information input by a user or information provided to the user, and various menus of control apparatus 7000, and the like, and in the embodiment of the present invention, is mainly configured to display a display interface of each application in control apparatus 7000, and objects such as texts and pictures displayed in the display interface. The display unit 7040 may include a display panel 7041. The Display panel 7041 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The input unit 7050 may be used to receive information such as numbers or characters input by a user. The input unit 7050 may include a touch panel 7051 and other input devices 7052. Among other things, the touch panel 7051, also referred to as a touch screen, may collect touch operations by a user on or near the touch panel 7051 (e.g., operations by a user on or near the touch panel 7051 using any suitable object or attachment such as a finger, a stylus, etc.).

Specifically, the touch panel 7051 may detect a touch operation of a user, detect signals generated by the touch operation, convert the signals into touch point coordinates, transmit the touch point coordinates to the processor 7010, receive a command transmitted from the processor 7010, and execute the command. In addition, the touch panel 7051 can be implemented by various types such as resistive, capacitive, infrared, and surface acoustic wave. Other input devices 7052 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, power on and off keys, etc.), a trackball, a mouse, a joystick, and the like.

Of course, the touch panel 7051 may cover the display panel 7041, and when the touch panel 7051 detects a touch operation on or near the touch panel 7051, the touch operation is transmitted to the processor 7010 to determine the type of the touch event, and then the processor 7010 provides a corresponding visual output on the display panel 7041 according to the type of the touch event. Although in fig. 7, the touch panel 7051 and the display panel 7041 are two separate components to implement the input and output functions of the control device 7000, in some embodiments, the touch panel 7051 and the display panel 7041 may be integrated to implement the input and output functions of the control device 7000.

The control device 7000 may also comprise one or more sensors, such as pressure sensors, gravitational acceleration sensors, proximity light sensors, etc. Of course, the control device 7000 may also comprise other components such as a camera, which are not shown in fig. 7 and will not be described in detail, since they are not components used in the embodiments of the present application.

Those skilled in the art will appreciate that fig. 7 is merely an example of a control device and is not intended to be limiting and may include more or less components than those shown, or some components in combination, or different components.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be substantially implemented or portions thereof that contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method of database migration in any of the method embodiments described above.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or partially contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a control device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A method for migrating a database is applied to the database to be migrated, and comprises the following steps:

acquiring full backup data of the latest backup of the database to be migrated;

if the data synchronization is determined to be completed, forwarding the received data access message agent to the target database;

2. The method of claim 1, wherein synchronizing data between the database to be migrated and the target database based on the most recent full amount of backup data comprises:

sending the full backup data to the target database;

if the full backup data is determined to be synchronously completed, acquiring a transaction log and a global transaction identifier corresponding to the full backup data, wherein the transaction log comprises each database operation record and the corresponding global transaction identifier;

determining a database operation record corresponding to the unsynchronized data recorded in the transaction log according to the global transaction identifier corresponding to the full amount of backup data;

3. The method of claim 2, wherein sending the full amount of backup data to the target database comprises:

acquiring target address information and target authorization information of the target database;

and sending the full backup data to the target database according to the target address information and the target authorization information.

4. The method of claim 1, wherein proxying the received data access message to the target database if it is determined that data synchronization is complete, comprises:

transparently forwarding the data access message received by the appointed network port of the database to be migrated to the target database, so that the target database performs message verification according to the source address information contained in the data access message; alternatively, the first and second electrodes may be,

and receiving a data access message which is sent by the service equipment and contains authorized access information based on the designated network port, and forwarding the received data access message to the target database, so that the target database performs message verification according to the authorized access information contained in the data access message.

5. The method of any one of claims 1-4, further comprising:

6. A method for database migration, applied to a target database, includes:

and receiving a service data message sent by the service equipment.

7. An apparatus for database migration, comprising:

the acquisition unit is used for acquiring the latest full backup data, and the full backup data is acquired after the periodic data of the database to be migrated is backed up;

the forwarding unit is used for forwarding the received data access message to the target database by proxy if the data synchronization is determined to be completed;

and the sending unit is used for sending an address modification notification message to each service device so that each service device sends a service data message to the target database.

8. An apparatus for database migration, comprising:

the forwarding unit is used for receiving a data access message forwarded by an agent after the database to be migrated determines that data synchronization is completed, wherein the data access message is a message sent to the database to be migrated;

a returning unit, configured to return an address modification notification message to the service device that sends the data access message;

9. A control device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1-5 or 6 are implemented when the program is executed by the processor.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5 or 6.