CN116340425A - Data management method, device, medium and computing equipment of MHA (mobile high-definition architecture) - Google Patents

Abstract

The invention provides a data management method, a device, a medium and a computing device of an MHA high availability architecture, wherein the method comprises the steps of identifying a master database and each slave database of the MHA high availability architecture; creating a storage disk corresponding to the master database and the slave database, wherein the storage disk is a disk volume which can be unloaded and mounted at any time; and storing the binlog logs of the master database and the slave database into corresponding storage disks. The invention realizes the synchronization of master-slave data by placing binlog of the master database and the standby database switched in a peer-to-peer manner on the external storage disk and hanging the binlog on the host in a mode of mounting the volume instead of being placed on the respective local disk, truly realizes zero loss of data and confirms the integrity of production data.

Description

Data management method, device, medium and computing equipment of MHA (mobile high-definition architecture)

Technical Field

The invention relates to the technical field of computers, in particular to a data management method, a device, a medium and computing equipment of an MHA high-availability architecture.

Background

With the large-scale use of MySQL, which is an open source database, high availability is also becoming increasingly important. In order to guarantee rpo=0 of data as much as possible (data loss rate at the time of disaster), the industry also derives a wide variety of high availability architectures, the more commonly used one being MHA. The original MHA is high in availability, under normal conditions, the MySQL main database is abnormally unavailable, and when the MHA is switched, the new main database automatically performs binlog log compensation, so that the data of the new main database is consistent with that of the main database.

However, if the operating system where the MySQL master database is located is down, the MHA management node cannot log on to the operating system of the master database through the SSH, and if the new master database is delayed from the old master database, part of data is lost at this time, so that the master data and the slave data are inconsistent.

Disclosure of Invention

In view of the foregoing, the present invention proposes a data management method, apparatus, medium and computing device of MHA high availability architecture that overcomes or at least partially solves the foregoing problems.

According to one aspect of the present invention there is provided a data management method for an MHA high availability architecture, the method comprising:

identifying a master database and each slave database of the MHA high availability architecture;

creating a storage disk corresponding to the master database and the slave database, wherein the storage disk is a disk volume which can be unloaded and mounted at any time;

and storing the binlog logs of the master database and the slave database into corresponding storage disks.

Optionally, the creating a storage disk corresponding to the master database and the slave database includes:

mounting a first storage disk with the same path on each instance of the master database and the slave database;

a second storage disk is mounted on each instance of the primary database.

Optionally, the storing the binlog logs of the master database and the slave database in the corresponding storage disk includes:

storing the binlog logs of each instance of the slave database into a corresponding first storage disk;

and storing the binlog logs of each instance of the main database into corresponding first storage disk and second storage disk.

Optionally, using a binlog server technology to store a binlog log generated by a main database in the first storage disk to a corresponding first storage disk and a second storage disk;

the number and the content of the binlog logs in the second storage disk are the same.

Optionally, after the storing the binlog logs of the master database and the slave database in the corresponding storage disk, the method further includes:

when the MHA monitoring process finds that the main database is down and can not be ssh logged in, determining a standby main database selected in advance from the auxiliary databases, and mounting the second storage disk on the standby main database;

acquiring a first data file of the standby main database based on the first storage disk, and acquiring a second data file of the main database based on the second storage disk;

and comparing the first data file with the second data file, generating a difference data file, and applying the difference data file to the standby main database.

Optionally, the mounting the second storage disk onto the standby main database includes: and mounting the second storage disk on the standby main database through an operating system mount command.

Optionally, after the applying the difference data file to the standby main database, the method further includes:

and selecting a new standby main database from the auxiliary databases, and starting a main and standby copying process.

According to another aspect of the present invention, there is provided a data management apparatus of MHA high availability architecture, characterized in that the apparatus comprises:

the database identification module is used for identifying a master database and each slave database of the MHA high-availability architecture;

the storage disk creating module is used for creating a storage disk corresponding to the master database and the slave database, wherein the storage disk is a disk volume which can be unloaded and mounted at any time;

and the data storage module is used for storing the binlog logs of the master database and the slave database into corresponding storage disks.

Optionally, the storage disk creation module is further configured to: mounting a first storage disk with the same path on each instance of the master database and the slave database;

a second storage disk is mounted on each instance of the primary database.

Optionally, the data storage module is further configured to: storing the binlog logs of each instance of the slave database into a corresponding first storage disk;

and storing the binlog logs of each instance of the main database into corresponding first storage disk and second storage disk.

Optionally, the data storage module 430 is configured to store the binlog log generated by the primary database in the first storage disk to the corresponding first storage disk and second storage disk using a binlog server technology;

the number and the content of the binlog logs in the second storage disk are the same.

Optionally, the data management device of the MHA high availability architecture of the present invention may further include a failover module for: when the MHA monitoring process finds that the main database is down and can not be ssh logged in, determining a standby main database selected in advance from the auxiliary databases, and mounting the second storage disk on the standby main database;

acquiring a first data file of the standby main database based on the first storage disk, and acquiring a second data file of the main database based on the second storage disk;

and comparing the first data file with the second data file, generating a difference data file, and applying the difference data file to the standby main database.

Optionally, the failover module is further configured to: and mounting the second storage disk on the standby main database through an operating system mount command.

Optionally, the failover module is further configured to: and selecting a new standby main database from the auxiliary databases, and starting a main and standby copying process.

According to another aspect of the invention there is also provided a computer readable storage medium for storing program code for performing the data management method of the MHA high availability architecture of any one of the above.

According to another aspect of the present invention, there is also provided a computing device including a processor and a memory:

the memory is used for storing program codes and transmitting the program codes to the processor;

the processor is configured to perform the data management method of the MHA high availability architecture of any one of the above according to instructions in the program code.

The invention provides a data management method, a device, a medium and a computing device of an MHA high availability architecture. Even if the server is abnormally down, the disaster can not lose any data, the real zero loss of data is realized, the integrity of production data is confirmed, and the data property safety of users is ensured.

The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.

The above, as well as additional objectives, advantages, and features of the present invention will become apparent to those skilled in the art from the following detailed description of a specific embodiment of the present invention when read in conjunction with the accompanying drawings.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:

FIG. 1 shows a flow diagram of a data management method of an MHA high availability architecture according to an embodiment of the present invention;

FIG. 2 shows a schematic of an MHA high availability architecture upgrade according to an embodiment of the present invention;

FIG. 3 shows a database failover process schematic diagram in accordance with an embodiment of the invention;

FIG. 4 shows a schematic diagram of a data management device architecture of an MHA high availability architecture according to an embodiment of the present invention;

fig. 5 shows a schematic diagram of a data management device structure of an MHA high availability architecture according to another embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

The embodiment of the invention provides a data management method of an MHA high availability architecture, as shown in fig. 1, the data management method of the MHA high availability architecture provided by the embodiment of the invention at least may include the following steps S101 to S103.

S101, identifying a master database and each slave database of the MHA high-availability architecture.

MHA (Master High Availability) is a mature and open-source MySQL high-availability program, which realizes the function of automatically performing single failover after MASTER is down in a MySQL MASTER-slave environment, is written by perl language, and is convenient to install and simple to use. MHA consists of two parts, MHA Manager (management Node) and MHA Node (data Node). The MHA Manager (management node) may be deployed on an independent machine to manage multiple master-slave clusters, or may be deployed on a slave node. MHA Manager mainly runs some tools, such as the masterha_manager tool, to automatically monitor MySQL Master and implement Master failover, and other tools to manually implement Master failover, online Master transfer, connection check, etc. MHA Node

MHA Node (data Node) running on each MySQL server MHA Manager will periodically probe the master Node in the cluster, and when the master fails, it can automatically promote the slave of the latest data to a new master, and then redirect all other slave to the new master. The entire failover process is completely transparent to the application.

The MHA high-availability architecture is a Master node and a Slave node, namely a Master node is used as a Master database, and two Slave nodes are used as Slave databases to realize active data synchronization operation. As shown in fig. 2, in this embodiment, database a is a master database, database B and database C are slave databases, and database B is a backup master database.

S102, creating storage disks corresponding to the master database and the slave database, wherein the storage disks are disk volumes capable of being unloaded and mounted at any time.

The storage disk of the embodiment is a disk capable of being unloaded and mounted at any time, and particularly, the MHA tool source code can be modified through optimization to realize automatic unloading and mounting of the disk, and then the journal is replenished. The mount is that the access is realized by a certain directory related to the root file system outside the root file system, and the process of releasing the association relationship is called unloading. When the mounting disc is created, the method can modify and adjust source codes of the traditional MHA high-availability architecture.

S103, storing the binlog logs of the master database and the slave database into corresponding storage disks. The binlog, i.e., binary log, records all operations on MySQL database operations in the form of events. After the mount storage disks of the master database and the slave database are created, the binlog logs generated by the master database and the slave database can be stored in the mount storage disk created in the step S102, and master-slave synchronization is realized while data storage is performed.

The data management method of the MHA high-availability architecture provided by the embodiment carries out secondary development and architecture upgrading on the original MHA high-availability architecture, the binlog of the main database and the standby database which are switched in a peer-to-peer mode is placed on the external storage disk, and is hung on a host in a mode of hanging a volume instead of being placed on a local disk, so that synchronization of master-slave data is realized. Even if the server is abnormally down, the disaster can not lose any data, the real zero loss of data is realized, the integrity of production data is confirmed, and the data property safety of users is ensured.

The step S102 mentioned above refers to creating a storage disk corresponding to the master database and the slave database, and in some embodiments, two storage disks may be created: in this embodiment, a first storage disk D1 and a second storage disk D2 are denoted. That is, when creating a disk that can be mounted and dismounted at any time, first, a first disk having the same path is mounted on each instance of the master database and the slave database. Second, a second storage disk is mounted on each instance of the primary database.

Further, the storing the binlog logs of the master database and the slave database in the corresponding storage disk in step S103 may include: storing the binlog logs of each instance of the slave database into a corresponding first storage disk; the binlog logs of each instance of the master database are stored into corresponding first storage disk (D1 disk) and second storage disk (D2 disk).

That is, whether the master database or the slave database, a storage disk with the same path is mounted on each instance of mysql, which is denoted as a first storage disk D1 in this embodiment; in addition, a storage disk is required to be mounted on the main database, and this embodiment is named as a second storage disk D2, (1) detailed disk symbol information is as follows:

(1) the disk information of the main database A,/mybinlog is the mounting point of the D1 disk, the binlog log of the main database is stored,/mybinlog_ mha is the mounting point of the D2 disk, and the content of the mybinlog is the same as that of the binlog of the main database as the binlog server.

#df-Th

/dev/mapper/vgadg-lv_vgadg ext4 123G 33G 84G 29％/mybinlog

/dev/mapper/vgpri-lv_vgpri ext4 123G 25G 92G 22％/mybinlog_mha

(2) Disk information of the backup library B,/mybinlog is the mounting point of the D1 disk

#df-Th

/dev/mapper/vgpri-lv_vgpri ext4 123G 31G 87G 26％/mybinlog

In some embodiments, a binlog server technique is used to store a binlog log generated by a master database in the first storage disk to the corresponding first storage disk and second storage disk; the number and the content of the binlog logs in the second storage disk are the same.

In this example, binlog for each instance of mysql is stored on the D1 disk. In addition, the binlog server technology is used, the binlog generated by the main database A is put on the D2 disk one more, the number and the content of the binlog logs of the D1 disk and the D2 disk of the main database A are guaranteed to be identical, and therefore, when the main database A is down, after the D2 disk is mounted on a new main, the new main B can access the binlog of the original main A, and the next operation is performed.

The binlog server technology is a method of binlog replication, because master-slave replication often causes some partial delay due to various factors, and when a master fails and cannot ssh, a slave library may lose partial data and cannot synchronize. At this time, a server is started up as a binlog server, and a binlog log is pulled from the main database in real time, so that after the master is down, other slave libraries can recover data from the binlog server.

MySQL Binlog Server the connection of an IO thread of slave with a main database is simulated in a daemon process mode by using a mysqlbinglog command, so that the binlog of the main database can be conveniently and immediately synchronized, and the problem that data in the period from the last backup to the completion of the next backup in a timed backup strategy is easy to lose is solved. In this embodiment, the binlog server technology is used to connect to the first storage disk D1, so that a binlog of the main database can be obtained from the D1 disk in real time, and stored in the D2 disk.

In practical applications, the main data may reply to the situation of the downed amateur, so in some embodiments, the step S103 may further include a process of performing failover, which is specifically implemented as follows:

when the MHA monitoring process finds that the main database is down and can not be ssh logged in, the standby main database preselected in the auxiliary database is determined, and the second storage disk is mounted on the standby main database. Mounting the second storage disk onto the standby primary database includes: and mounting the second storage disk on the standby main database through an operating system mount command.

In other words, when the main database a is down in the system level, the MHA monitor process discovers that the main database a is dead and cannot log in ssh, the D2 disc is directly mounted on the new main database B.

A second step of acquiring a first data file of the standby main database based on the first storage disk and acquiring a second data file of the main database based on the second storage disk;

and thirdly, comparing the first data file with the second data file, generating a difference data file, and applying the difference data file to the standby main database.

In the embodiment, the binlog in the D2 disc is compared with the binlog in the D1 disc in the database B, and the corresponding difference file is generated by the difference binlog part between the database a and the database B and is applied to the new main database B, so that after the old main database a dies, the excessive data is synchronized to the database B, and the consistency of the data of the database a and the database B can be ensured. As shown in fig. 3, the present embodiment performs the following steps for a detailed description of the database failover process.

The MHA monitor process is a process that starts a resident background on the MHA manager management node; the process is as follows,

# monitoring process

$ps axu|grep mbsp00

root 13345 0.0 0.0 197260 19672S Jul14 65:33 perl/usr/bin/masterha_manager--global_conf＝/paic/mysql/mha/masterha_default.conf--conf＝/paic/mysql/mha/mbsp00_3308/app1.conf--ignore_fail_on_start--ignore_last_failover--wait_on_monitor_error＝60--wait_on_failover_error＝60

(2) The identification logic of the monitoring process depends on the setting of ping_type in mha configuration file app1.Conf, ping_type=connect is used in production, that is, when the database is logged in each time and select 1 as Value is executed to detect whether the database survives, connection needs to be re-created each time, if the connection is tried for 4 times (or other set times) or can not be created successfully, the monitoring process considers that the database is down, service can not be provided, and the failover switch is triggered.

(3) When the main database A dies, the D2 disk is directly mounted on the new main database B, a command and a mode are mounted, a code segment is intercepted, mainly a function stop_umount_ mha () is called, and after the main database is failover, the/mybinlog_ mha disk is mounted on the new main database through an operating system mount command.

(4) After the D2 disk of the original main database A is mounted on the new main data B, a show slave status command is used for checking whether the master_Log_File & & read_Master_Log_Pos and the Relay_Master_Log_File & & Exec_Master_Log_Pos are consistent. If the two types of information are consistent, no delay exists, and the backup library is directly switched without compensating binlog; if the two types of the binary Log logs are inconsistent, the standby database B has delay, the difference content of the binary Log logs which are not yet available and applied to the database B is found by comparing the difference sizes of the parameter Master_Log_File & & read_Master_Log_Pos and the value of Relay_Master_Master_Log_Pos, and finally the difference content is imported into a new main database B through a mysqlbinglog command to compensate the difference Log of the new standby database and the main database.

Fourth step: and selecting a new standby main database from the auxiliary databases, and starting a main and standby copying process.

In this embodiment, the original slave database B is set as the standby master database, when the original master database a is down, the database B may be used as a new master database, meanwhile, the database C may be used as a standby master database of the new master database, the database C may be hung behind the database B, and after the master-slave replication process is started, the binlog of the B may be started to complete master-slave synchronization tasks of other slave databases.

The step of hanging the database C behind the database B is realized by a mysql native command change master to, an ip and a port of a new master B are designated, and the account and the password of a user are copied, for example, change master to master _host= '21.64.153.188', master_port=3308, master_user= 'dbsync', master_password= 'SfCd5nwGJ5f9xzrvE6 wrj', master_auto_position=1;

start slave；

after the database B is used as a new main database, besides the original first storage disk D1 is mounted, a second storage disk D2 can be mounted on the database B, and the binlog logs of the database B stored in the D1 disk can be synchronized into the D2 disk by adopting a binlog server technology, so that the data synchronization of the binlog logs in the new main database is realized, and further, the data backup of a new round is realized.

In practical application, the master-slave replication often causes some part delay due to various factors, and by simply adjusting the MHA architecture, even when the master database fails and cannot be SSH, the backup database can perform log compensation through binlog of the master database in the shared storage disk, so that data consistent with the original master database is ensured, the problem of pain caused by data loss due to delay switching of the master database and the backup database downtime and incapability of SSH login is solved, and the method provided by the embodiment has more pertinence.

Based on the same inventive concept, the embodiment of the present invention further provides a data management device of an MHA high availability architecture, as shown in fig. 4, where the data management device of an MHA high availability architecture provided by the embodiment of the present invention may include: database identification module 410, storage disk creation module 420, and data storage module 430, wherein each module functions as follows:

a database identification module 410 for identifying a master database and each slave database of the MHA high availability architecture;

a storage disk creating module 420, configured to create a storage disk corresponding to the master database and the slave database, where the storage disk is a disk volume that can be unloaded and mounted at any time;

and the data storage module 430 is configured to store the binlog logs of the master database and the slave database into corresponding storage disks.

In some embodiments, the storage disk creation module 420 may also be configured to: mounting a first storage disk with the same path on each instance of the master database and the slave database;

a second storage disk is mounted on each instance of the primary database.

In some embodiments, the data storage module 430 may also be used to: storing the binlog logs of each instance of the slave database into a corresponding first storage disk;

and storing the binlog logs of each instance of the main database into corresponding first storage disk and second storage disk.

In some embodiments, the data storage module 430 may be further configured to store the binlog logs generated by the primary database in the first storage disk to the corresponding first storage disk and second storage disk using a binlog server technique;

the number and the content of the binlog logs in the second storage disk are the same.

In some embodiments, as shown in fig. 5, the data management apparatus of the MHA high availability architecture of embodiments of the invention may further include a failover module 440.

The failover module 440 is configured to: when the MHA monitoring process finds that the main database is down and can not be ssh logged in, determining a standby main database selected in advance from the auxiliary databases, and mounting the second storage disk on the standby main database;

acquiring a first data file of the standby main database based on the first storage disk, and acquiring a second data file of the main database based on the second storage disk;

and comparing the first data file with the second data file, generating a difference data file, and applying the difference data file to the standby main database.

In some embodiments, failover module 440 may also be configured to: and mounting the second storage disk on the standby main database through an operating system mount command.

In some embodiments, failover module 440 may also be configured to: and selecting a new standby main database from the auxiliary databases, and starting a main and standby copying process.

An embodiment of the present invention also provides a computer readable storage medium for storing a program code for executing the data management method of the MHA high availability architecture described in the above embodiment.

An embodiment of the present invention also provides a computing device including a processor and a memory: the memory is used for storing program codes and transmitting the program codes to the processor; the processor is configured to execute the data management method of the MHA high availability architecture according to the above embodiment according to the instructions in the program code.

It will be clear to those skilled in the art that the specific working processes of the above-described systems, devices, modules and units may refer to the corresponding processes in the foregoing method embodiments, and for brevity, the description is omitted here.

In addition, each functional unit in the embodiments of the present invention may be physically independent, two or more functional units may be integrated together, or all functional units may be integrated in one processing unit. The integrated functional units may be implemented in hardware or in software or firmware.

Those of ordinary skill in the art will appreciate that: the integrated functional units, if implemented in software and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in essence or in whole or in part in the form of a software product stored in a storage medium, comprising instructions for causing a computing device (e.g., a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present invention when the instructions are executed. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a read-only memory (ROM), a random-access memory (RAM), a magnetic disk, or an optical disk, etc.

Alternatively, all or part of the steps of implementing the foregoing method embodiments may be implemented by hardware (such as a personal computer, a server, or a computing device such as a network device) associated with program instructions, where the program instructions may be stored on a computer-readable storage medium, and where the program instructions, when executed by a processor of the computing device, perform all or part of the steps of the method according to the embodiments of the present invention.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all technical features thereof can be replaced by others within the spirit and principle of the present invention; such modifications and substitutions do not depart from the scope of the invention.

Claims (10)

1. A method of data management for an MHA high availability architecture, the method comprising:

identifying a master database and each slave database of the MHA high availability architecture;

creating a storage disk corresponding to the master database and the slave database, wherein the storage disk is a disk volume which can be unloaded and mounted at any time;

and storing the binlog logs of the master database and the slave database into corresponding storage disks.

2. The method of claim 1, wherein the creating a storage disk corresponding to the master database and slave database comprises:

mounting a first storage disk with the same path on each instance of the master database and the slave database;

a second storage disk is mounted on each instance of the primary database.

3. The method of claim 2, wherein storing the binlog logs of the master and slave databases into corresponding storage disks comprises:

storing the binlog logs of each instance of the slave database into a corresponding first storage disk;

and storing the binlog logs of each instance of the main database into corresponding first storage disk and second storage disk.

4. The method of claim 3, wherein the binlog logs generated by the master database in the first storage disk are stored to the corresponding first storage disk and second storage disk using a binlog server technique;

the number and the content of the binlog logs in the second storage disk are the same.

5. A method according to claim 3, wherein after said storing the binlog logs of the master and slave databases in the corresponding storage disks, the method further comprises:

when the MHA monitoring process finds that the main database is down and can not be ssh logged in, determining a standby main database selected in advance from the auxiliary databases, and mounting the second storage disk on the standby main database;

acquiring a first data file of the standby main database based on the first storage disk, and acquiring a second data file of the main database based on the second storage disk;

and comparing the first data file with the second data file, generating a difference data file, and applying the difference data file to the standby main database.

6. The method of claim 5, wherein said mounting said second storage disk onto said standby primary database comprises: and mounting the second storage disk on the standby main database through an operating system mount command.

7. The method of claim 5, wherein after said applying said difference data file to said standby master database, said method further comprises:

and selecting a new standby main database from the auxiliary databases, and starting a main and standby copying process.

8. A data management apparatus for an MHA high availability architecture, the apparatus comprising:

the database identification module is used for identifying a master database and each slave database of the MHA high-availability architecture;

the storage disk creating module is used for creating a storage disk corresponding to the master database and the slave database, wherein the storage disk is a disk volume which can be unloaded and mounted at any time;

and the data storage module is used for storing the binlog logs of the master database and the slave database into corresponding storage disks.

9. A computer readable storage medium for storing program code for performing the data management method of the MHA high availability architecture of any one of claims 1 to 7.

10. A computing device, the computing device comprising a processor and a memory:

the memory is used for storing program codes and transmitting the program codes to the processor;

the processor is configured to perform the data management method of the MHA high availability architecture of any one of claims 1 to 7 according to instructions in the program code.

Images