CN110377664B - Data synchronization method, device, server and storage medium - Google Patents

Data synchronization method, device, server and storage medium Download PDF

Info

Publication number
CN110377664B
CN110377664B CN201910662737.2A CN201910662737A CN110377664B CN 110377664 B CN110377664 B CN 110377664B CN 201910662737 A CN201910662737 A CN 201910662737A CN 110377664 B CN110377664 B CN 110377664B
Authority
CN
China
Prior art keywords
database
standby
time
synchronization
main
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910662737.2A
Other languages
Chinese (zh)
Other versions
CN110377664A (en
Inventor
刘永松
朱翀
潘安群
雷海林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Tencent Cloud Computing Beijing Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201910662737.2A priority Critical patent/CN110377664B/en
Publication of CN110377664A publication Critical patent/CN110377664A/en
Application granted granted Critical
Publication of CN110377664B publication Critical patent/CN110377664B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273—Asynchronous replication or reconciliation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/80—Database-specific techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a data synchronization method, a data synchronization device, a server and a storage medium, and belongs to the technical field of databases. The method comprises the following steps: the method comprises the steps of obtaining synchronization time consumption of a main database and at least one standby database in a database system, wherein the synchronization time consumption is used for representing time consumption required by each stage when data synchronization is carried out between the main database and any one of the standby databases, switching a current system synchronization mode to a target system synchronization mode when the synchronization time consumption is larger than a first target time length, and writing an execution result of a database instruction into a disk of the main database on the premise of not requiring synchronous response of any one of the standby databases to the database instruction when the main database receives the database instruction in the target system synchronization mode. When receiving the database instruction, the main database does not need to wait for the response of the standby database, and the execution result is directly written into the disk, so that the main database can execute the database instruction faster, and the processing performance of the database system is improved.

Description

Data synchronization method, device, server and storage medium
Technical Field
The present invention relates to the field of database technologies, and in particular, to a data synchronization method, apparatus, server, and storage medium.
Background
With the rapid development of internet technology, the dependence of enterprises on information systems is higher and higher, and a database serves as a core of the information systems and plays an important role. In some industries with high data reliability requirements, such as banking, securities, telecommunications, e-commerce and the like, if the database is unexpectedly down or data is lost, serious loss is caused to the enterprise, so the enterprise needs to make detailed database backup and disaster recovery strategies according to the service type of the enterprise. When the database is backed up, the database instructions executed in the main database need to be synchronized to the standby database to ensure that the data in the standby database is consistent with the data in the main database.
At present, paxos (paxos algorithm, a distributed consistency algorithm) protocol is generally used to ensure data consistency in the primary database and the backup database. The principle of paxos protocol is to add a paxos layer to the database layer, where the main database stores executed database instructions in binlog (binary log file) logs, and the paxos layer distributes the binlog logs of the main database to other standby databases. After receiving the binlog log, each standby database executes the database command stored in the binlog log, and after the execution is finished, the main database sends a response to the main database.
The technical scheme has the problem that when a paxos layer fails or a network fluctuates, the primary database may not receive a response returned by the secondary database, so that a database instruction executed by the primary database cannot be executed or is executed slowly, and the processing performance of the database system is affected.
Disclosure of Invention
The embodiment of the invention provides a data synchronization method, a data synchronization device, a server and a storage medium, which are used for solving the problem that a main database possibly cannot receive a response returned by a standby database when the existing database is synchronized, so that a database instruction executed by the main database cannot be executed or is executed slowly, and the processing performance of a database system is influenced. The technical scheme is as follows:
in one aspect, a data synchronization method is provided, including:
acquiring the synchronization time consumption of a main database and at least one standby database in a database system, wherein the synchronization time consumption is used for representing the time consumption required by each stage when data synchronization is carried out between the main database and any standby database;
when the synchronization time consumption is larger than the first target time length, switching the current system synchronization mode to a target system synchronization mode;
and in the target system synchronization mode, when the main database receives a database instruction, writing an execution result of the database instruction into a disk of the main database on the premise of not requiring synchronous response of any standby database to the database instruction.
In another aspect, a data synchronization apparatus is provided, including:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring the synchronization time consumption of a main database and at least one standby database in a database system, and the synchronization time consumption is used for representing the time consumption required by each stage when data synchronization is performed between the main database and any standby database;
the first switching module is used for switching the current system synchronization mode to a target system synchronization mode when the synchronization time consumption is longer than a first target time length;
and the writing module is used for writing the execution result of the database instruction into a disk of the main database under the condition that synchronous response of any standby database to the database instruction is not needed when the main database receives the database instruction in the target system synchronous mode.
In an optional implementation manner, the first obtaining module is further configured to obtain, in the database system, first periodic consumed time required by each stage when data synchronization is performed between the main database and one standby database, and use a sum of the first periodic consumed time as single synchronization consumed time of the database system; and taking the average value of a plurality of single synchronization time consumptions of the database system as the synchronization time consumption.
In another optional implementation manner, the first obtaining module is further configured to obtain a wait time consumed for the master database to execute the database instruction, where the wait time is a time consumed for the master database to wait for a backup database to return a synchronization response, and a first write time is a time consumed for the master database to write an execution result of the database instruction into a disk of the master database; respectively determining the weights of the waiting consumed time and the first writing consumed time; and taking the weight sum of the waiting time consumption and the first writing time consumption as the single synchronization time consumption of the database system.
In another optional implementation manner, the apparatus further includes:
a second obtaining module, configured to obtain execution time consumption of the standby database for each standby database in the at least one standby database, where the execution time consumption is used to indicate time consumption required by the standby database to complete the database instruction;
the second switching module is used for switching the current stand-alone synchronization mode of the standby database to a target stand-alone synchronization mode when the execution time consumption is longer than a second target time; and in the target single machine synchronization mode, the main database does not need to wait for the synchronization response of the standby database.
In another optional implementation manner, the second obtaining module is further configured to obtain second stage-specific consumed time required by each stage when the database completes the database instruction, and take a sum of the second stage-specific consumed time as a consumed time for the database to complete single execution of the database instruction; and taking the average value of the plurality of single execution time consumptions of the database as the execution time consumption.
In another optional implementation manner, the second obtaining module is further configured to obtain communication consumed time spent when the standby database completes the database instruction and second writing consumed time, where the communication consumed time is consumed time required for the standby database to communicate with the main database, and the second writing consumed time is consumed time required for the standby database to write the execution result of the database instruction into a disk of the standby database; respectively determining the weight of the communication elapsed time and the weight of the second writing elapsed time; and taking the weight sum of the communication elapsed time and the second writing elapsed time as the single execution elapsed time of the database.
In another optional implementation manner, the apparatus further includes:
and the third switching module is used for switching the current system synchronization mode to the target system synchronization mode when all the at least one standby database in the database system is switched to the target stand-alone synchronization mode.
In another optional implementation manner, the apparatus further includes:
a fourth switching module, configured to switch the current system synchronization mode to a default system synchronization mode when the synchronization time consumption is less than the first target time length;
in the default system synchronization mode, when the main database receives a database instruction, the main database executes the database instruction, the database instruction is sent to the at least one standby database, and when the main database receives a synchronization response of any standby database, an execution result of the database instruction is written into a disk of the main database.
In another optional implementation manner, the apparatus further includes:
the third acquisition module is used for acquiring resource occupation information of a database server where the database system is located, wherein the database server comprises at least two database systems;
and the allocation module is used for allocating available hardware resources to the database system according to the resource occupation information, wherein the size of the available hardware resources is related to the load state of the database system.
In another optional implementation manner, the allocation module is further configured to obtain a resource configuration file of each database system in the database server, where the resource configuration file is used to indicate a size of a hardware resource that can be used by each database system; when the hardware resources of the database server are not completely occupied, allocating available hardware resources to the database systems according to the load state of each database system, wherein the size of the available hardware resources can exceed the size of the hardware resources indicated by the resource configuration files of the database systems; and when the hardware resources of the database server are completely used, distributing the available hardware resources for the database system according to the resource configuration file.
In another optional implementation manner, the allocation module is further configured to determine a priority of each database system in the database server; and when the total resource occupation amount of the database server is determined to exceed a target occupation threshold value according to the resource occupation information, switching the system synchronization mode of at least one low-priority database system in the database server to the target system synchronization mode.
In another optional implementation manner, the apparatus further includes:
a fourth obtaining module, configured to obtain running state information of the master database and the at least one standby database in the database system, where the running state information is used to indicate whether the master database and the at least one standby database operate normally;
the determining module is used for determining the fault reason causing the abnormal operation of the master database when the master database operates abnormally;
and the fifth switching module is used for switching the main database to a standby database according to the fault reason and switching the standby database in the at least one standby database to a new main database.
In another optional implementation manner, the fifth switching module is further configured to determine a fault type corresponding to the fault cause; when the fault type is not a target fault type, switching the main database into a standby database, and switching a standby database in the at least one standby database into a new main database, wherein the target fault type is a fault type which can be solved after the standby database in the at least one standby database is switched into the new main database; when the fault type is a target fault type, starting a switchover-free mechanism, wherein the switchover-free mechanism is used for indicating that the main database is kept unchanged when a fault error reason of the target fault type is detected in a target switchover-free time.
In another aspect, a server is provided, which includes a processor and a memory for storing program code, which is loaded and executed by the processor to implement the operations performed in the data synchronization method in the embodiments of the present invention.
In another aspect, a storage medium is provided, in which program codes for executing the data synchronization method in the embodiment of the present invention are stored.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, the synchronization time consumption of a main database and at least one standby database in a database system is obtained, the synchronization time consumption is used for representing the time consumption required by each stage when data synchronization is carried out between the main database and any standby database, when the synchronization time consumption is longer than a first target time, the current system synchronization mode is switched to a target system synchronization mode, and in the target system synchronization mode, when the main database receives a database instruction, the execution result of the database instruction is written into a disk of the main database on the premise of not needing synchronous response of any standby database to the database instruction. The synchronous consumed time of a main database and at least one standby database in the database system is obtained, the synchronous consumed time is compared with a first target time length, and when the synchronous consumed time is larger than the first target time length, the current system synchronization mode is switched to the target system synchronization mode, so that the main database does not need to wait for the response of the standby database when receiving a database instruction, and an execution result is directly written into a disk, so that the main database can execute the database instruction faster, and the processing performance of the database system is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a diagram of an implementation environment of a data synchronization method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of data synchronization provided in accordance with an exemplary embodiment;
FIG. 3 is a flow chart of another method of data synchronization provided in accordance with an exemplary embodiment;
FIG. 4 is a flow chart of another method of data synchronization provided in accordance with an exemplary embodiment;
fig. 5 is a block diagram of a data synchronization apparatus provided according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
The embodiment of the invention mainly relates to a scene for data synchronization in a database system. In order to ensure the security of data, a primary database and at least one backup database are usually provided in a database system. When the main database fails, one of the at least one standby database can be used as a new main database to replace the failed main database to provide services to the outside.
It should be noted that, in the database system, any operation on data may be implemented by a database instruction, for example, in the MYSQL system, operations such as adding, deleting, modifying, and the like may be implemented based on a corresponding SQL (Structured Query Language) statement.
In the embodiment of the invention, two system synchronization modes are involved, one is a strong synchronization mode, which means that after a main database receives a database instruction, when a synchronous response of any standby database to the database instruction is received, an execution result of the database instruction is written into a disk. The other mode is an asynchronous mode, which means that when the main database receives a database instruction, the execution result of the database instruction is written into a disk without synchronous response of any backup database to the database instruction.
The following describes a process of database synchronization by a database system using a strong synchronization mode: after receiving the database instruction, the main database executes the database instruction, and caches an execution result of the database instruction in a memory. The main database writes the database instruction into a corresponding binlog log after receiving the database instruction, sends the binlog log to a standby database in a database system, and the standby database analyzes the binlog respectively, acquires and executes the database instruction and writes an execution result into respective disks. And the standby database sends a synchronous response to the main database after writing the execution result into the disk, the synchronous response indicates that the data are synchronized, and the main database writes the execution result cached in the memory into the disk of the main database when receiving the synchronous response of any standby database. At this time, the database system completes one database instruction, which may be referred to as completing one data synchronization. For the SQL statement used for query, the master database does not store the SQL statement used for query into the binlog log because the query does not modify the data in the database.
The following describes how to switch between the primary database and the standby database when the primary database fails. When the main database fails and can not provide service, the standby database replaces the main database to provide service, the main database is switched to the standby database, and the database system selects one standby database from at least one standby database to be switched to the main database to provide a server for the outside.
Fig. 1 is a diagram of an implementation environment of a data synchronization method according to an embodiment of the present invention, and referring to fig. 1, includes a plurality of database servers 101 and a scheduling server 102.
The database server 101 may be connected to the scheduling server 102 through a wireless network or a wired network. The database server 101 may include at least one of a server, a plurality of servers, or a cloud computing server cluster. The database server machine 101 is used to deploy at least one database. Each database may be correspondingly deployed with an agent module, and the agent module is configured to collect relevant information of the database and a database server where the database is located, such as heartbeat information, communication time consumption, writing time consumption, resource occupation information of the database server, and the like, and may also be referred to as a data collection module. The scheduling server 102 is used for receiving the information sent by the agent module.
Fig. 2 is a flowchart of a data synchronization method provided according to an exemplary embodiment, as shown in fig. 2. The method comprises the following steps:
201. and the scheduling server acquires the synchronization time consumption of a main database and at least one standby database in the database system, and the synchronization time consumption is used for identifying the time consumption required by each stage when data synchronization is carried out between the main database and any standby database.
In the embodiment of the present invention, the synchronization time consumption may be time consumption required by each stage when data synchronization is performed between the master database and any one of the slave databases last time, and the synchronization time consumption may also be an average value of time consumption required by each stage when data synchronization is performed between the master database and any one of the slave databases for multiple times. The data synchronization between the main database and any one of the standby databases can comprise four stages: in the first stage, a main database is communicated with each standby database; in the second stage, each backup database executes a database instruction and writes an execution result into a disk; in the third stage, the main database receives synchronous response of the standby database; and in the fourth stage, the main database writes the execution result of the database instruction into a disk of the main database.
In an optional implementation manner, the scheduling server may obtain, in the database system, first periodic consumed time required by each stage when data synchronization is performed between the primary database and one backup database, use a sum of the first periodic consumed time as single synchronization consumed time of the database system, and use an average value of a plurality of single synchronization consumed times of the database system as the synchronization consumed time. The sum of the first-stage time consumption may be the sum of the first-stage time consumption to the fourth-stage time consumption. By taking the average value of the single synchronization time consumption as the synchronization time consumption, the overlong single synchronization time consumption caused by network jitter can be avoided, so that the synchronization time consumption can correctly reflect the time consumption required by each stage of data synchronization.
In an alternative implementation, the single synchronization time may be a weighted sum of the wait time spent by the master database executing the database instruction and the first write time. Correspondingly, the step of acquiring, by the scheduling server, the first stage time consumption required by each stage when performing data synchronization between the primary database and the secondary database, and taking the sum of the first stage time consumption as the single synchronization time consumption of the database system may be: the scheduling server can obtain a waiting time and a first writing time spent by the main database executing the database instruction, respectively determine weights of the waiting time and the first writing time, and take the weights of the waiting time and the first writing time as a single synchronization time of the database system. The weight sum of the waiting time consumption and the first writing time consumption is used as the single synchronization time consumption of the database system, so that the time consumption of each stage of data synchronization and the total time consumption of the database system in data synchronization can be reflected visually.
The waiting time is the time required for the master database to wait for one standby database to return a synchronization response, for example, the master database sends binlog log files to all the standby databases until any one of the standby databases returns a synchronization response. The first writing time is the time required for the master database to write the execution result of the database instruction into the disk of the master database. The weight of the wait time may be any value between 1 and 2, and the weight of the first write time may be any value between 1.5 and 2. In one implementation, when the master database uses a common disk, the weight of the first write elapsed time may be set to be greater than the wait elapsed time; when the host database uses an SSD (Solid State Drive) disk, the wait time may be weighted more than the first write time. This is because the writing speed of the SSD is much higher than that of the ordinary disk, i.e., it takes longer waiting time than writing time.
It should be noted that, when the database system is deployed on one database server, the scheduling server may be the same server as the database server, that is, the database server is both deployed with the database system and with the scheduling module that implements the function of the scheduling server.
202. And when the synchronization time consumption is longer than the first target time length, the scheduling server switches the current system synchronization mode to the target system synchronization mode.
In the embodiment of the present invention, the synchronization time is a delay time for the database system to process one database instruction. The first target duration may be a maximum value of the allowable delay. For example, the first target time period may be 1 second, 2 seconds, 3 seconds, 5 seconds, and the like. The target system synchronous mode may be an asynchronous mode.
In an optional implementation manner, the scheduling server may obtain a current system synchronization mode of the database system, and when the current system synchronization mode is not the target system synchronization mode, may send a mode switching instruction to the database system, and the database system switches the current system synchronization mode to the target system synchronization mode according to the mode switching instruction. When the current system synchronization mode is the target system synchronization mode, the scheduling server does not send the mode switching instruction, and the database system can keep the current system synchronization mode unchanged.
In an alternative implementation manner, the scheduling server may further switch the current stand-alone synchronization mode of the standby database according to the execution time of the standby database. For each standby database in at least one standby database, the scheduling server may obtain execution time consumption of the standby database, where the execution time consumption is used to indicate time consumption required by the standby database to complete a database instruction, and when the execution time consumption is longer than a second target time, the scheduling server may switch a current stand-alone synchronization mode of the standby database to a target stand-alone synchronization mode, where in the target stand-alone synchronization mode, the master database does not need to wait for a synchronization response of the standby database. When at least one standby database included in the database system is completely switched to the target single machine synchronization mode, the scheduling server can switch the current system synchronization mode of the database system to the target system synchronization mode. The target stand-alone synchronous mode may be a stand-alone asynchronous mode. And switching the current stand-alone synchronization mode of the standby database when the execution time consumption of the standby database is longer than the second target time length, so that the main database can be strongly synchronized with other standby databases with smaller execution time consumption. When all the standby databases are switched to the target single machine synchronization mode, the current system synchronization mode of the database system is switched to the target system synchronization mode, so that the problem of reduced processing performance of the database system caused by the problems of network delay or slow disk writing and the like can be solved.
In an alternative implementation manner, the step that the scheduling server takes time to execute to obtain the standby database may be: the scheduling server may obtain second stage time consumption required by each stage when the database completes the database instruction, and use a sum of the second stage time consumption as single execution time consumption for completing the database instruction of the database, and the scheduling server may use an average value of a plurality of single execution time consumption of the database as the execution time consumption. By taking the average value of the multiple single execution time consumptions as the execution time consumption, the problem that the standby database frequently switches a stand-alone synchronization mode due to high time consumption of certain execution of the standby database caused by network jitter can be avoided.
For example, the single execution time consumption of a certain backup database is T, the scheduling server stores the latest N single execution time consumptions of the backup database, where N is a positive integer. The execution time is T avg ＝(T 1 +T 2 +…+T N )/N。
In an optional implementation manner, the step of taking the sum of the second stage-specific time consumptions as the single execution time consumption of the standby database by the scheduling server, where the second stage-specific time consumptions are required by the stages when the standby database completes the database instruction, may be: the scheduling server may obtain communication time consumption and second write time consumption spent when the database completes a database instruction, determine weights of the communication time consumption and the second write time consumption respectively, and use the weights of the communication time consumption and the second write time consumption as single execution time consumption of the database. The weight of the communication time consumption and the second writing time consumption of writing into the disk is used as the single execution time consumption of the standby database, so that the time spent by the standby database for executing the database instruction once can be accurately reflected.
The weight of the communication elapsed time may be any value between 1 and 2, and the weight of the second write elapsed time may be any value between 1.5 and 2. When the disk of the standby database is a common disk, setting the weight of the second write-in consumed time to be greater than the weight of the communication consumed time; when the disk of the backup database is the SSD, the communication elapsed time weight may be set to be greater than the second write elapsed time weight.
For example, a communication between a backup database and a primary database takes T net And the second write time to disk is T disk The weight of communication time is W net The weight of the second write time is W disk . W is the disk of the backup database is SSD net >W disk Setting the weight of the communication time consumption as W net =2, the weight of the second write elapsed time is set to W disk =1.5, then a single execution of the standby database takes T =2 × T net *W net +T disk *W disk 。
203. And the database system performs data synchronization in the target system synchronization mode.
In the embodiment of the present invention, in the target system synchronization mode, when the primary database receives a database instruction, the execution result of the database instruction may be written into the disk of the primary database without requiring a synchronous response of any backup database to the database instruction.
For example, when the primary database receives an SQL statement for modifying data, the SQL statement is executed to obtain an execution result, and the SQL statement is written into the binlog log and sent to all the standby databases, but the execution result can be written into the disk of the primary database without waiting for the synchronous response of the standby databases, and then the execution success is fed back.
It should be noted that the scheduling server may continuously obtain the synchronization elapsed time, and when the synchronization elapsed time is less than the first target time length, step 204 and step 205 may be executed to switch the current system synchronization mode to the strong synchronization mode. Certainly, this is only a trigger condition for switching the current system synchronization mode to the strong synchronization mode, and the scheduling server may also switch the current system synchronization mode to the strong synchronization mode according to the resource occupation condition of the database server and the load state of the database system. When the database system meets the condition of the strong synchronization mode, the strong synchronization mode is used for data synchronization, so that the strong consistency of data in the main database and the at least one standby database is maintained.
204. When the synchronization time is less than the first target time, the scheduling server may switch the current system synchronization mode to a default system synchronization mode.
The default system synchronization mode may be a strong synchronization mode.
205. And the database system performs data synchronization in the default system synchronization mode.
In the default system synchronization mode, when the master database receives a database instruction, the master database executes the database instruction and sends the database instruction to all the standby databases, namely at least one standby database included in the database system, and when the master database receives a synchronization response of any one of the standby databases, the master database writes an execution result of the database instruction into a disk of the master database. By switching the current system synchronization mode of the database system to the strong synchronization mode when the synchronization time consumption is less than the first target time, the strong consistency of the data in the main database and each backup database can be ensured when the network condition is better.
It should be noted that, steps 202 to 205 are only processes in which the scheduling server according to the embodiment of the present invention exemplarily shows a process of switching a system synchronization mode of a database system from a strong synchronization mode to an asynchronous mode according to a synchronization time consumption, when the synchronization time consumption is greater than a first target time length, and after the switching, when the synchronization time consumption is less than the first target time length, switching the system synchronization mode of the database system from the asynchronous mode to the strong synchronization mode. The step numbers are for convenience of description only and do not indicate an actual execution order, and may be executed in the order of 201 to 205, or in the order of 201, 204, 205, 202, and 203, for example.
In the embodiment of the invention, the synchronization time consumption of the main database and at least one standby database in the database system is obtained, the synchronization time consumption is compared with the first target time length, and when the synchronization time consumption is longer than the first target time length, the current system synchronization mode is switched to the target system synchronization mode, so that the main database does not need to wait for the response of the standby database when receiving the database instruction, and the execution result is directly written into a disk, so that the main database can execute the database instruction faster, and the processing performance of the database system is improved.
Fig. 3 is a flow chart of another data synchronization method provided in accordance with an example embodiment, as shown in fig. 3. The method comprises the following steps:
301. and the scheduling server acquires the resource occupation information of the database server where the database system is located.
In the embodiment of the present invention, the scheduling server may obtain the resource occupation information of the database server in real time, or may obtain the resource occupation information of the database server periodically, for example, every 1 minute, 3 minutes, or 5 minutes. The database system may include a primary database and at least one secondary database, each database corresponding to a database server. The resource occupation information is occupation information of hardware resources in the database server, and the hardware resources may be CPU (Central Processing Unit) resources, memory resources, hard disk resources, and the like. The scheduling server can obtain the resource occupation information of each database server, and adjust the hardware resources allocated to each database according to the resource occupation information.
For example, database system A includes a master database A 1 And a database A 2 And database A 3 . Master database A 1 Deployed in database server D 1 Upper and standby database A 2 Deployed in database server D 2 Upper, standby database A 3 Deployed in database server D 3 The above.
In an alternative implementation manner, in order to save cost, one database server may include at least two database systems, and the scheduling server may obtain resource occupation information of each database system in the database server. The number of databases included in at least two database systems included in the database server may be the same or different, and this is not limited in the embodiment of the present invention.
302. And the scheduling server allocates available hardware resources to the database system according to the resource occupation information, wherein the available hardware resources are related to the load state of the database system.
In the embodiment of the present invention, an example in which one database server includes at least two database systems is described. The higher the load of the database system in the database server is, the more hardware resources required by the database system are, and the scheduling server can allocate larger available hardware resources to the database system with the high load. Of course, the total size of the available hardware resources allocated by the scheduling server to each database system cannot exceed the size of the hardware resources included in the database server.
In an alternative implementation manner, the size of the available hardware resource of the database system is dynamically variable, and the size of the available hardware resource is related to the resource occupation condition of the database server, the resource configuration file of the database system, and the load condition of the database.
Correspondingly, the step of allocating the available hardware resources to the database system by the scheduling server according to the resource occupation information may be: and acquiring a resource configuration file of each database system in the database server, wherein the resource configuration file is used for indicating the size of the hardware resource which can be used by each database system. When the hardware resources of the database server are not completely occupied, allocating available hardware resources for the database system according to the load state of each database system, wherein the size of the available hardware resources can exceed the size of the hardware resources indicated by the resource configuration file of the database system. When the hardware resources of the database server are completely used, the available hardware resources are allocated to the database system according to the resource configuration file. By dynamically adjusting the available hardware resources of each database system according to the load state of each database system, the waste of the hardware resources of the database server can be avoided, and the processing efficiency of the database system with higher load is improved.
For example, the scheduling server obtains a resource profile a of a database system a and a resource profile B of a database system B in the database server. Resource profile a indicates that database system a may use 50% of the hardware resources of the database server and resource profile B indicates that database system B may use 50% of the hardware resources. Database system a has a lower load and uses only 25% of the hardware resources of the database server, and database system B has a higher load and already uses 50% of the hardware resources of the database server. However, the database system B has a problem that the processing capability is reduced due to insufficient hardware resources, 20% of the hardware resources of the database server are also required, and the scheduling server indicates that 20% of the idle hardware resources of the database server are allocated to the database system B, so that the database system B can normally process database instructions. When the load of the database system a is increased, when the database system a uses 30% of the hardware resources of the database server, the hardware resources of the database server are completely used, and the available hardware resources of the database system a and the database system B are respectively set to 50% according to the resource configuration file. If the load of the database system A does not rise any more, the scheduling server can allocate the idle hardware resources of the database server to the database system B, and if the load of the database system A continuously rises until 50%, the database system A and the database system B use the allocated hardware resources to perform data processing.
In an optional implementation manner, each database system has a priority, the scheduling server can adjust the resource occupation amount of each database system according to the priority when the total resource occupation amount of the database server is high, and the higher the priority of the database system is, the more hardware resources are allocated.
In an alternative implementation manner, each database system has a priority, and the scheduling server may switch the system synchronization mode of at least one database system with a lower priority when the total resource occupation amount of the database server is higher.
Correspondingly, the step of allocating the available hardware resources to the database system by the scheduling server according to the resource occupation information may be: the scheduling server may determine a priority of each database system in the database server, and when it is determined that the total amount of resource occupation of the database server exceeds a target occupation threshold according to the resource occupation information, switch the system synchronization mode of at least one database system with a lower priority in the database server to the target system synchronization mode. And when the total resource occupation amount of the database server is determined to be lower than a preset occupation threshold value according to the resource occupation information, the database system of the target system synchronization mode is recovered to a strong synchronization mode. Wherein, the target occupancy threshold may be 85%, 90%, 95%, etc. The target system synchronous mode may be an asynchronous mode. The preset occupancy threshold may be 70%, 75%, 80%, etc. When the total resource occupation amount of the database server is large, the database system with the lower priority is switched to the asynchronous mode, so that part of hardware resources of the database server can be released, and the database system with the higher priority can use more hardware resources to process data.
In an optional implementation manner, the scheduling server may switch the system synchronization mode of the database system with the lowest priority to the target system synchronization mode when the total resource occupancy amount of the database server exceeds the target occupancy threshold, and if the total resource occupancy amount of the database server still exceeds the target occupancy threshold after switching, the scheduling server may switch the system synchronization mode of the database system with the lowest current priority to the target system synchronization mode until the total resource occupancy amount of the database server is less than the target occupancy threshold. And when the total resource occupation amount of the database server is determined to be lower than a preset occupation threshold value according to the resource occupation information, the database system in the target system synchronization mode is recovered to be in a strong synchronization mode.
For example, there are database system a, database system B, database system C, and database system D in the database server, where the priority ordering is database system a > database system B > database system C > database system D. And when the total amount of occupied hardware resources of the database server exceeds 95%, the scheduling server indicates that the system synchronous mode of the database system D is switched to the asynchronous mode so as to save the hardware resources. And when the total occupied hardware resource of the database server is lower than 80%, switching the system synchronization mode of the database system D into a strong synchronization mode.
In the embodiment of the invention, the resource occupation information of the database server is obtained, and the available hardware resources are distributed to the database system according to the resource occupation information, so that the hardware resources of the database server can be utilized to the maximum extent, the hardware resource waste of the database server is avoided, and the processing efficiency of the database system with higher load is improved.
Fig. 4 is a flow chart of another data synchronization method provided in accordance with an example embodiment, as shown in fig. 4. The method comprises the following steps:
401. the scheduling server acquires the running state information of a main database and at least one standby database in the database system, wherein the running state information is used for indicating whether the main database and the at least one standby database run normally or not.
In the embodiment of the present invention, the running state information may include an input/output thread state, a memory state, a network state, heartbeat information, and the like of the database. The scheduling server may obtain the operation state information in real time, or may obtain the operation state information periodically.
In an alternative implementation manner, when the scheduling server determines that the master database is abnormal according to the operation state information, the scheduling server may send a connection request to the master database to establish a connection with the master database, where the connection request is used to detect whether the master database is normal. When the scheduling server judges that the main database is abnormal based on the connection request, determining that the main database is abnormal; and when the scheduling server judges that the master database is normal based on the connection request, determining that the running state information is wrong. Whether the master database is normal or not is judged secondarily through the scheduling server, the problem of mistaken switching of the master database caused by running state information errors due to agent module abnormity or network fluctuation can be avoided, and the stability of the system is improved.
402. When the master database is abnormally operated, the scheduling server determines the fault reason causing the abnormal operation of the master database.
The scheduling server can determine the fault reasons of abnormal operation of the main database according to the operation state information, such as thread queue congestion caused by overlarge load, slow disk writing, abnormal CPU occupation, memory damage and the like.
403. And the scheduling server switches the main database into the standby database according to the fault reason and switches the standby database in at least one standby database into a new main database.
The scheduling server can directly switch the main database and the standby database when determining that the main database is abnormally operated, and can also switch the main database and the standby database according to the type of the fault reason. This process may be referred to as a disaster recovery handover.
In an optional implementation manner, the step of switching the primary database to the standby database by the scheduling server according to the failure reason, and switching the standby database in the at least one standby database to the new primary database may be: and the scheduling server determines a fault type corresponding to the fault reason, and when the fault type is not the target fault type, the main database is switched to the standby database, and the standby database in at least one standby database is switched to a new main database. The target fault type is a fault type which can be solved after a standby database in at least one standby database is switched to a new main database. And when the fault type is the target fault type, starting a switchover-free mechanism, wherein the switchover-free mechanism is used for keeping the main database unchanged when a fault error reason of the target fault type is detected in the target switchover-free time. Whether the main database and the standby database are switched or not is determined according to the fault type corresponding to the fault reason, switching can be performed when the problem can be solved by switching the databases, and the problem that the databases cannot be normally provided due to frequent switching of the databases caused by the problems of overlarge access amount and the like is avoided.
The target fault type can be instantaneous access quantity too large, load too high, network congestion, hardware damage and the like. The length of the target cut-free time may be dynamically modified according to actual requirements, such as 1 minute, 10 minutes, 30 minutes, and the like, which is not specifically limited in the embodiment of the present invention.
In an alternative implementation manner, the scheduling server may trigger an alarm message after each disaster recovery handover, and use the alarm message to indicate the reason for currently causing the handover between the primary database and the backup database. Through the alarm information, the staff can repair and adjust the database system based on the alarm information, and the performance of the database system is improved.
In an optional implementation manner, when a hardware fault occurs in a database server where the database system is located, the scheduling server may cancel the cut-free mechanism after the hardware fault is repaired, and delete the database related to the cut-free mechanism from the cut-free configuration file. By canceling the switchover-free mechanism after the hardware fault is repaired, the situation that disaster recovery switchover cannot be carried out when other databases are in fault can be avoided.
In the embodiment of the invention, by acquiring the running state of the database in the database system, disaster recovery switching can be performed when the database fails so as to ensure that the database system can normally provide the server, and meanwhile, a switching-free rule is set according to the failure type of the failure reason, so that the problem that the database system cannot provide service due to frequent switching can be solved.
Fig. 5 is a block diagram of a data synchronization apparatus provided according to an embodiment of the present invention. The apparatus is used for executing the steps of the data synchronization method, and referring to fig. 5, the apparatus includes: a first obtaining module 501, a first switching module 502 and a writing module 503.
A first obtaining module 501, configured to obtain synchronization time consumption of a main database and at least one standby database in a database system, where the synchronization time consumption is used to indicate time consumption required by each stage when data synchronization is performed between the main database and any one of the standby databases;
a first switching module 502, configured to switch a current system synchronization mode to a target system synchronization mode when synchronization time consumption is greater than a first target time length;
the writing module 503 is configured to, in the target system synchronization mode, write an execution result of the database instruction into a disk of the primary database without requiring a synchronous response of any backup database to the database instruction when the primary database receives the database instruction.
In an optional implementation manner, the first obtaining module 501 is further configured to obtain, in the database system, first stage consumed time required by each stage when data synchronization is performed between the main database and one standby database, and use a sum of the first stage consumed time as single synchronization consumed time of the database system; and taking the average value of the plurality of single synchronization time consumptions of the database system as the synchronization time consumption.
In another optional implementation manner, the first obtaining module 501 is further configured to obtain wait time consumed for the master database to execute the database instruction and first write time consumed for the master database to wait for a backup database to return a synchronization response, where the first write time consumed for the master database to write the execution result of the database instruction into the disk of the master database; respectively determining weights of waiting time consumption and first writing time consumption; and taking the weight sum of the waiting time and the first writing time as the single synchronization time of the database system.
In another optional implementation manner, the apparatus further includes:
the second acquisition module is used for acquiring the execution time consumption of the standby database for each standby database in the at least one standby database, and the execution time consumption is used for expressing the time consumption required by the standby database for completing the database instruction;
the second switching module is used for switching the current stand-alone synchronization mode of the standby database to a target stand-alone synchronization mode when the execution time consumption is longer than the second target time length; in the target stand-alone synchronization mode, the master database does not need to wait for the synchronization response of the standby database.
In another optional implementation manner, the second obtaining module is further configured to obtain second stage consumed time required by each stage when the database completes the database instruction, and use a sum of the second stage consumed time as single execution consumed time for completing the database instruction of the database; and taking the average value of the plurality of single execution time consumptions of the database as the execution time consumption.
In another optional implementation manner, the second obtaining module is further configured to obtain communication consumed time spent by the standby database when the database completes the database instruction and second writing consumed time, where the communication consumed time is consumed time required for the standby database to communicate with the main database, and the second writing consumed time is consumed time required for the standby database to write the execution result of the database instruction into a disk of the standby database; respectively determining the weight of the communication time consumption and the weight of the second writing time consumption; and weighting the communication time consumption and the second writing time consumption to obtain the single execution time consumption of the database.
In another optional implementation manner, the apparatus further includes:
and the third switching module is used for switching the current system synchronization mode to the target system synchronization mode when all at least one standby database in the database system is switched to the target single-machine synchronization mode.
In another optional implementation manner, the apparatus further includes:
the fourth switching module is used for switching the current system synchronization mode to the default system synchronization mode when the synchronization time consumption is less than the first target time length;
in a default system synchronization mode, when a main database receives a database instruction, the main database executes the database instruction, the database instruction is sent to at least one standby database, and when the main database receives a synchronization response of any standby database, an execution result of the database instruction is written into a disk of the main database.
In another optional implementation manner, the apparatus further includes:
the third acquisition module is used for acquiring resource occupation information of a database server where the database system is located, and the database server comprises at least two database systems;
and the allocation module is used for allocating available hardware resources to the database system according to the resource occupation information, wherein the size of the available hardware resources is related to the load state of the database system.
In another optional implementation manner, the allocation module is further configured to obtain a resource configuration file of each database system in the database server, where the resource configuration file is used to indicate a size of a hardware resource that can be used by each database system; when the hardware resources of the database server are not completely occupied, distributing available hardware resources for the database system according to the load state of each database system, wherein the size of the available hardware resources can exceed the size of the hardware resources indicated by the resource configuration file of the database system; and when the hardware resources of the database server are completely used, distributing the available hardware resources for the database system according to the resource configuration file.
In another optional implementation manner, the allocation module is further configured to determine a priority of each database system in the database server; and when the total resource occupation amount of the database server is determined to exceed the target occupation threshold value according to the resource occupation information, switching the system synchronization mode of at least one database system with low priority in the database server to the target system synchronization mode.
In another optional implementation manner, the apparatus further includes:
the fourth acquisition module is used for acquiring running state information of a main database and at least one standby database in the database system, and the running state information is used for indicating whether the main database and the at least one standby database run normally or not;
the determining module is used for determining the fault reason causing the abnormal operation of the main database when the main database operates abnormally;
and the fifth switching module is used for switching the main database into the standby database according to the fault reason and switching the standby database in at least one standby database into a new main database.
In another optional implementation manner, the fifth switching module is further configured to determine a fault type corresponding to a fault cause; when the fault type is not the target fault type, switching the main database into a standby database, switching a standby database in at least one standby database into a new main database, wherein the target fault type is a fault type which can be solved after switching the standby database in at least one standby database into the new main database; and when the fault type is the target fault type, starting a switchover-free mechanism, wherein the switchover-free mechanism is used for indicating that the fault error reason of the target fault type is detected in the target switchover-free time, and keeping the master database unchanged.
In the embodiment of the invention, the synchronization time consumption of the main database and at least one standby database in the database system is obtained, the synchronization time consumption is compared with the first target time length, and when the synchronization time consumption is longer than the first target time length, the current system synchronization mode is switched to the target system synchronization mode, so that the main database does not need to wait for the response of the standby database when receiving the database instruction, and the execution result is directly written into a disk, so that the main database can execute the database instruction faster, and the processing performance of the database system is improved.
It should be noted that: in the above embodiment, when the device runs an application program, only the division of the functional modules is described as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Fig. 6 is a schematic structural diagram of a server according to an embodiment of the present invention, where the server 600 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 601 and one or more memories 602, where at least one instruction is stored in the memory 602, and the at least one instruction is loaded and executed by the processor 601 to implement the methods provided by the foregoing method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input/output, and the server may also include other components for implementing the functions of the device, which are not described herein again.
The embodiment of the invention also provides a storage medium, the storage medium is applied to the server, and the storage medium stores program codes which are used for executing the data synchronization method in the embodiment of the invention.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a storage medium, where the storage medium may be a read-only memory, a magnetic disk or an optical disk.
The present invention is not limited to the above embodiments, and any modifications, equivalent substitutions, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (15)

1. A method for synchronizing data, the method comprising:
acquiring first stage time consumption required by each stage when data synchronization is performed between a main database and a standby database in a database system, and taking the sum of the first stage time consumption as single synchronization time consumption of the database system; the master database is used for receiving the synchronous response of the standby databases, and the master database is used for writing the execution result of the database instruction into the disk of the master database;
taking an average value of a plurality of single synchronization time consumptions of the database system as synchronization time consumptions of the main database and at least one standby database, wherein the synchronization time consumptions are used for representing time consumptions required by all stages when data synchronization is carried out between the main database and any standby database;
when the synchronization time consumption is longer than the first target time length, switching the current system synchronization mode to a target system synchronization mode;
and in the target system synchronization mode, when the main database receives the database command, writing an execution result of the database command into a disk of the main database on the premise of not requiring synchronous response of any standby database to the database command.
2. The method according to claim 1, wherein the obtaining a first stage time consumption required by each stage when data synchronization is performed between a primary database and a standby database, and taking a sum of the first stage time consumption as a single synchronization time consumption of the database system comprises:
acquiring waiting time consumed by the main database for executing the database instruction and first writing time consumed by the main database for waiting for a standby database to return the synchronous response, wherein the first writing time consumed by the main database for writing the execution result of the database instruction into a disk of the main database;
respectively determining the weights of the waiting consumed time and the first writing consumed time;
and taking the weight sum of the waiting time consumption and the first writing time consumption as the single synchronization time consumption of the database system.
3. The method of claim 1, wherein after it takes time to synchronize the primary database and the at least one backup database in the database system, the method further comprises:
for each standby database in the at least one standby database, acquiring execution time consumption of the standby database, wherein the execution time consumption is used for representing time consumption required by the standby database for completing the database instruction;
when the execution time consumption is longer than a second target time, switching the current stand-alone synchronization mode of the standby database to a target stand-alone synchronization mode;
in the target stand-alone synchronization mode, the primary database does not need to wait for the synchronization response of the standby database.
4. The method of claim 3, wherein the retrieving the backup database is time consuming, comprising:
acquiring second stage consumed time required by each stage when the database instruction is completed by the standby database, and taking the sum of the second stage consumed time as single execution consumed time of the database instruction completion by the standby database;
and taking the average value of the plurality of single execution time consumptions of the database as the execution time consumption.
5. The method according to claim 4, wherein the obtaining of the second stage time consumption required by the database to complete each stage of the database instruction, and taking the sum of the second stage time consumption as the single execution time consumption of the database to complete the database instruction comprises:
acquiring communication time consumption and second writing time consumption which are consumed when the standby database completes the database instruction, wherein the communication time consumption is time consumption required by communication between the standby database and the main database, and the second writing time consumption is time consumption required by the standby database to write an execution result of the database instruction into a disk of the standby database;
respectively determining the weight of the communication elapsed time and the weight of the second writing elapsed time;
and taking the weight sum of the communication elapsed time and the second writing elapsed time as the single execution elapsed time of the database.
6. The method according to claim 3, wherein after switching the current standalone synchronization mode of the standby database to the target standalone synchronization mode, the method further comprises:
and when all the at least one standby database in the database system is switched to the target single machine synchronization mode, switching the current system synchronization mode to the target system synchronization mode.
7. The method of claim 1, wherein after the synchronization of the primary database and the at least one standby database in the database system is time-consuming, the method further comprises:
when the synchronization time consumption is less than the first target time length, switching the current system synchronization mode to a default system synchronization mode;
in the default system synchronization mode, when the main database receives a database instruction, the main database executes the database instruction, the database instruction is sent to the at least one standby database, and when the main database receives a synchronization response of any standby database, an execution result of the database instruction is written into a disk of the main database.
8. The method of claim 1, wherein before it is time consuming to synchronize the primary database and the at least one standby database in the database system, the method further comprises:
acquiring resource occupation information of a database server in which the database system is located, wherein the database server comprises at least two database systems;
and distributing available hardware resources for the database system according to the resource occupation information, wherein the size of the available hardware resources is related to the load state of the database system.
9. The method of claim 8, wherein said allocating available hardware resources to said database system according to said resource occupancy information comprises:
acquiring a resource configuration file of each database system in the database server, wherein the resource configuration file is used for indicating the size of hardware resources which can be used by each database system;
when the hardware resources of the database server are not completely occupied, allocating available hardware resources to the database systems according to the load state of each database system, wherein the size of the available hardware resources can exceed the size of the hardware resources indicated by the resource configuration files of the database systems;
and when the hardware resources of the database server are completely used, distributing the available hardware resources for the database system according to the resource configuration file.
10. The method of claim 8, wherein said allocating available hardware resources to said database system according to said resource occupancy information comprises:
determining a priority of each database system in the database server;
and when the total resource occupation amount of the database server is determined to exceed a target occupation threshold value according to the resource occupation information, switching the system synchronization mode of at least one low-priority database system in the database server to the target system synchronization mode.
11. The method of claim 1, wherein after the synchronization of the primary database and the at least one standby database in the database system is time-consuming, the method further comprises:
acquiring running state information of the main database and the at least one standby database in the database system, wherein the running state information is used for indicating whether the main database and the at least one standby database run normally or not;
when the master database is abnormally operated, determining a fault reason causing the abnormal operation of the master database;
and switching the main database into a standby database according to the fault reason, and switching the standby database in the at least one standby database into a new main database.
12. The method of claim 11, wherein switching the primary database to a standby database and switching a standby database of the at least one standby database to a new primary database according to the failure cause comprises:
determining a fault type corresponding to the fault reason;
when the fault type is not a target fault type, switching the main database to a standby database, and switching a standby database in the at least one standby database to a new main database, wherein the target fault type is a fault type which can be solved after the standby database in the at least one standby database is switched to the new main database;
when the fault type is the target fault type, starting a switchover-free mechanism, wherein the switchover-free mechanism is used for indicating that the main database is kept unchanged when a fault error reason of the target fault type is detected in a target switchover-free time.
13. A data synchronization apparatus, the apparatus comprising:
the first acquisition module is used for acquiring first stage consumed time required by each stage when data synchronization is carried out between a main database and a standby database in a database system, and taking the sum of the first stage consumed time as single synchronization consumed time of the database system; the master database is used for receiving the synchronous response of the standby databases, and the master database is used for writing the execution result of the database instruction into the disk of the master database;
taking an average value of a plurality of single synchronization time consumptions of the database system as synchronization time consumptions of the main database and at least one standby database, wherein the synchronization time consumptions are used for representing time consumptions required by all stages when data synchronization is carried out between the main database and any standby database;
the first switching module is used for switching the current system synchronization mode to a target system synchronization mode when the synchronization time consumption is longer than a first target time length;
and the writing module is used for writing the execution result of the database instruction into a disk of the main database under the premise of not requiring synchronous response of any backup database to the database instruction when the main database receives the database instruction in the target system synchronous mode.
14. A server, characterized in that the server comprises a processor and a memory for storing program code, which is loaded by the processor and which performs the data synchronization method of any of claims 1 to 12.
15. A storage medium for storing a program code for executing the data synchronization method of any one of claims 1 to 12.
CN201910662737.2A 2019-07-22 2019-07-22 Data synchronization method, device, server and storage medium Active CN110377664B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910662737.2A CN110377664B (en) 2019-07-22 2019-07-22 Data synchronization method, device, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910662737.2A CN110377664B (en) 2019-07-22 2019-07-22 Data synchronization method, device, server and storage medium

Publications (2)

Publication Number Publication Date
CN110377664A CN110377664A (en) 2019-10-25
CN110377664B true CN110377664B (en) 2022-10-21

Family

ID=68254859

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910662737.2A Active CN110377664B (en) 2019-07-22 2019-07-22 Data synchronization method, device, server and storage medium

Country Status (1)

Country Link
CN (1) CN110377664B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111309742A (en) * 2020-01-20 2020-06-19 深圳壹账通智能科技有限公司 Data synchronization method and device, computer equipment and storage medium
CN114328034B (en) * 2022-03-15 2022-06-03 苏州浪潮智能科技有限公司 Control method and device of storage equipment, electronic equipment and storage medium
CN115544147B (en) * 2022-10-18 2023-12-08 山东水发紫光大数据有限责任公司 Machine room data reading method, device, equipment and storage medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102955851B (en) * 2012-10-30 2016-05-11 北京奇虎科技有限公司 A kind of switching method and apparatus of database
US9792346B2 (en) * 2014-06-26 2017-10-17 Sybase, Inc. Automatic mode switching in a synchronous replication environment
CN105159795A (en) * 2015-08-21 2015-12-16 小米科技有限责任公司 Data synchronization method, apparatus and system
CN108897658B (en) * 2018-05-31 2023-03-21 康键信息技术（深圳）有限公司 Method and device for monitoring master database, computer equipment and storage medium

Also Published As

Publication number Publication date
CN110377664A (en) 2019-10-25

Similar Documents

Publication Publication Date Title
US11360854B2 (en) Storage cluster configuration change method, storage cluster, and computer system
CN108664496B (en) Data migration method and device
US11397648B2 (en) Virtual machine recovery method and virtual machine management device
CN110427284B (en) Data processing method, distributed system, computer system, and medium
CN110377664B (en) Data synchronization method, device, server and storage medium
US20140304306A1 (en) Database Management System With Database Hibernation and Bursting
CN105069152B (en) data processing method and device
CN111049928B (en) Data synchronization method, system, electronic device and computer readable storage medium
US10445295B1 (en) Task-based framework for synchronization of event handling between nodes in an active/active data storage system
CN112256433B (en) Partition migration method and device based on Kafka cluster
CN111209110A (en) Task scheduling management method, system and storage medium for realizing load balance
CN114064414A (en) High-availability cluster state monitoring method and system
CN114253743A (en) Message synchronization method, device, node and readable storage medium
US11128708B2 (en) Managing remote replication in storage systems
US10049021B2 (en) Redundant system and redundancy method
CN115292408A (en) Master-slave synchronization method, device, equipment and medium for MySQL database
CN111538585A (en) Js-based server process scheduling method, system and device
CN111240901A (en) Node dynamic expansion system, method and equipment of distributed block storage system
CN107943615B (en) Data processing method and system based on distributed cluster
CN112579550B (en) Metadata information synchronization method and system of distributed file system
CN114281508A (en) Data batch-flow fusion offline calculation method
CN112631994A (en) Data migration method and system
CN106776153B (en) Job control method and server
CN115562805A (en) Resource migration method and device and electronic equipment
CN115438021A (en) Resource allocation method and device for database server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230920

Address after: 518057 Tencent Building, No. 1 High-tech Zone, Nanshan District, Shenzhen City, Guangdong Province, 35 floors

Patentee after: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd.

Patentee after: TENCENT CLOUD COMPUTING (BEIJING) Co.,Ltd.

Address before: 518057 Tencent Building, No. 1 High-tech Zone, Nanshan District, Shenzhen City, Guangdong Province, 35 floors

Patentee before: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd.