CN117112695A - Database synchronization method and device, electronic equipment and storage medium - Google Patents
Database synchronization method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117112695A CN117112695A CN202311094664.4A CN202311094664A CN117112695A CN 117112695 A CN117112695 A CN 117112695A CN 202311094664 A CN202311094664 A CN 202311094664A CN 117112695 A CN117112695 A CN 117112695A
- Authority
- CN
- China
- Prior art keywords
- database
- synchronization
- log
- standby
- response time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 230000004044 response Effects 0.000 claims abstract description 91
- 230000001360 synchronised effect Effects 0.000 claims description 35
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 abstract description 26
- 230000001680 brushing effect Effects 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011010 flushing procedure Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a database synchronization method, a database synchronization device, electronic equipment and a storage medium. The method specifically comprises the following steps: acquiring an archive log of a target transaction sent by a database host; wherein the archive log consists of at least one event; determining a standby machine synchronization mode according to the number of events in the archive log; and synchronizing the data of the database host to the database standby according to the standby synchronization mode, and feeding back a synchronization response to the database host. According to the technical scheme of the embodiment of the application, the synchronization mode is determined according to the number of the events in the archive log so as to synchronize the data in the database host to the database standby, so that the performance jitter caused by frequent disk brushing operation and I/O operation can be avoided while the processing performance stability of the database is ensured, and the synchronization efficiency between the database hosts and the database standby is improved.
Description
Technical Field
The present application relates to the field of database technologies, and in particular, to a database synchronization method, a device, an electronic device, and a storage medium.
Background
The data is the core of the database, so that the data loss is prevented, and the data security is ensured on the premise that the database is high in availability. Adding multiple hosts (multiple copies) is the basic way the industry achieves high availability of databases. Both the primary centralized database and the distributed database typically employ archive logs back for synchronization of data, and the host synchronizes waiting for the backup to receive a back-off response to ensure strong consistency of the primary and backup data.
Currently, in the synchronous operation of the main machine and the standby machine of various databases, when the main machine executes a transaction with a write statement, the change of the related write operation is recorded in an archive log. When the transaction begins to commit or completes commit, the archive log information increment is sent to the standby machine, and the standby machine response is synchronously waited. The execution result of the transaction is not returned to the client until the standby machine responds or a waiting timeout is reached. However, this approach may result in reduced synchronization efficiency between the primary and secondary machines, and timeout waiting may also affect the processing capacity of the database.
Disclosure of Invention
The application provides a database synchronization method, a database synchronization device, electronic equipment and a storage medium, which are used for improving the synchronization efficiency between a database host and a database standby machine while guaranteeing the overall performance of a database.
According to a first aspect of the present application, there is provided a database synchronization method applied to a database backup machine, the method comprising:
acquiring an archive log of a target transaction sent by a database host; wherein the archive log consists of at least one event;
determining a standby machine synchronization mode according to the number of events in the archive log;
and synchronizing the data of the database host to the database standby according to the standby synchronization mode, and feeding back a synchronization response to the database host.
According to a second aspect of the present application, there is provided a database synchronization method applied to a database host computer, the method comprising:
acquiring the log quantity of the archive log corresponding to the target transaction;
determining theoretical response time length of the target transaction according to the log quantity, the number of the target transaction and preset influence parameters, and sending the archive log to a database backup machine;
and determining whether the database standby machine generates transaction synchronization timeout according to the actual response time length and the theoretical response time length of the feedback synchronization response provided by the database standby machine so as to carry out timeout alarm.
According to a third aspect of the present application, there is provided a database synchronization apparatus for use in a database backup machine, the apparatus comprising:
the archive log acquisition module is used for acquiring archive logs of target transactions sent by the database host; wherein the archive log consists of at least one event;
the synchronous mode determining module is used for determining a standby synchronous mode according to the number of events in the archive log;
and the standby machine data synchronization module is used for synchronizing the data of the database host machine to the database standby machine according to the standby machine synchronization mode and feeding back a synchronization response to the database host machine.
According to a fourth aspect of the present application, there is provided a database synchronization apparatus for use with a database host computer, the apparatus comprising:
the log quantity acquisition module is used for acquiring the log quantity of the archive log corresponding to the target transaction;
the response time length determining module is used for determining the theoretical response time length of the target transaction according to the log quantity, the number of the target transaction and the preset influence parameter, and sending the archive log to a database backup machine;
and the synchronous timeout judging module is used for determining whether the database standby machine generates the transaction synchronous timeout or not according to the actual response time length and the theoretical response time length of the feedback synchronous response provided by the database standby machine so as to carry out timeout alarm.
According to a fifth aspect of the present application, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the database synchronization method according to the embodiment of the first aspect of the present application; and/or executing the database synchronization method according to the embodiment of the second aspect of the present application.
According to a sixth aspect of the present application, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a database synchronization method according to an embodiment of the first aspect of the present application; and/or implementing the database synchronization method according to the embodiment of the second aspect of the present application.
According to the technical scheme of the embodiment of the application, the synchronization mode is determined according to the number of the events in the archive log so as to synchronize the data in the database host to the database standby, so that the performance jitter caused by frequent disk brushing operation and I/O operation can be avoided while the processing performance stability of the database is ensured, and the synchronization efficiency between the database hosts and the database standby is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a database synchronization method according to a first embodiment of the present application;
FIG. 2 is a flowchart of a database synchronization method according to a second embodiment of the present application;
fig. 3A is a schematic flow chart of a standby policy brush disc synchronization according to a third embodiment of the present application;
fig. 3B is a schematic flow chart of a method for checking timeout of standby response of a host according to the third embodiment of the present application;
fig. 4 is a schematic structural diagram of a database synchronization device according to a fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of a database synchronization device according to a fifth embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device implementing a database synchronization method according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a database synchronization method according to an embodiment of the present application, where the method is applicable to a case where a host and a standby of a database perform data synchronization, and the method is applied to a side of the database standby, and the method may be performed by a database synchronization device, where the database synchronization device may be implemented in a form of hardware and/or software, and the database synchronization device may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, acquiring an archiving log of a target transaction sent by a database host; wherein the archive log is comprised of at least one event.
The database may be any type of database, including, but not limited to, a centralized database and a distributed database, for example; correspondingly, the database host can be a master device of any database, and the database standby can be a redundant device of the database host, so that a plurality of database standby can be set for synchronizing the data in the database host in order to ensure the safety of the data. Of course, the embodiments of the present application are described only in terms of interactions between one database host and one database standby, and are not to be construed as limiting the number of database standby. The target transaction may be a collection of data that requires synchronization between the primary and secondary machines. An archive log may be information used to record the archive of transactions in a database host. The event is a record of operations such as query, call and the like performed on data in the database in the running process of the database host, and a plurality of events can be recorded in the archive log. It should be noted that, after updating the archive logs, the database host will generally directly push to each database backup machine, so that the database backup machine performs data synchronization according to the archive logs.
S120, determining a standby machine synchronization mode according to the number of events in the archive log.
The number of events may be the number of events in the archive log, and it may be understood that in the prior art, an incremental transmission mode is adopted for updating the archive log and synchronizing data, that is, after a new archive log is generated, the database host sends a corresponding incremental log to the database standby, and the database standby synchronously backs up the data according to the content of the continuously sent incremental log. But doing so tends to result in frequent disk brushing operations that start the standby, thereby wasting database resources. The embodiment of the application takes the number of the events as the basis for synchronizing the database backup machine. The database backup machine can synchronize after receiving all archive logs of a complete transaction and reply to the database host machine. Because a plurality of events may exist in one transaction, different synchronization modes are distinguished according to the number of the events, and the resource consumption of the database standby machine can be reduced while the synchronization quality and the efficiency can be guaranteed to the greatest extent. Alternatively, if the number of events in a single transaction is small, each transaction may be processed directly; if the number of events in a single transaction is large, then these event data packets may be processed.
In an optional embodiment, the determining the standby synchronization mode according to the number of events in the archive log may include: if the number of the events is larger than a preset number threshold, grouping the events according to the number threshold to obtain at least one event group to be synchronized; and respectively taking each event group to be synchronized as granularity for synchronization.
The number threshold may be a processing limit number set corresponding to the number of events, that is, the highest number of the database backup machine processing the events in the archive log at one time. Of course, the number threshold may be set by a person skilled in the relevant art based on a large number of experiments or manual experience, which is not limited by the embodiment of the present application. If the number of events in a transaction exceeds the number threshold, all events in the archive log may be grouped by the number threshold to obtain a set of events to be synchronized. Each event group to be synchronized is treated as a single granularity for subsequent synchronization processing.
For example, assuming that the set number threshold is N, if the number of events in the archive log exceeds N, then every N events are combined into a set of events to be synchronized as a granularity for synchronization. For example, if the threshold number is 10 and the total number of events in the archive log is 50, then it is possible to divide into 5 different event groups to be synchronized. It can be understood that the events in the archive log are divided according to the number, so that the defect that the processing performance of the database is affected by frequent disk brushing operation of the database backup machine for synchronizing data can be effectively avoided, the efficiency of database synchronization is improved, and the performance of processing data of the database is improved.
In another alternative embodiment, the determining the standby synchronization mode according to the number of events in the archive log may include: and if the number of the events is not greater than the number threshold, using the target transaction as a granularity for synchronization.
It will be appreciated that if the number of events in the archive log is less than the set number threshold, the database backup is able to handle the data synchronization corresponding to those events with better performance. Therefore, the whole target transaction with the event quantity less than the threshold value can be directly treated as one granularity, and the target transaction is directly synchronized subsequently.
S130, synchronizing the data of the database host to the database standby according to the standby synchronization mode, and feeding back a synchronization response to the database host.
According to the synchronization method determined by the foregoing embodiments, each data to be synchronized by the database host is flushed (write cache operation) and synced (operation of flushing the cache and writing to the physical disk) in the database standby. And then feeding back a message of the target transaction synchronization completion to the database host, namely replying a synchronization response.
According to the technical scheme of the embodiment of the application, the synchronization mode is determined according to the number of the events in the archive log so as to synchronize the data in the database host to the database standby, so that the performance jitter caused by frequent disk brushing operation and I/O operation can be avoided while the processing performance stability of the database is ensured, and the synchronization efficiency between the database hosts and the database standby is improved.
Example two
Fig. 2 is a flowchart of a database synchronization method provided in a second embodiment of the present application, where the present embodiment is applicable to a case where a host and a standby of a database perform data synchronization, the method is applied to a side of the host of the database, the method may be performed by a database synchronization device, the database synchronization device may be implemented in a form of hardware and/or software, and the database synchronization device may be configured in an electronic device. As shown in fig. 2, the method includes:
s210, acquiring the log quantity of the archive log corresponding to the target transaction.
Wherein the target transaction may be a data set that requires synchronization between the primary and secondary machines. An archive log may be information used to record the archive of transactions in a database host. The log amount may be the amount of data that archives the recorded information of various transactions or events in the log. Since the archive log is generated directly in the database host, it can be directly invoked by the database host to obtain the log amount of the archive log. It should be noted that, in actual situations, the log amounts of the archive logs synchronized by each transaction between the primary and the secondary computers are different in size, so that the synchronization policy of the secondary computers can be correspondingly adjusted for different secondary computers.
S220, determining theoretical response time length of the target transaction according to the log quantity, the number of the target transaction and preset influence parameters, and sending the archive log to a database backup machine.
The number of target transactions may be the number of transactions that need to be data synchronized, and the influencing parameter may be an auxiliary parameter for calculating the theoretical response time length. In an alternative embodiment, the preset influence parameter may include: scaling factor, adjustment amount, and timeout threshold lower limit. The proportionality coefficient can be the embodiment of the processing capacity of the database and is used for amplifying or reducing the parameters of the processing time length corresponding to the log quantity according to the proportionality of the processing capacity of the database; the adjustment quantity is the adaptively-supplemented time length after the proportion adjustment; the lower limit of the timeout threshold is the limit of the timeout of the standby machine acceptable by the database host, and if the timeout exceeds the limit, the standby machine is not responded, and the phenomenon of timeout of the standby machine in the synchronization process is indicated. Of course, the scaling factor, the adjustment amount, and the lower timeout threshold limit may be determined by a skilled person through a plurality of experiments or actual conditions, which are not limited by the embodiment of the present application.
And calculating the log quantity and the number of the target transactions through preset influence parameters to obtain the theoretical response time length of the target transactions. It is understood that the theoretical response time may be the longest time that the database backup machine theoretically feeds back a response to the database host machine after synchronization. It should be noted that, the target transactions (including the amount of events in the archive log) needed to be backed up between different database backup machines are different, so that the target transactions can be calculated for the different database backup machines according to the different log amounts corresponding to the different database backup machines, so as to obtain the theoretical response time lengths corresponding to the respective backup machines.
Further, the determining the theoretical response time of the target transaction according to the log amount, the number of the target transactions and the preset influence parameter may include: determining and calculating response time according to the log quantity, the proportionality coefficient and the adjustment quantity; and taking the larger one of the calculated response time length and the overtime threshold lower limit as the theoretical response time length.
Wherein, for calculation of theoretical response time length, analysis is needed for different situations. For an archive log of a single transaction, directly taking the sum of the product of the log quantity of the archive log and the proportional coefficient and the adjustment quantity as the calculation response time; and comparing the calculated response time with the lower limit of the timeout threshold, wherein the larger party can be used as the theoretical response time.
It can be understood that, according to different situations of log amounts of different database backup machines needing to perform synchronous processing, the calculation response time is calculated based on the log amounts, but the calculation response time may be shorter, if the database host machine continuously waits for a shorter time and does not receive the feedback response of the database backup machine, the backup machine is judged to be overtime, and erroneous judgment is easily caused to influence the synchronous efficiency. Setting a timeout threshold lower limit, and selecting the calculated response time length as the theoretical response time length if the calculated response time length is larger than the timeout threshold lower limit; similarly, if the calculated response time is less than or equal to the timeout threshold lower limit, the timeout threshold lower limit is selected as the theoretical response time.
In addition, for the archive logs of multiple transactions, the database host needs to start a group commit mode, and in each synchronization process, an archive log corresponding to a group of transactions can be generated, and then all log amounts in the group of transactions need to be overlapped to obtain a whole group of log amounts. Then taking the sum of the product of the whole group of log quantity and the proportional coefficient and the adjustment quantity as the calculated response time length; similarly, the calculated response time length is compared with the lower limit of the overtime threshold value, and the larger party can be used as the theoretical response time length. If the calculated response time length is larger than the lower limit of the timeout threshold, selecting the calculated response time length as the theoretical response time length; similarly, if the calculated response time is less than or equal to the timeout threshold lower limit, the timeout threshold lower limit is selected as the theoretical response time.
S230, determining whether the database standby machine generates the transaction synchronization timeout according to the actual response time length and the theoretical response time length of the feedback synchronization response provided by the database standby machine so as to carry out timeout alarm.
The actual response time is the actual time used by the database standby machine for synchronization according to the archive log of the target transaction, when the actual response time exceeds the theoretical response time, the database standby machine can be determined to have overtime phenomenon in the synchronization process and carry out overtime alarm, and of course, any alarm mode in the related technology can be adopted, such as acousto-optic alarm and the like, and the embodiment of the application is not limited to the above.
According to the technical scheme, based on the log quantity needed to be synchronized by different database backup machines, the response time which can be allowed by each database backup machine in theory in the synchronization process is calculated respectively, and further, the phenomenon of synchronization timeout of which database backup machines is judged to alarm, so that different synchronization conditions of different database backup machines can be controlled respectively, and the adaptability and the efficiency of data synchronization are further improved.
Example III
The third embodiment of the present application is a preferred embodiment provided on the basis of the foregoing embodiments, and fig. 3A is a schematic flow diagram of a standby policy brush disc synchronization provided by the embodiment of the present application, as shown in fig. 3A:
when the database backup machine receives a requestThe whole archive log of a complete transaction is composed of a plurality of event, and the number of the event in the archive log of the transaction received by the standby machine is smaller than a threshold value N events When the method is used, the transaction is used as granularity to perform flush and sync operations, so that the performance of the database backup machine is prevented from being influenced by frequent disk brushing; for inclusion of event numbers greater than N events Per receive N large transactions events The individual events perform one flush and sync operation (corresponding to a threshold of N events Event packet processing) to avoid performance jitter caused by centralized I/O operations while ensuring performance.
Fig. 3B is a schematic flow chart of checking timeout of standby response of a host according to an embodiment of the present application, as shown in fig. 3B:
since the amount of archive log synchronized for each transaction between the primary and the secondary is different in size. Before waking up the worker thread to synchronize the archive log, the responding processing thread will acquire the log S of the transaction requiring synchronization from the cache of the archive log log The proportionality coefficient of the waiting response timeout time is R timeout The adjustment amount is delta, and the lower limit of the waiting synchronization response timeout threshold is L timeout The wait response timeout time for each log sync is:
T timeout ＝max{R timeout *S log +Δ,L timeout }；
wherein the proportionality coefficient R timeout Adjustment amount delta and wait response timeout threshold lower limit L timeout All are configuration items which can be dynamically effective, and can be flexibly allocated according to different production environments.
When the database turns on the group commit mode, each time a set of transaction generated archive logs is synchronized. If the number of transactions synchronized together is n, then
Correspondingly, the waiting response timeout time of each log synchronization is:
when dump thread (synchronous thread) of host prepares to synchronously archive log to standby machine, response processing thread will calculate response timeout time T by using the above formula timeout The response timeout time for the corresponding transaction is set and polled once per second. When the standby response is not received by the timeout moment, the transaction is marked as a timeout transaction, and standby information responding to the timeout is recorded. The host sends all information of the current transaction synchronous response overtime to the alarm monitoring platform through the alarm module, and correspondingly adjusts the master-slave state information of the cluster.
According to the scheme provided by the embodiment of the application, the host is used for respectively adaptively processing the characteristics of different log amounts of the standby computers, so that the problems that in the prior art, the processing mode of a main and standby architecture is single and the main and standby states are abnormal due to different services are effectively avoided, and the availability of a database cluster is obviously enhanced; meanwhile, the log is subjected to strategic disc brushing in the standby machine part, so that the main and standby delay is effectively reduced, the overall performance is improved, and the stability of the performance is ensured on the premise of ensuring the consistency of main and standby data and the data reliability. And synchronous processing is carried out according to the granularity of the transaction, so that the consistency of the main and standby is ensured, and frequent network interaction is avoided.
Example IV
Fig. 4 is a schematic structural diagram of a database synchronization device according to a fourth embodiment of the present application. As shown in fig. 4, the apparatus 400 includes:
an archive log obtaining module 410, configured to obtain an archive log of a target transaction sent by a database host; wherein the archive log consists of at least one event;
a synchronization mode determining module 420, configured to determine a standby synchronization mode according to the number of events in the archive log;
and the standby machine data synchronization module 430 is configured to synchronize data of the database host machine to the database standby machine according to the standby machine synchronization mode, and feed back a synchronization response to the database host machine.
According to the technical scheme of the embodiment of the application, the synchronization mode is determined according to the number of the events in the archive log so as to synchronize the data in the database host to the database standby, so that the performance jitter caused by frequent disk brushing operation and I/O operation can be avoided while the processing performance stability of the database is ensured, and the synchronization efficiency between the database hosts and the database standby is improved.
In an alternative embodiment, the synchronization mode determining module 420 may include:
the event group determining unit is used for grouping the events according to the number threshold value if the number of the events is larger than a preset number threshold value, so as to obtain at least one event group to be synchronized;
the granularity determining unit is used for respectively taking each event group to be synchronized as granularity for synchronization.
In another alternative embodiment, the synchronization mode determining module 420 may specifically be configured to:
and if the number of the events is not greater than the number threshold, using the target transaction as a granularity for synchronization.
The database synchronization device provided by the embodiment of the application can execute the database synchronization method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the database synchronization methods.
Example five
Fig. 5 is a schematic structural diagram of a database synchronization device according to a fifth embodiment of the present application. As shown in fig. 5, the apparatus 500 includes:
the log amount obtaining module 510 is configured to obtain a log amount of an archive log corresponding to the target transaction;
the response time length determining module 520 is configured to determine a theoretical response time length of the target transaction according to the log amount, the number of the target transactions, and a preset influence parameter, and send the archive log to a database backup machine;
and the synchronization timeout judging module 530 is configured to determine whether the database standby machine has a transaction synchronization timeout according to the actual response duration and the theoretical response duration of the feedback synchronization response provided by the database standby machine, so as to perform timeout alarm.
According to the technical scheme, based on the log quantity needed to be synchronized by different database backup machines, the response time which can be allowed by each database backup machine in theory in the synchronization process is calculated respectively, and further, the phenomenon of synchronization timeout of which database backup machines is judged to alarm, so that different synchronization conditions of different database backup machines can be controlled respectively, and the adaptability and the efficiency of data synchronization are further improved.
In an alternative embodiment, the preset influencing parameters include: scaling factor, adjustment amount, and timeout threshold lower limit.
Further, the response time determining module 520 may include:
the time length calculation unit is used for determining and calculating the response time length according to the log quantity, the proportionality coefficient and the adjustment quantity;
and the theoretical duration determining unit is used for taking the larger one of the calculated response duration and the timeout threshold lower limit as the theoretical response duration.
The database synchronization device provided by the embodiment of the application can execute the database synchronization method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the database synchronization methods.
Example six
Fig. 6 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 6, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as database synchronization methods.
In some embodiments, the database synchronization method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the database synchronization method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the database synchronization method in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present application, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present application are achieved, and the present application is not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (10)
1. A database synchronization method, applied to a database backup machine, the method comprising:
acquiring an archive log of a target transaction sent by a database host; wherein the archive log consists of at least one event;
determining a standby machine synchronization mode according to the number of events in the archive log;
and synchronizing the data of the database host to the database standby according to the standby synchronization mode, and feeding back a synchronization response to the database host.
2. The method of claim 1, wherein determining a standby synchronization pattern based on the number of events in the archive log comprises:
if the number of the events is larger than a preset number threshold, grouping the events according to the number threshold to obtain at least one event group to be synchronized;
and respectively taking each event group to be synchronized as granularity for synchronization.
3. The method of claim 2, wherein determining the backup synchronization based on the number of events in the archive log comprises:
and if the number of the events is not greater than the number threshold, using the target transaction as a granularity for synchronization.
4. A database synchronization method, applied to a database host, the method comprising:
acquiring the log quantity of the archive log corresponding to the target transaction;
determining theoretical response time length of the target transaction according to the log quantity, the number of the target transaction and preset influence parameters, and sending the archive log to a database backup machine;
and determining whether the database standby machine generates transaction synchronization timeout according to the actual response time length and the theoretical response time length of the feedback synchronization response provided by the database standby machine so as to carry out timeout alarm.
5. The method of claim 4, wherein the preset influencing parameters comprise: scaling factor, adjustment amount, and timeout threshold lower limit.
6. The method of claim 5, wherein determining the theoretical response time of the target transaction according to the log amount, the number of target transactions, and a preset impact parameter comprises:
determining and calculating response time according to the log quantity, the proportionality coefficient and the adjustment quantity;
and taking the larger one of the calculated response time length and the overtime threshold lower limit as the theoretical response time length.
7. A database synchronization apparatus for use with a database backup machine, the apparatus comprising:
the archive log acquisition module is used for acquiring archive logs of target transactions sent by the database host; wherein the archive log consists of at least one event;
the synchronous mode determining module is used for determining a standby synchronous mode according to the number of events in the archive log;
and the standby machine data synchronization module is used for synchronizing the data of the database host machine to the database standby machine according to the standby machine synchronization mode and feeding back a synchronization response to the database host machine.
8. A database synchronization apparatus for use with a database host, the apparatus comprising:
the log quantity acquisition module is used for acquiring the log quantity of the archive log corresponding to the target transaction;
the response time length determining module is used for determining the theoretical response time length of the target transaction according to the log quantity, the number of the target transaction and the preset influence parameter, and sending the archive log to a database backup machine;
and the synchronous timeout judging module is used for determining whether the database standby machine generates the transaction synchronous timeout or not according to the actual response time length and the theoretical response time length of the feedback synchronous response provided by the database standby machine so as to carry out timeout alarm.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the database synchronization method of any one of claims 1-3; and/or performing the database synchronization method of any of claims 4-6.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the database synchronization method of any one of claims 1-3; and/or implementing the database synchronization method of any of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311094664.4A CN117112695A (en) | 2023-08-29 | 2023-08-29 | Database synchronization method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311094664.4A CN117112695A (en) | 2023-08-29 | 2023-08-29 | Database synchronization method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117112695A true CN117112695A (en) | 2023-11-24 |
Family
ID=88797975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311094664.4A Pending CN117112695A (en) | 2023-08-29 | 2023-08-29 | Database synchronization method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117112695A (en) |
-
2023
- 2023-08-29 CN CN202311094664.4A patent/CN117112695A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115291806A (en) | Processing method, processing device, electronic equipment and storage medium | |
| CN115146000A (en) | Database data synchronization method and device, electronic equipment and storage medium | |
| CN117725115A (en) | Database sequence processing method, device, equipment and storage medium | |
| CN117633116A (en) | Data synchronization method, device, electronic equipment and storage medium | |
| CN117667144A (en) | Annotating hot refreshing method, annotating hot refreshing device, annotating hot refreshing equipment and annotating hot refreshing medium | |
| CN117314068A (en) | Tobacco shred task scheduling method and device, electronic equipment and storage medium | |
| CN116383207A (en) | Data tag management method and device, electronic equipment and storage medium | |
| CN115103024B (en) | Sequence number generation method and device, electronic equipment and storage medium | |
| CN117112695A (en) | Database synchronization method and device, electronic equipment and storage medium | |
| CN115525666A (en) | Real-time data updating method and device, electronic equipment and storage medium | |
| CN114924882A (en) | Distributed service processing method, device, equipment and storage medium | |
| CN115757047A (en) | Cloud platform state synchronization method, multi-cloud management platform and storage medium | |
| CN114691781A (en) | Data synchronization method, system, device, equipment and medium | |
| CN113961641A (en) | Database synchronization method, device, equipment and storage medium | |
| CN116974825A (en) | Backup method, device, equipment and storage medium | |
| CN115599869B (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN116016265B (en) | Message all-link monitoring method, device, system, equipment and storage medium | |
| CN115203139A (en) | Log query method and device, electronic equipment and storage medium | |
| CN115269128A (en) | Distributed transaction processing method, device, equipment and storage medium | |
| CN116450411A (en) | Method, device, equipment and storage medium for determining replication delay state | |
| CN117519983A (en) | Memory-based data processing method and device | |
| CN115438099A (en) | Data reading method and device, database node and medium | |
| CN117785403A (en) | Heartbeat task execution device for equipment, electronic equipment and storage medium | |
| CN115174447A (en) | Network communication method, device, system, equipment and storage medium | |
| CN115442432A (en) | Control method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |