CN113836230A - Log analysis service switching method and device under database main and standby environment - Google Patents

Log analysis service switching method and device under database main and standby environment Download PDF

Info

Publication number
CN113836230A
CN113836230A CN202111114468.XA CN202111114468A CN113836230A CN 113836230 A CN113836230 A CN 113836230A CN 202111114468 A CN202111114468 A CN 202111114468A CN 113836230 A CN113836230 A CN 113836230A
Authority
CN
China
Prior art keywords
database
data synchronization
equipment
time
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111114468.XA
Other languages
Chinese (zh)
Inventor
孙峰
余院兰
徐娜
彭青松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Dream Database Co Ltd
Wuhan Dameng Database Co Ltd
Original Assignee
Wuhan Dream Database Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Dream Database Co Ltd filed Critical Wuhan Dream Database Co Ltd
Priority to CN202111114468.XA priority Critical patent/CN113836230A/en
Publication of CN113836230A publication Critical patent/CN113836230A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2477—Temporal data queries

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to a method and equipment for switching log analysis services in a database main-standby environment. The method mainly comprises the following steps: creating an auxiliary table for updating the current time column at regular time on the equipment of the source end main database; data synchronization services are deployed on both devices of a source end main database and a target end database; the source end data synchronization service acquires the current data snapshot time of the equipment database; the source end data synchronization service processes the acquired current data snapshot time according to the difference of the main state and the standby state; the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization. The method provided by the invention does not depend on judging the state of the database to realize the switching of the log analysis service, but judges whether to start the log analysis service preferentially or not by judging the current data snapshot time of the database on the current equipment.

Description

Log analysis service switching method and device under database main and standby environment
[ technical field ] A method for producing a semiconductor device
The invention relates to the technical field of computers, in particular to a method and equipment for switching log analysis services in a database main-standby environment.
[ background of the invention ]
At present, heterogeneous database replication technology based on database log analysis is widely applied. The technology captures the incremental data of the database at the source end, then sends the incremental data to the target end, and applies the incremental data to the target database at the target end through a general database access interface to realize data replication. The technology uses a universal database interface, so that heterogeneous database system replication is supported, heterogeneous operating system environments are supported, and the database system of the target standby equipment can read and write, so that the system is a 'double-active' system.
In order to achieve high reliability, a source database usually builds a master database (e.g., Oracle DataGuard) to achieve high availability. When data synchronization is built under the environment, the log analysis service is built on the main equipment and the standby equipment, but the log analysis service is started on only one equipment to realize the data synchronization. When the standby equipment fails (for example, the standby equipment is down or the log analysis service is abnormally down), the log analysis service on the main equipment takes over the data synchronization work, and the log analysis service is started to analyze logs and then synchronize.
In the prior art for establishing data synchronization in a primary and secondary database environment, although a data synchronization log analysis service is established on a source-end database master device and a secondary device at the same time, in order to reduce the influence of the log analysis service on the performance of the database, the log analysis service is generally operated on the secondary device, and only when the secondary device is abnormal, the data synchronization on the database master device starts the log analysis service to take over, so that the high availability of the data synchronization in the primary and secondary database environment is realized. However, the existing technical scheme has a problem that when a primary and secondary environment of a source-side database is built, a dedicated network may be used for the primary device and the secondary device (for example, the primary and secondary devices are directly connected by a dedicated network card), when a fault occurs in the dedicated network, the primary and secondary devices of the source-side database cannot communicate with each other, which may cause the synchronization between the primary and secondary devices of the database to be interrupted, and the secondary device of the database cannot receive the REDO log of the primary device to realize synchronization, so that the data of the secondary device is not real-time. If the log analysis service of data synchronization is still running on the standby device, the newly added data modification of the main device cannot be synchronized to the target end of the data synchronization, and the data synchronization is interrupted.
In view of this, how to overcome the defects existing in the prior art, and solve the problem of interruption of synchronization between source data and target data caused by a failure of a primary/standby dedicated network of a source database, are problems to be solved in the technical field.
[ summary of the invention ]
Aiming at the defects or the improvement requirements of the prior art, the invention provides a method and equipment for switching log analysis services in a database master-slave environment, wherein source-end data synchronization services are respectively operated on source-end master-slave database equipment, and the data synchronization services are not realized only by judging the state of the database when judging whether the log analysis services of the local equipment need to be started or not (the existing scheme is that if the database on the current equipment is in the state of the slave equipment, the log analysis services are preferentially started), but the log analysis services are determined to be preferentially started or not by judging the current data snapshot time of the database on the current equipment. The current data snapshot time refers to the latest modification time of the data in the database.
The embodiment of the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for switching log parsing services in a primary and standby database environment, including:
creating an auxiliary table for updating the current time column at regular time on the equipment of the source end main database;
data synchronization services are deployed on both devices of a source end main database and a target end database;
the source end data synchronization service inquires a current time column in the auxiliary table of the equipment so as to obtain the current data snapshot time of the equipment database;
the source end data synchronization service acquires the state of the equipment database in the main and standby environment, and processes the acquired current data snapshot time according to the difference of the main and standby states;
the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization.
Further, the creating of the auxiliary table on the device of the source-side master database for updating the current time column in a timing manner specifically includes:
creating an auxiliary table on equipment of a source-end main database, and inserting current system time into a time column of the auxiliary table;
a process is initiated to periodically update the current system time in the time column of the secondary table every second.
Further, the deploying the data synchronization service on both the device of the source-side active-standby database and the device of the target-side database specifically includes:
data synchronization services comprising log analysis services are respectively deployed on two devices of a source-end master-slave database, and the log analysis services of the source-end data synchronization services are used for reading logs from the source-end database, analyzing the logs and then sending the logs to a target end;
and deploying a data synchronization service on the device of the target end database, wherein the target end data synchronization service is used for applying the synchronization operation sent by the source end to the target end database.
Further, the querying, by the source data synchronization service, a current time column in the auxiliary table of the device, so as to obtain the current data snapshot time of the device database specifically includes:
the source end data synchronization service inquires a time column of an auxiliary table in a device database, and the obtained current system time is used as the current data snapshot time;
and converting the current data snapshot time into an integer taking seconds as a unit after the current data snapshot time is acquired.
Further, the source data synchronization service acquiring the state of the device database in the active/standby environment, and processing the acquired current data snapshot time according to the difference of the active/standby state specifically includes:
the source end data synchronization service acquires the state of the equipment database in the main and standby environment, and judges whether the equipment database is a main database or a standby database;
if the current data snapshot time is the standby database, adding N seconds to the acquired current data snapshot time, and if the current data snapshot time is the main database, keeping the current data snapshot time unchanged;
wherein the larger the N configuration is, the longer the synchronous failover time is; the smaller the N configuration, the shorter the synchronous failover time.
Further, the source data synchronization service registers the current data snapshot time processed by the device with the target data synchronization service, and if the registration is successful, starting the log analysis service to perform log analysis and data synchronization specifically includes:
the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service;
the target terminal data synchronization service judges whether the registration is successful after receiving the registration request command;
the source end data synchronization service receives a registration result returned by the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization; and if the registration fails, the current data snapshot time of the equipment database is obtained again after sleeping for S seconds and registration is applied, wherein the S seconds represent the interval time of timing attempt registration.
Further, the specific judgment rule for judging whether the registration is successful after the target data synchronization service receives the registration request command includes:
if no other registration node exists in the target end data synchronization service, allowing the registration request of the current registration equipment and returning the success of registration;
if the current data snapshot time of the registered equipment is less than the current data snapshot time of the registered equipment, closing a synchronous channel of the registered equipment, allowing a registration request of the registered equipment, and returning registration success; and if the current data snapshot time of the registered equipment is greater than the current data snapshot time of the current registered equipment, rejecting the registration request of the current registered equipment and returning registration failure.
Further, the method also comprises the following steps:
after the log analysis service is started, a scheduling thread is established, the current data snapshot time of the equipment database is obtained every 1 second and sent to the target end data synchronization service;
after receiving the current data snapshot time of the equipment database, the target end data synchronization service finds the registration information of the corresponding equipment and updates the current data snapshot time of the corresponding equipment;
and if so, adding N seconds to the current data snapshot time and then sending the current data snapshot time to the target end for data synchronization service.
Further, the method also comprises the following steps:
after the log analysis service is started, if the link channel is found to be closed when the synchronous data is sent, the current data snapshot time of the equipment database is obtained again and registration is applied.
On the other hand, the invention provides a device for switching log analysis services in a primary and standby database environment, which specifically comprises the following steps: the log analysis service switching method comprises at least one processor and a memory, wherein the at least one processor is connected with the memory through a data bus, the memory stores instructions capable of being executed by the at least one processor, and the instructions are used for completing the log analysis service switching method under the active and standby database environments in the first aspect after being executed by the processor.
Compared with the prior art, the invention has the beneficial effects that: the device serving as the master synchronization service is determined by respectively comparing the current data snapshot time of the source master-slave database, and the decision is made by the data synchronization service of the target end, so that the selection of the source synchronization device can be effectively carried out without depending on a special network between the source master-slave devices. In the technical scheme of the invention, after the special network fails for a certain time, the current data snapshot time of the standby equipment is definitely less than that of the main equipment, so that the data synchronization service on the standby equipment can be switched to the main equipment of the database, the main equipment starts the log analysis service, and the data synchronization of a target end is realized.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a flowchart of a method for switching log parsing services in a primary and standby database environment according to embodiment 1 of the present invention;
FIG. 2 is a flowchart illustrating a step 100 provided in embodiment 1 of the present invention;
FIG. 3 is a flowchart of step 200 provided in embodiment 1 of the present invention;
FIG. 4 is a flowchart illustrating a step 300 according to embodiment 1 of the present invention;
FIG. 5 is a flowchart illustrating a step 400 provided in embodiment 1 of the present invention;
FIG. 6 is a flowchart illustrating a step 500 provided in embodiment 1 of the present invention;
FIG. 7 is a flowchart illustrating a step 600 provided in embodiment 1 of the present invention;
fig. 8 is a schematic structural diagram of a device for switching log analysis services in a primary/standby database environment according to embodiment 3 of the present invention.
[ detailed description ] embodiments
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The present invention is a system structure of a specific function system, so the functional logic relationship of each structural module is mainly explained in the specific embodiment, and the specific software and hardware implementation is not limited.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other. The invention will be described in detail below with reference to the figures and examples.
Example 1:
as shown in fig. 1, an embodiment of the present invention provides a method for switching log analysis services in a primary and standby database environment, which includes the following specific steps.
Step 100: and creating an auxiliary table on the device of the source-end main database, wherein the auxiliary table is used for updating the current time column in a timing mode.
Step 200: data synchronization service is deployed on both devices of the source end main and standby databases and the target end database.
Step 300: the source end data synchronization service inquires the current time column in the auxiliary table of the device, so as to obtain the current data snapshot time of the device database.
Step 400: the source end data synchronization service acquires the state of the equipment database in the main and standby environment, and processes the acquired current data snapshot time according to the difference of the main and standby states.
Step 500: the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization.
Through the steps, the embodiment of the invention abandons the defect that the data synchronization service is realized only by judging the state of the database when the main and standby environments of the traditional source end database are built, and sends the current data snapshot time of the main equipment and the standby equipment to the target end data synchronization service, judges who the registration is successful through the target end data synchronization service, and gives priority to the successful registration so as to realize the main and standby switching function of the log analysis service. Therefore, even if the private network between the main equipment and the standby equipment fails and the standby equipment cannot update data in real time, the main equipment can still accept the synchronous operation again according to the registration judgment of the target end data synchronization service, and the interruption of data synchronization cannot be caused.
As shown in fig. 2, in the preferred embodiment, the step 100 (creating an auxiliary table on the device of the source-side master database for updating the current time column at regular time) specifically includes:
step 101: creating a secondary table on the device of the source-side main database, and inserting the current system time into the time column of the secondary table.
Step 102: a process is initiated to periodically update the current system time in the time column of the secondary table every second.
For the above steps of this embodiment, in the active/standby environment of the database, the primary device database is readable and writable, the standby device database provides read-only service, and under a normal condition, the standby device database is synchronized with the primary device database in real time, that is, under a normal condition of the primary/standby database, a certain data is written in the primary device of the database, and the standby device can immediately query the data. In the database which can not directly obtain the current log time through the LSN conversion of the database log, the current system time can be inserted by creating an auxiliary table on a database system, and then the current system time in the auxiliary table is updated on the main equipment of the main and standby databases at regular time, so that the current log time (consistent with the current system time) of the standby equipment can be obtained on the standby equipment of the database by inquiring the auxiliary table, and the log time of the latest data of the standby database can be obtained through the time column of the auxiliary table in a phase-changing manner.
Because the source-end main database has a time column for regularly updating the auxiliary table, and updates the time column to the latest system time (namely, the current system time) of the equipment, the latest data snapshot time can be obtained by inquiring the time column in the main database; the time column of the auxiliary table is not updated regularly by the source side standby database, the value in the time column is synchronized from the main database, the data snapshot time of data synchronization can be obtained by inquiring the time column in the standby database, and the difference between the time column and the value in the auxiliary table time column of the main database is the synchronization delay time between the source side database and the main database.
As shown in fig. 3, in the preferred embodiment, step 200 (data synchronization service is deployed on both devices of the active-standby source database and the target-side database) specifically includes:
step 201: and respectively deploying data synchronization services containing log analysis services on two devices of the source-side active/standby database. In this step, the log parsing service of the source-side data synchronization service is used to read the log from the source-side database, parse the log, and send the parsed log to the target side.
Step 202: and deploying a data synchronization service on the equipment of the target-side database. In this step, the target data synchronization service is configured to apply the synchronization operation sent by the source end to the target database, that is, the data synchronized by the source end to the target end is received by the target data synchronization service and is synchronized to the target database. In addition, the target end data synchronization service is further configured to interact with the source end data synchronization services of the active/standby devices, for example, respectively obtain current data snapshot times sent by the source end data synchronization services of the active/standby devices.
It should be noted that the two steps are parallel steps, belong to a data synchronization service deployment phase in the embodiment of the present invention, and are not in a sequential order during actual deployment.
As shown in fig. 4, in the preferred embodiment, the step 300 (the source data synchronization service queries the current time column in the device auxiliary table, so as to obtain the current data snapshot time of the device database) specifically includes:
step 301: and the source end data synchronization service inquires a time column of an auxiliary table in the device database, and the obtained current system time is used as the current data snapshot time.
Step 302: and converting the current data snapshot time into an integer taking seconds as a unit after the current data snapshot time is acquired.
In the above steps of this preferred embodiment, because the source-side database is built as a master-slave environment, and because the source-side database is built as a master-slave environment, the current system time obtained from the auxiliary table on the master device is the current system time regularly updated per second in step 102; on the backup device, the data on the backup device is synchronously restored through the REDO log of the database, so that the current system time in the auxiliary table acquired on the backup device can directly reflect the master-backup synchronization state of the database, and under the condition that the master-backup synchronization is normal, the time acquired by inquiring the auxiliary table from the backup device is consistent with the time acquired by inquiring the master device, which indicates that the master-backup synchronization of the database has no fault. In addition, the current data snapshot time is converted into an integer in units of seconds, so that the comparison between the current data snapshot time on the main device and the current data snapshot time on the standby device is performed for the convenience of the data synchronization service at the target end.
As shown in fig. 5, in the preferred embodiment, the step 400 (the source data synchronization service acquiring the state of the local device database in the active-standby environment, and processing the acquired current data snapshot time according to the difference of the active-standby state) specifically includes:
step 401: the source end data synchronization service acquires the state of the equipment database in the main and standby environment, and judges whether the equipment database is a main database or a standby database.
Step 402: and if the current data snapshot time is the master database, adding N seconds to the obtained current data snapshot time, and if the current data snapshot time is the master database, keeping the current data snapshot time unchanged.
In the foregoing steps of this preferred embodiment, when the source-side database is in the primary-standby environment, in order to not affect the performance of the source-side database system as much as possible, the log analysis service of the data synchronization service should be run on the standby database device of the primary-standby database as much as possible, so that when the current device is obtained as the standby device of the database, the current data snapshot time needs to be added by N seconds, so that the standby database has a chance of preferentially taking a master role, when a synchronization failure problem occurs between the primary and standby devices, the primary device can take over the data synchronization service after N seconds, where N can be configured according to the requirement of the actual synchronization environment, and the larger the N configuration, the longer the synchronization failure switching time; the smaller the N configuration, the shorter the synchronous failover time.
In the preferred embodiment, N may be configured as a dynamic configuration during configuration, specifically, the data synchronization on the standby database device may determine the value of N according to the amount of logs read per second, and when the amount of logs read per second is large, it indicates that there are a lot of operations on the database at this time, and the pressure of the master database device is large, so the value configuration of N is increased, and the host for data synchronization is left on the device of the standby database as much as possible, so as to avoid adding load to the device on the master database. When the amount of the log read per second is small or equal to 0, it may be that the load of the master database is small or the network is interrupted, and the value configuration of N should be reduced, so that if the load of the master database is small, the switching of the master of the data synchronization to the master database device does not cause too much load to the master database device, and if the load of the master database is network interrupted, the switching of the master of the data synchronization to the master database device is more important to continue the data synchronization.
As shown in fig. 6, in the preferred embodiment, step 500 (the source data synchronization service registers the current data snapshot time processed by the device with the target data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization) specifically includes:
step 501: and the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service. In this step, if the device is a device where the standby database is located, that is, a standby device, the time of the sent current data snapshot is the time after adding N seconds; if the device is the device where the main database is located, namely the main device, the sent current data snapshot time does not need to be processed.
Step 502: and the target terminal data synchronization service judges whether the registration is successful or not after receiving the registration request command. The specific judgment rule for judging whether the registration is successful in this step includes: if no other registration node exists in the target end data synchronization service, allowing the registration request of the current registration equipment and returning the success of registration; if the current data snapshot time of the registered equipment is less than the current data snapshot time of the registered equipment, closing a synchronous channel of the registered equipment, allowing a registration request of the registered equipment, and returning registration success; and if the current data snapshot time of the registered equipment is greater than the current data snapshot time of the current registered equipment, rejecting the registration request of the current registered equipment and returning registration failure.
Step 503: and the source end data synchronization service receives a registration result returned by the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization. In this step, if the registration fails, after sleeping for S seconds, the current data snapshot time of the local device database is obtained again, and registration is applied (i.e., the loop is repeated from step 300), where S seconds represents the interval time for attempting registration at regular time. It should be noted that after the registration fails, it is described that the other device has a priority level, and the current device enters the alternative state, so that it needs to inquire whether the log resolution service can be taken over by trying to register regularly, and this function can be realized by specifying a certain sleep time S, where S can configure an appropriate value according to actual needs.
In the preferred embodiment, the value of S may be dynamically configured to be an appropriate value according to the current device. When data synchronously run on the standby database equipment, S can be set to be larger, because the standby equipment has the priority to master at the moment; when data are synchronously operated on the main database equipment, S can be set to be small, so that the data synchronization host can be switched to the standby database equipment in time to relieve load of the main database equipment.
In the preferred embodiment, in addition to the basic steps of the above steps 100 to 500, the present embodiment further includes step 600: and the target end data synchronization service updates the current data snapshot time of the registered equipment at regular time.
Specifically, as shown in fig. 7, in the preferred embodiment, the step 600 specifically includes:
step 601: after the log analysis service is started, a scheduling thread is created, the current data snapshot time of the equipment database is obtained every 1 second and sent to the target end data synchronization service. In this step, when obtaining the current data snapshot time of the local device database, the scheduling thread determines whether the local device database belongs to the standby database, and if so, adds N seconds to the current data snapshot time and then sends the current data snapshot time to the target end for data synchronization service.
Step 602: and after receiving the current data snapshot time of the equipment database, the target-end data synchronization service finds the registration information of the corresponding equipment and updates the current data snapshot time of the corresponding equipment.
In the foregoing steps of the preferred embodiment, the target-side data synchronization service updates the current data snapshot time registered by the corresponding device at regular time, and ensures that the current data snapshot time of the device is in a latest state. The data synchronization service on the other device of the source main and standby continuously sends a registration request to inquire whether the device can be used as a master, and if the current data snapshot time of the existing device which is used as the master synchronously is not updated, the log analysis services on the source main and standby databases are continuously switched.
In the preferred embodiment, the method further comprises the following steps: after the log analysis service is started, if the link channel is found to be closed when the synchronous data is sent, the current data snapshot time of the equipment database is obtained again and registration is applied. Specifically, when the device where the current data synchronization service is located at the source end is used as the master, if another device has a higher master priority at this time, it will successfully register in the data synchronization service at the target end, and at this time, the link channel established between the device where the source end is used as the master and the target end is closed. When the original data synchronization service which is taken as the owner finds that the link channel is closed, the current data snapshot time of the device database is obtained again, registration is applied, and the qualification of the owner is tried to be obtained again.
In summary, in the present embodiment, in an environment where the source-side database is active and standby, the data snapshot time (i.e., the current system time) of the standby device is synchronized by the primary device, so that the current data snapshot time directly obtained from the database on the current device is the time of synchronizing data. The function of switching between the main database and the standby database can be realized by judging the current data snapshot time of the main database and the standby database and setting the operation master with the current data snapshot time. When the data synchronization service of the target terminal is registered, the characteristic that the data synchronization on the standby equipment database is prioritized can be realized by adding a reasonable value to the current data snapshot time on the standby equipment database, and the method is simple and efficient. In this case, as long as the failure time exceeds the set value N, the data synchronization service on the primary device will have a longer time than the data snapshot registered on the secondary device after the current data snapshot time is N seconds, so as to obtain the opportunity of taking the primary device.
Secondly, on the database which does not support the conversion from the database log LSN to the current log time, the time column in the auxiliary table is regularly updated on the database host computer every second, and then the log time (namely the data snapshot time) of the latest data in the database log file can be obtained in a phase-changing manner by inquiring the time column in the auxiliary table. The synchronization state and the synchronization delay between the main database and the standby database can be obtained through the time, and the main computer which needs to be used for data synchronization is judged without depending on the self state of the main database and the standby database, so that the problem of data asynchronization of a data synchronization system caused by special network faults in the environment of the main database and the standby database is effectively solved.
Example 2:
based on the method for switching log parsing services in the active/standby database environment provided in embodiment 1, embodiment 2 describes the present invention in more detail through a specific application scenario.
In this embodiment, taking the dreams database as an example, the whole data synchronization environment is:
the source database uses dreams active-standby environment (adopts REDO log to carry out active-standby synchronization).
The target database is an ORACLE database.
Assuming that both N and S are configured with fixed values of 5 seconds and 3 seconds, respectively
Specifically, the operation steps of this embodiment 2 based on embodiment 1 are as follows:
a secondary table T is created on the primary database device of the darmon database and inserted with the current time.
CREATE TABLE T(C DATETIME)；
INSERT INTO T(C)VALUES(SYSDATE)；
And creating a process, and regularly updating the time column C in the auxiliary table T every second to be the current system time.
UPDATE T SET C＝SYSDATE；
And respectively building a data synchronization service on a main database and a standby database (the main database and the standby database correspond to the main equipment) of the Dameng database for log analysis.
And building a data synchronization service on the ORACLE database to receive and store the log operation of the source database.
And starting a data synchronization service on the main equipment of the Dameng database, and acquiring the current system time of a time column C in the current database auxiliary table T so as to acquire the current data snapshot time.
Obtaining a statement of the current data snapshot time: SELECT C FROM T;
and judging the main and standby states of the current database, and sending the acquired current data snapshot time to a target end data synchronization service for registration if the current database is the main device.
The target end data synchronization service has no other registration information yet and returns the successful registration according to the rule.
And after the data synchronization service on the database main equipment reaches dream is successfully registered at the target end, starting a log analysis service and sending data for synchronization.
And creating a scheduling thread, acquiring the current system time of a time column C in an auxiliary table T on the main equipment every second, sending the current system time as the current data snapshot time to a target end data synchronization service, and updating the current data snapshot time registered in a synchronization channel of the main equipment by the target end in a synchronous manner.
And starting a data synchronization service on the standby equipment of the Dameng database, and acquiring the current system time of a time column C in the current database auxiliary table T as the current data snapshot time.
And judging the main and standby states of the current database, wherein the current database is standby equipment, and sending the acquired current data snapshot time plus 5 seconds to a target end data synchronization service for registration.
The target end data synchronization service currently registers the synchronization channel on the source end database main device, but the current data snapshot time of the standby device registration is greater than the current data snapshot time on the main device channel after adding 5 seconds, so the target end data synchronization service closes the synchronization channel on the main device, and then returns registration success to the standby device, so that the standby device serves as a data synchronization main.
And after the log analysis is successfully started, the data synchronization service on the standby equipment of the Tomeng database creates a scheduling thread, acquires the current system time of a time column C in an auxiliary table T on the standby equipment every second as the current data snapshot time plus 5 seconds, and then sends the current data snapshot time to the target end data synchronization service to update the current data snapshot time registered in the synchronization channel of the standby equipment.
And after the synchronization channel of the data synchronization service on the main equipment of the database in dream is disconnected, the current system time of the time column C in the auxiliary table T is periodically re-acquired as the current data snapshot time, and the data synchronization service of the target end is tried to be registered, wherein the data snapshot time of the standby equipment on the target end is always updated to be the latest data snapshot time plus a value of 5 seconds, so that the registration is unsuccessful.
And (4) disconnecting the special network between the main and standby source-end dreams database, and manufacturing the asynchronous fault of the main and standby databases. At this time, the current system time of the time column C in the auxiliary table T of the source-end-to-dream database standby device is no longer updated, and the current data snapshot time of the standby device will not change. In this case, the current data snapshot time registered by the data synchronization channel on the source-side dreams database backup device will not change any more, and the current data snapshot time on the source-side dreams database master device is still increasing.
After the data synchronization service channel on the main device of the dream database is disconnected, the current system time of the time column C in the auxiliary table T is regularly acquired as the current data snapshot time every 3 seconds, the data synchronization service tries to register to the target end data synchronization service, and after the private network between the main device and the standby device of the source dream database is disconnected for 5 seconds, the current data snapshot time used on the main device of the database is larger than the current data snapshot time registered on the standby device of the current dream database.
The target end data synchronization service currently registers a synchronization channel on the source end database standby equipment, but the current data snapshot time registered by the standby equipment stays before 5 seconds, so the target end data synchronization service closes the synchronization channel on the standby equipment, then returns registration success to the main equipment, and enables the main equipment to serve as a data synchronization master, thereby solving the problem that log analysis service on the database main equipment does not take over after the source end database main and standby special network fails.
It can be seen from the above steps that, in this embodiment, the device serving as the master synchronization service is determined by respectively comparing the current data snapshot times of the source master and slave databases, and the decision is made by the data synchronization service of the target, which can effectively select the source synchronization device without depending on the dedicated network between the source master and slave devices. In the technical scheme of the invention, after the special network fails for a certain time, the current data snapshot time of the standby equipment is definitely less than that of the main equipment, so that the data synchronization service on the standby equipment can be switched to the main equipment of the database, the main equipment starts the log analysis service, and the data synchronization of a target end is realized.
Example 3:
on the basis of the method for switching log analysis services in the primary and secondary database environments provided in embodiments 1 to 2, the present invention further provides a device for switching log analysis services in the primary and secondary database environments, which is used for implementing the method, and as shown in fig. 8, the device is a schematic diagram of a device architecture according to an embodiment of the present invention. The device for switching log parsing services in the primary and standby database environments of the present embodiment includes one or more processors 21 and a memory 22. In fig. 8, one processor 21 is taken as an example.
The processor 21 and the memory 22 may be connected by a bus or other means, and fig. 8 illustrates the connection by a bus as an example.
The memory 22 is a non-volatile computer-readable storage medium, and can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the method and system for switching log parsing services in the active/standby database environment in embodiments 1 to 2. The processor 21 executes various functional applications and data processing of the device for switching the log analysis service in the primary and standby database environments by operating the nonvolatile software program, instructions, and modules stored in the memory 22, that is, the method for switching the log analysis service in the primary and standby database environments in embodiments 1 to 2 is implemented.
The memory 22 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Program instructions/modules are stored in memory 22 and, when executed by one or more processors 21, perform the method for switching log parsing service in the active-standby environment of the database in embodiments 1 to 2, for example, perform the steps illustrated in fig. 1 to 7 described above.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the embodiments may be implemented by associated hardware as instructed by a program, which may be stored on a computer-readable storage medium, which may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (10)

1. A method for switching log analysis service under the environment of main and standby databases is characterized by comprising the following steps:
creating an auxiliary table for updating the current time column at regular time on the equipment of the source end main database;
data synchronization services are deployed on both devices of a source end main database and a target end database;
the source end data synchronization service inquires a current time column in the auxiliary table of the equipment so as to obtain the current data snapshot time of the equipment database;
the source end data synchronization service acquires the state of the equipment database in the main and standby environment, and processes the acquired current data snapshot time according to the difference of the main and standby states;
the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization.
2. The method of claim 1, wherein the step of creating an auxiliary table on a device of the source-side primary database for updating the current time column periodically comprises:
creating an auxiliary table on equipment of a source-end main database, and inserting current system time into a time column of the auxiliary table;
a process is initiated to periodically update the current system time in the time column of the secondary table every second.
3. The method of switching log resolution services under active-standby database environment according to claim 1, wherein said deploying data synchronization services on both devices of the active-standby source-side database and the target-side database specifically comprises:
data synchronization services comprising log analysis services are respectively deployed on two devices of a source-end master-slave database, and the log analysis services of the source-end data synchronization services are used for reading logs from the source-end database, analyzing the logs and then sending the logs to a target end;
and deploying a data synchronization service on the device of the target end database, wherein the target end data synchronization service is used for applying the synchronization operation sent by the source end to the target end database.
4. The method according to claim 2, wherein the querying, by the source data synchronization service, a current time column in the auxiliary table of the device to obtain a current data snapshot time of the device database specifically includes:
the source end data synchronization service inquires a time column of an auxiliary table in a device database, and the obtained current system time is used as the current data snapshot time;
and converting the current data snapshot time into an integer taking seconds as a unit after the current data snapshot time is acquired.
5. The method according to claim 1, wherein the source data synchronization service acquires a state of the device database in the active-standby environment, and processing the acquired snapshot time of the current data according to a difference between the active-standby state specifically includes:
the source end data synchronization service acquires the state of the equipment database in the main and standby environment, and judges whether the equipment database is a main database or a standby database;
if the current data snapshot time is the standby database, adding N seconds to the acquired current data snapshot time, and if the current data snapshot time is the main database, keeping the current data snapshot time unchanged;
wherein the larger the N configuration is, the longer the synchronous failover time is; the smaller the N configuration, the shorter the synchronous failover time.
6. The method according to claim 1, wherein the source data synchronization service registers the current data snapshot time processed by the device with the target data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization specifically includes:
the source end data synchronization service registers the current data snapshot time processed by the equipment to the target end data synchronization service;
the target terminal data synchronization service judges whether the registration is successful after receiving the registration request command;
the source end data synchronization service receives a registration result returned by the target end data synchronization service, and if the registration is successful, the log analysis service is started to perform log analysis and data synchronization; and if the registration fails, the current data snapshot time of the equipment database is obtained again after sleeping for S seconds and registration is applied, wherein the S seconds represent the interval time of timing attempt registration.
7. The method according to claim 6, wherein the specific rule for determining whether the registration is successful after the target data synchronization service receives the registration request command includes:
if no other registration node exists in the target end data synchronization service, allowing the registration request of the current registration equipment and returning the success of registration;
if the current data snapshot time of the registered equipment is less than the current data snapshot time of the registered equipment, closing a synchronous channel of the registered equipment, allowing a registration request of the registered equipment, and returning registration success; and if the current data snapshot time of the registered equipment is greater than the current data snapshot time of the current registered equipment, rejecting the registration request of the current registered equipment and returning registration failure.
8. The method for switching log resolution service in the active-standby environment of database according to any of claims 1-7, further comprising:
after the log analysis service is started, a scheduling thread is established, the current data snapshot time of the equipment database is obtained every 1 second and sent to the target end data synchronization service;
after receiving the current data snapshot time of the equipment database, the target end data synchronization service finds the registration information of the corresponding equipment and updates the current data snapshot time of the corresponding equipment;
and if so, adding N seconds to the current data snapshot time and then sending the current data snapshot time to the target end for data synchronization service.
9. The method for switching log resolution service in the active-standby environment of database according to any of claims 1-7, further comprising:
after the log analysis service is started, if the link channel is found to be closed when the synchronous data is sent, the current data snapshot time of the equipment database is obtained again and registration is applied.
10. A log analysis service switching device under a database master-slave environment is characterized in that:
the method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected through a data bus, and the memory stores instructions capable of being executed by the at least one processor, and the instructions are used for completing the method for switching the log resolution service in the master and standby database environments according to any one of claims 1 to 9 after being executed by the processor.
CN202111114468.XA 2021-09-23 2021-09-23 Log analysis service switching method and device under database main and standby environment Pending CN113836230A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111114468.XA CN113836230A (en) 2021-09-23 2021-09-23 Log analysis service switching method and device under database main and standby environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111114468.XA CN113836230A (en) 2021-09-23 2021-09-23 Log analysis service switching method and device under database main and standby environment

Publications (1)

Publication Number Publication Date
CN113836230A true CN113836230A (en) 2021-12-24

Family

ID=78969252

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111114468.XA Pending CN113836230A (en) 2021-09-23 2021-09-23 Log analysis service switching method and device under database main and standby environment

Country Status (1)

Country Link
CN (1) CN113836230A (en)

Similar Documents

Publication Publication Date Title
US10833919B2 (en) Node device operation method, work status switching apparatus, node device, and medium
US7565572B2 (en) Method for rolling back from snapshot with log
US10895996B2 (en) Data synchronization method, system, and apparatus using a work log for synchronizing data greater than a threshold value
WO2017177941A1 (en) Active/standby database switching method and apparatus
CN100397803C (en) N+1 duplicates data real-time synchronising method
US20090240974A1 (en) Data replication method
GB2484086A (en) Reliability and performance modes in a distributed storage system
CN109189860A (en) A kind of active and standby increment synchronization method of MySQL based on Kubernetes system
CN108259227B (en) Data synchronization method of dual-computer hot standby interlocking system
CN104486319A (en) Real-time synchronization method and real-time synchronization system for configuration file applied to high-availability system
WO2018010501A1 (en) Global transaction identifier (gtid) synchronization method, apparatus and system, and storage medium
CN102937955A (en) Main memory database achieving method based on My structured query language (SQL) double storage engines
CN114268532A (en) Raft protocol-based election method, distributed system and storage medium
WO2014177085A1 (en) Distributed multicopy data storage method and device
CN108920301B (en) Data backup method and system, and data recovery method and system
CN102045187B (en) Method and equipment for realizing HA (high-availability) system with checkpoints
CN110750594B (en) Real-time cross-network database synchronization method based on mysql incremental log
US20100274758A1 (en) Data processing method, computer, and data processing program
CN114337944A (en) System-level main/standby redundancy general control method
CN113836231B (en) Log analysis service switching method and device under database main and standby environment
CN113836230A (en) Log analysis service switching method and device under database main and standby environment
CN110716828B (en) Database real-time backup method
CN111813606A (en) Fault-tolerant method, system, equipment and medium for double-node virtual machine
CN114564340B (en) High availability method for distributed software of aerospace ground system
US20050268188A1 (en) Backup method, backup system, disk controller and backup program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination