CN115421976A - Remote disaster recovery data processing method and device - Google Patents
Remote disaster recovery data processing method and device Download PDFInfo
- Publication number
- CN115421976A CN115421976A CN202211076743.8A CN202211076743A CN115421976A CN 115421976 A CN115421976 A CN 115421976A CN 202211076743 A CN202211076743 A CN 202211076743A CN 115421976 A CN115421976 A CN 115421976A
- Authority
- CN
- China
- Prior art keywords
- disaster recovery
- database
- storage system
- distributed storage
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1456—Hardware arrangements for backup
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2315—Optimistic concurrency control
- G06F16/2322—Optimistic concurrency control using timestamps
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a method and a device for processing remote disaster recovery data, which relate to the field of distribution and comprise the following steps: the method comprises the steps that log files of a production environment database are read in parallel through a plurality of links, operation records of the production environment database are obtained, and the operation records are sent to distributed message middleware to generate database execution statements of a remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; acquiring finished transaction records and replication time stamps on different databases on the same service chain, determining the consistent mirror image time of the next replication transaction and writing the consistent mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistent mirror image time; according to the method and the device, the risk of data loss during the remote disaster recovery and account patching can be effectively reduced, and the high availability in the account patching process is improved.
Description
Technical Field
The application relates to the field of distribution, in particular to a method and a device for processing remote disaster recovery data.
Background
The same-city disaster recovery and post-accounting of the open platform generally adopts a high-availability design, the different-place disaster recovery and post-accounting generally adopts a disk-level replication technology to assist manual post-accounting, the influence on the transaction of a replicated source database is large, the problems of low replication efficiency, no support for heterogeneous databases and easy loss of partial data exist, the risk during post-accounting is high, and the business continuity level of external services of a bank is influenced.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a remote disaster recovery data processing method and device, which can effectively reduce the risk of data loss during remote disaster recovery and reconciliation and improve the high availability in the reconciliation process.
In order to solve at least one of the above problems, the present application provides the following technical solutions:
in a first aspect, the present application provides a method for processing disaster recovery data in a different place, including:
the method comprises the steps that log files of a production environment database are read in parallel through a plurality of links, operation records of the production environment database are obtained, and the operation records are sent to distributed message middleware to generate database execution statements of the remote disaster recovery environment;
sending the database execution statements to a distributed storage system according to a set frequency sequence;
and acquiring completed transaction records and replication time stamps on different databases on the same service chain, determining the consistency mirror image time of the next replication transaction and writing the consistency mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistency mirror image time.
Further, after the obtaining the operation record of the production environment database, the method includes:
splicing binary codes of the operation records of the production environment database into target character strings according to a set rule;
and sending the target character string to the distributed message middleware through a preset producer program.
Further, the generating of the database execution statement of the disaster recovery environment comprises:
acquiring a target character string in the distributed message middleware through a preset consumer program;
analyzing the target character string in the distributed message middleware, acquiring the operation record of the production environment and splicing to generate a database execution statement of the remote disaster recovery environment.
Further, before the obtaining of the completed transaction records and the time stamp of the copy on different databases in the same service chain, the method includes:
starting a multi-database time sequence control node;
and accessing the playback logs of each database in the remote disaster recovery environment through the multi-database time sequence control node.
Further, the accessing the distributed storage system and performing a unified data update operation according to the consistent mirror time includes:
accessing the distributed storage system at regular time to obtain the execution statement of the database to be executed and the next consistent mirror image time;
and uniformly updating the data from the current time point to the next consistent mirroring time into a database to perform the account-repairing operation of the batch.
Further, after the accessing the distributed storage system and performing a unified data update operation according to the consistent mirroring time, the method includes:
judging whether the distributed storage system has data to be reconciled;
if yes, a new batch of transaction data is copied.
In a second aspect, the present application provides a remote disaster recovery data processing apparatus, including:
the disaster recovery statement determining module is used for reading the log file of the production environment database in parallel by a plurality of links to obtain the operation record of the production environment database and sending the operation record to the distributed message middleware to generate a database execution statement of the remote disaster recovery environment;
the sequence distribution module is used for sending the database execution statements to a distributed storage system according to a set frequency sequence;
and the data updating module is used for acquiring the completed transaction records and the copying time stamps on different databases on the same service chain, determining the consistent mirror image time of the next copying transaction and writing the consistent mirror image time into the distributed storage system so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistent mirror image time.
In a third aspect, the present application provides an electronic device, which includes a memory, a processor, and a computer program that is stored in the memory and is executable on the processor, where the processor implements the steps of the remote disaster recovery data processing method when executing the program.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the remote disaster recovery data processing method.
In a fifth aspect, the present application provides a computer program product, which includes a computer program/instruction, and when the computer program/instruction is executed by a processor, the computer program/instruction implements the steps of the remote disaster recovery data processing method.
According to the technical scheme, the application provides a method and a device for processing remote disaster recovery data, wherein log files of a production environment database are read in parallel through a plurality of links to obtain operation records of the production environment database, and the operation records are sent to a distributed message middleware to generate database execution statements of a remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining completed transaction records and copying timestamps on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out data unified updating operation according to the consistency mirror image time, the risk of data loss during remote disaster recovery and account patching can be effectively reduced, and the high availability in the account patching process is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following descriptions are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flow chart of a remote disaster recovery data processing method in an embodiment of the present application;
fig. 2 is a second schematic flow chart of the remote disaster recovery data processing method in the embodiment of the present application;
fig. 3 is a third schematic flow chart of the remote disaster recovery data processing method in the embodiment of the present application;
fig. 4 is a fourth schematic flow chart of the remote disaster recovery data processing method in the embodiment of the present application;
fig. 5 is a fifth flowchart illustrating a remote disaster recovery data processing method in an embodiment of the present application;
fig. 6 is a sixth schematic flow chart of the remote disaster recovery data processing method in the embodiment of the present application;
fig. 7 is a structural diagram of a remote disaster recovery data processing apparatus in an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
According to the technical scheme, the data acquisition, storage, use, processing and the like meet the relevant regulations of national laws and regulations.
In view of the problems in the prior art, the application provides a method and a device for processing remote disaster recovery data, wherein a log file of a production environment database is read in parallel through a plurality of links to obtain an operation record of the production environment database, and the operation record is sent to a distributed message middleware to generate a database execution statement of a remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining completed transaction records and copying timestamps on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out data unified updating operation according to the consistency mirror image time, the risk of data loss during remote disaster recovery and account patching can be effectively reduced, and the high availability in the account patching process is improved.
In order to effectively reduce the risk of data loss during the remote disaster recovery and make up account and improve the high availability in the make up account process, the application provides an embodiment of a remote disaster recovery data processing method, and referring to fig. 1, the remote disaster recovery data processing method specifically includes the following contents:
step S101: and reading the log file of the production environment database in parallel by a plurality of links to obtain the operation record of the production environment database, and sending the operation record to the distributed message middleware to generate a database execution statement of the remote disaster recovery environment.
Optionally, the present application may read log files of databases (such as ORACLE, mySQL, etc.) in the production environment in parallel through multiple links, extract binary codes from the log files, and analyze the binary codes to obtain operation records (such as UPDATE, DELETE, INSERT, etc.) of the production environment databases.
Optionally, the binary codes of the operation records may be spliced according to a certain rule to generate the target character string. For example, the binary code of the inserted data in the operation record corresponds to the fifth bit to the tenth bit in the log file, and the binary code of the fifth bit to the tenth bit is converted to generate a target character string (such as a Json character string) which can be recognized by the distributed message center. And writing a producer program, and sending the generated multilink target character string to the distributed message middleware. Meanwhile, the unique index value can be used as a key value, so that the sequentiality of sending data to the distributed message middleware is ensured.
Step S102: and sending the database execution statements to a distributed storage system according to a set frequency sequence.
Optionally, the application may write a consumer program to obtain the target character string in the distributed message middleware. According to preset analysis conditions, analyzing a target character string in the distributed message middleware to obtain operation records of a production environment, splicing to generate database execution statements (such as UPDATE, DELETE, INSERT and the like) of the remote disaster recovery environment, and if the target character string is found not to meet the current analysis conditions in the analysis process, selecting whether to skip the operation record or not through a predefined configuration file and storing the record into an exception table.
After the target character string is analyzed, the obtained execution statements are sequentially sent to a distributed storage system (such as ETCD, zooKeeper, and the like) at certain time intervals (such as 200 MS).
Step S103: and acquiring the finished transaction records and the copying time stamps on different databases on the same service chain, determining the consistent mirror image time of the next copying transaction and writing the consistent mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistent mirror image time.
Optionally, the application may start a multi-database timing control node, access a log playback unit of each database in the disaster recovery environment in the remote area, obtain completed transaction records SQL and replication timestamps on different databases in the same service chain, take the latest replication timestamp as the next replication transaction consistency time point, and if the latest replication timestamp of each replication link is completely consistent, add a preset time interval to the current latest replication timestamp as the next replication transaction consistency time point. And after the consistency time point of the next replication transaction is obtained, writing the result into the distributed storage system.
Optionally, in the application, the reconciliation module of each application system can access the distributed storage system at regular time to obtain the database execution statement to be executed and the next consistent mirroring time, and uniformly update the data from the current time point to the next consistent mirroring time into the database to complete the reconciliation of the batch.
As can be seen from the above description, in the remote disaster recovery data processing method provided in the embodiment of the present application, the log file of the production environment database can be read in parallel through multiple links, the operation record of the production environment database is obtained, and the operation record is sent to the distributed message middleware to generate a database execution statement in the remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining completed transaction records and copying timestamps on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out data unified updating operation according to the consistency mirror image time, the risk of data loss during remote disaster recovery and account patching can be effectively reduced, and the high availability in the account patching process is improved.
In an embodiment of the remote disaster recovery data processing method according to the present application, referring to fig. 2, the following may be specifically included:
step S201: and splicing the binary codes of the operation records of the production environment database into a target character string according to a set rule.
Step S202: and sending the target character string to the distributed message middleware through a preset producer program.
Optionally, the binary codes of the operation records may be spliced according to a certain rule to generate the target character string. For example, the binary code of the inserted data in the operation record corresponds to the fifth bit to the tenth bit in the log file, and the binary code of the fifth bit to the tenth bit is converted to generate a target character string (such as a Json character string) which can be recognized by the distributed message middle. And writing a producer program, and sending the generated multilink target character string to the distributed message middleware. Meanwhile, the unique index value can be used as a key value, so that the sequentiality of data sent to the distributed message middleware is guaranteed.
In an embodiment of the remote disaster recovery data processing method according to the present application, referring to fig. 3, the following may be specifically included:
step S301: and acquiring the target character string in the distributed message middleware through a preset consumer program.
Step S302: and analyzing the target character string in the distributed message middleware, acquiring the operation record of the production environment, and splicing to generate a database execution statement of the remote disaster recovery environment.
Optionally, the application may write a consumer program to obtain the target character string in the distributed message middleware. Analyzing a target character string in the distributed message middleware according to a preset analysis condition, acquiring an operation record of a production environment, splicing to generate a database execution statement (such as UPDATE, DELETE, INSERT and the like) of the remote disaster recovery environment, and if the target character string is found not to meet the current analysis condition in the analysis process, selecting whether to skip the operation record or not through a predefined configuration file and storing the record into an exception table.
In an embodiment of the remote disaster recovery data processing method according to the present application, referring to fig. 4, the following contents may be further specifically included:
step S401: and starting the multi-database sequential control node.
Step S402: and accessing the playback logs of each database in the remote disaster recovery environment through the multi-database time sequence control node.
Optionally, the application may start a multi-database timing control node, access a log playback unit of each database in the disaster recovery environment in the remote area, obtain completed transaction records SQL and replication timestamps on different databases in the same service chain, take the latest replication timestamp as the next replication transaction consistency time point, and if the latest replication timestamp of each replication link is completely consistent, add a preset time interval to the current latest replication timestamp as the next replication transaction consistency time point. And after the next replication transaction consistency time point is obtained, writing the result into the distributed storage system.
In an embodiment of the remote disaster recovery data processing method according to the present application, referring to fig. 5, the following may be specifically included:
step S501: and accessing the distributed storage system at regular time to obtain the execution statement of the database to be executed and the next consistent mirror image time.
Step S502: and uniformly updating the data from the current time point to the next consistent mirroring time into a database to perform the account-repairing operation of the batch.
Optionally, the distributed storage system may be accessed at regular time to obtain the database execution statement to be executed and the next consistent mirror image time, and the data from the current time point to the next consistent mirror image time is updated to the database in a unified manner to complete the reconciliation of the batch.
In an embodiment of the remote disaster recovery data processing method according to the present application, referring to fig. 6, the following contents may be further specifically included:
step S601: and judging whether the distributed storage system has data to be reconciled.
Step S602: if yes, a new batch of transaction data is copied.
Optionally, after the completion of the reconciliation of the batch, the method and the device can judge whether the distributed storage system has data to be reconciled, if so, copy a new batch of transaction data, and effectively ensure the time consistency of all databases when an unplanned disaster occurs by continuously advancing the consistency mirror images of all databases.
In order to effectively reduce the risk of data loss during the remote disaster recovery and make up account, and improve the high availability in the make up account process, the application provides an embodiment of a remote disaster recovery data processing apparatus for implementing all or part of the contents of the remote disaster recovery data processing method, and referring to fig. 7, the remote disaster recovery data processing apparatus specifically includes the following contents:
and the disaster recovery statement determining module 10 is configured to read log files of the production environment database in parallel through multiple links, obtain operation records of the production environment database, and send the operation records to the distributed message middleware to generate database execution statements of the remote disaster recovery environment.
And the sequence distribution module 20 is configured to send the database execution statements to the distributed storage system according to a set frequency sequence.
And the data updating module 30 is configured to obtain transaction records and replication timestamps that have been completed on different databases in the same service chain, determine consistency mirror image time of a next replication transaction, and write the consistency mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs a data unified updating operation according to the consistency mirror image time.
As can be seen from the above description, the remote disaster recovery data processing apparatus provided in this embodiment of the present application can read a log file of a production environment database in parallel through multiple links, obtain an operation record of the production environment database, and send the operation record to the distributed message middleware to generate a database execution statement in a remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining transaction records and copying time stamps which are finished on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out unified data updating operation according to the consistency mirror image time, the risk of data loss during remote disaster preparation and account compensation can be effectively reduced, and the high availability in the account compensation process is improved.
In terms of hardware, in order to effectively reduce the risk of data loss during disaster recovery from a different place and reimbursement and improve the high availability in the reimbursement process, the present application provides an embodiment of an electronic device for implementing all or part of the contents in the method for processing disaster recovery data from a different place, where the electronic device specifically includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete mutual communication through the bus; the communication interface is used for realizing information transmission between the remote disaster recovery data processing device and relevant equipment such as a core service system, a user terminal, a relevant database and the like; the logic controller may be a desktop computer, a tablet computer, a mobile terminal, and the like, but the embodiment is not limited thereto. In this embodiment, the logic controller may refer to an embodiment of the remote disaster recovery data processing method and an embodiment of the remote disaster recovery data processing apparatus in the embodiment, and the contents thereof are incorporated herein, and repeated details are not repeated.
It is understood that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), a vehicle-mounted device, a smart wearable device, and the like. Wherein, intelligence wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..
In practical applications, part of the remote disaster recovery data processing method may be executed on the electronic device side as described in the above, or all operations may be completed in the client device. The selection may be specifically performed according to the processing capability of the client device, the limitation of the user usage scenario, and the like. This is not a limitation of the present application. The client device may further include a processor if all operations are performed in the client device.
The client device may have a communication module (i.e., a communication unit) and may be communicatively connected to a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.
Fig. 8 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 8, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this FIG. 8 is exemplary; other types of structures may also be used in addition to or in place of the structures to implement telecommunications or other functions.
In one embodiment, the remote disaster recovery data processing method function may be integrated into the central processor 9100.
The central processor 9100 may be configured to control as follows:
step S101: and reading the log file of the production environment database in parallel by a plurality of links to obtain the operation record of the production environment database, and sending the operation record to the distributed message middleware to generate a database execution statement of the remote disaster recovery environment.
Step S102: and sending the database execution statements to a distributed storage system according to a set frequency sequence.
Step S103: and acquiring the finished transaction records and the copying time stamps on different databases on the same service chain, determining the consistent mirror image time of the next copying transaction and writing the consistent mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistent mirror image time.
As can be seen from the above description, in the electronic device provided in the embodiment of the present application, the log file of the production environment database is read in parallel through multiple links, the operation record of the production environment database is obtained, and the operation record is sent to the distributed message middleware to generate the database execution statement of the remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining transaction records and copying time stamps which are finished on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out unified data updating operation according to the consistency mirror image time, the risk of data loss during remote disaster preparation and account compensation can be effectively reduced, and the high availability in the account compensation process is improved.
In another embodiment, the remote disaster recovery data processing apparatus may be configured separately from the central processor 9100, for example, the remote disaster recovery data processing apparatus may be configured as a chip connected to the central processor 9100, and the remote disaster recovery data processing method function is realized by the control of the central processor.
As shown in fig. 8, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worthy to note that the electronic device 9600 also does not necessarily include all of the components shown in fig. 8; in addition, the electronic device 9600 may further include components not shown in fig. 8, which may be referred to in the prior art.
As shown in fig. 8, the central processor 9100, which is sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, the central processor 9100 receives input and controls the operation of various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, but is not limited to, an LCD display.
The memory 9140 can be a solid state memory, e.g., read Only Memory (ROM), random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present application further provides a computer-readable storage medium capable of implementing all the steps in the disaster recovery data processing method for a remote location where an execution subject in the above embodiment is a server or a client, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps in the disaster recovery data processing method for a remote location where an execution subject in the above embodiment is a server or a client, for example, when the processor executes the computer program, the processor implements the following steps:
step S101: and reading the log file of the production environment database in parallel by a plurality of links to obtain the operation record of the production environment database, and sending the operation record to the distributed message middleware to generate a database execution statement of the remote disaster recovery environment.
Step S102: and sending the database execution statements to a distributed storage system according to a set frequency sequence.
Step S103: and acquiring completed transaction records and replication time stamps on different databases on the same service chain, determining the consistency mirror image time of the next replication transaction and writing the consistency mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistency mirror image time.
As can be seen from the above description, the computer-readable storage medium provided in this embodiment reads, through multiple links, a log file of a production environment database in parallel, obtains an operation record of the production environment database, and sends the operation record to the distributed message middleware to generate a database execution statement in a remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining transaction records and copying time stamps which are finished on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out unified data updating operation according to the consistency mirror image time, the risk of data loss during remote disaster preparation and account compensation can be effectively reduced, and the high availability in the account compensation process is improved.
The embodiments of the present application further provide a computer program product capable of implementing all steps in the remote disaster recovery data processing method with a server or a client as an execution subject in the above embodiments, where the computer program/instruction is executed by a processor to implement the steps of the remote disaster recovery data processing method, for example, the computer program/instruction implements the following steps:
step S101: and reading the log file of the production environment database in parallel by a plurality of links to obtain the operation record of the production environment database, and sending the operation record to the distributed message middleware to generate a database execution statement of the remote disaster recovery environment.
Step S102: and sending the database execution statements to a distributed storage system according to a set frequency sequence.
Step S103: and acquiring the finished transaction records and the copying time stamps on different databases on the same service chain, determining the consistent mirror image time of the next copying transaction and writing the consistent mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistent mirror image time.
As can be seen from the above description, in the computer program product provided in the embodiment of the present application, the log file of the production environment database is read in parallel through multiple links, so as to obtain the operation record of the production environment database, and the operation record is sent to the distributed message middleware to generate the database execution statement in the remote disaster recovery environment; sending the database execution statements to a distributed storage system according to a set frequency sequence; the method comprises the steps of obtaining transaction records and copying time stamps which are finished on different databases on the same service chain, determining the consistency mirror image time of the next copying transaction and writing the consistency mirror image time into the distributed storage system, so that each application system can access the distributed storage system and carry out unified data updating operation according to the consistency mirror image time, the risk of data loss during remote disaster preparation and account compensation can be effectively reduced, and the high availability in the account compensation process is improved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A remote disaster recovery data processing method is characterized by comprising the following steps:
the method comprises the steps that log files of a production environment database are read in parallel through a plurality of links, operation records of the production environment database are obtained, and the operation records are sent to distributed message middleware to generate database execution statements of the remote disaster recovery environment;
sending the database execution statements to a distributed storage system according to a set frequency sequence;
and acquiring completed transaction records and replication time stamps on different databases on the same service chain, determining the consistency mirror image time of the next replication transaction and writing the consistency mirror image time into the distributed storage system, so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistency mirror image time.
2. The disaster recovery ex-situ data processing method according to claim 1, wherein after said obtaining of the operation record of the production environment database, comprising:
splicing binary codes of the operation records of the production environment database into target character strings according to set rules;
and sending the target character string to distributed message middleware through a preset producer program.
3. The remote disaster recovery data processing method according to claim 1, wherein the generating of the database execution statement of the remote disaster recovery environment comprises:
acquiring a target character string in the distributed message middleware through a preset consumer program;
analyzing the target character string in the distributed message middleware, acquiring the operation record of the production environment and splicing to generate a database execution statement of the remote disaster recovery environment.
4. The disaster recovery remote data processing method according to claim 1, wherein before said obtaining the completed transaction records and the time stamps of the copies of the transaction records and the time stamps of the copies of the transaction records, the method comprises:
starting a multi-database time sequence control node;
and accessing the playback logs of each database in the remote disaster recovery environment through the multi-database time sequence control node.
5. The disaster recovery remote data processing method according to claim 1, wherein said accessing said distributed storage system and performing a unified data update operation according to said consistent mirroring time comprises:
accessing the distributed storage system at regular time to obtain the execution statement of the database to be executed and the next consistent mirror image time;
and uniformly updating the data from the current time point to the next consistent mirroring time into a database to perform the account-repairing operation of the batch.
6. The disaster recovery displaced data processing method according to claim 1, wherein after said accessing said distributed storage system and performing a unified data update operation according to said consistent mirroring time, said method comprises:
judging whether the distributed storage system has data to be reconciled;
if yes, a new batch of transaction data is copied.
7. A remote disaster recovery data processing device is characterized by comprising:
the disaster recovery statement determining module is used for reading the log file of the production environment database in parallel by a plurality of links to obtain the operation record of the production environment database and sending the operation record to the distributed message middleware to generate a database execution statement of the remote disaster recovery environment;
the sequence distribution module is used for sending the database execution statements to a distributed storage system according to a set frequency sequence;
and the data updating module is used for acquiring the completed transaction records and the copying time stamps on different databases on the same service chain, determining the consistent mirror image time of the next copying transaction and writing the consistent mirror image time into the distributed storage system so that each application system accesses the distributed storage system and performs data unified updating operation according to the consistent mirror image time.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the disaster recovery data processing method according to any one of claims 1 to 6 when executing the program.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the disaster recovery data processing method according to any one of claims 1 to 6.
10. A computer program product comprising computer program/instructions, characterized in that the computer program/instructions, when executed by a processor, implement the steps of the disaster recovery data processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076743.8A CN115421976A (en) | 2022-09-05 | 2022-09-05 | Remote disaster recovery data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211076743.8A CN115421976A (en) | 2022-09-05 | 2022-09-05 | Remote disaster recovery data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115421976A true CN115421976A (en) | 2022-12-02 |
Family
ID=84202907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211076743.8A Pending CN115421976A (en) | 2022-09-05 | 2022-09-05 | Remote disaster recovery data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115421976A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117874145A (en) * | 2024-03-13 | 2024-04-12 | 连连（杭州）信息技术有限公司 | Strong agreement method, device, equipment and storage medium for master-slave database |
-
2022
- 2022-09-05 CN CN202211076743.8A patent/CN115421976A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117874145A (en) * | 2024-03-13 | 2024-04-12 | 连连（杭州）信息技术有限公司 | Strong agreement method, device, equipment and storage medium for master-slave database |
| CN117874145B (en) * | 2024-03-13 | 2024-05-28 | 连连（杭州）信息技术有限公司 | Strong agreement method, device, equipment and storage medium for master-slave database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108268501B (en) | Service processing method and device in online data migration process | |
| CN111708841A (en) | Database data synchronization method and device | |
| CN112612851B (en) | Multi-center data synchronization method and device | |
| EP4213037A1 (en) | Data storage and reconciliation method and system | |
| CN113110829B (en) | Multi-UI component library data processing method and device | |
| CN112463634A (en) | Software testing method and device under micro-service architecture | |
| CN112394932A (en) | Automatic browser webpage skin changing method and device | |
| CN115421976A (en) | Remote disaster recovery data processing method and device | |
| CN114153856A (en) | Host data double-writing method and device | |
| CN111930624A (en) | Test link message data processing method and device | |
| CN115495519A (en) | Report data processing method and device | |
| CN114840576A (en) | Data standard matching method and device | |
| CN112905464B (en) | Application running environment data processing method and device | |
| CN113392091A (en) | Distributed cluster data migration method and device | |
| CN113791984A (en) | Automatic interface testing method and device | |
| CN114490402A (en) | Method and device for checking validity of message field | |
| CN113419957A (en) | Rule-based big data offline batch processing performance capacity scanning method and device | |
| CN113505076A (en) | File coverage detection method and device | |
| CN112766827A (en) | Automatic processing method and device for asset inventory data | |
| CN112965952B (en) | Data asset processing method and device | |
| CN113342501B (en) | System fault processing method and device | |
| CN112910970B (en) | Remote disaster recovery data synchronization method, device and system | |
| CN113342330B (en) | Front-end engineering generation method and device | |
| CN115563137A (en) | Stationary point data processing method and device | |
| CN113656380A (en) | Data life cycle management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |