CN112559247A - Database data backup method and system based on third-party assistance - Google Patents
Database data backup method and system based on third-party assistance Download PDFInfo
- Publication number
- CN112559247A CN112559247A CN202011510557.1A CN202011510557A CN112559247A CN 112559247 A CN112559247 A CN 112559247A CN 202011510557 A CN202011510557 A CN 202011510557A CN 112559247 A CN112559247 A CN 112559247A
- Authority
- CN
- China
- Prior art keywords
- binary
- log
- server
- backup
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000899 pressurised-fluid extraction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a database data backup method and a database data backup system based on third party assistance, wherein the method comprises the following steps: firstly, the method comprises the following steps: starting an I/O thread from the slave server, and configuring user information to be connected with the main server; II, secondly: reading the binary log of the master server from the slave server, and writing the read binary log into a local relay log; thirdly, the method comprises the following steps: and starting the SQL thread from the server to check the relay log at regular time, if a new binary log is found in the relay log, immediately executing new binary content by the local database, and backing up the new binary content to the local database. The invention can support the own intellectual property database by a newly-built information system, and simultaneously needs to provide the capability of implementing backup without excessively depending on foreign databases for the established information system, thereby ensuring the safety and the availability of data.
Description
Technical Field
The invention relates to the field, in particular to a database data backup method and system based on third-party assistance.
Background
With the rapid development of office automation and electronic commerce, the dependence of enterprises on information systems is higher and higher, and a database plays an important role as the core of the information systems. Especially in some industries where data reliability is a high requirement, such as banks, securities, telecommunications, etc., the loss of data can be very disastrous if unexpected shutdown or data loss occurs. Therefore, it is necessary to backup the database. Therefore, after the database is damaged due to faults, the database can be recovered through the backup database.
Database backup research is a demand-driven area in which large corporations have begun relatively early on, and some backup technologies have been in use for a considerable time in various application environments. Foreign research on backup software began in the mid-80's of the 20 th century, and commercial backup products matured to date include: tivoli from EMC, NetVault from BakBone, BrihtStor from CA, etc.
At present, the backup capability of domestic databases mostly depends on foreign data to implement data backup, autonomous backup cannot be realized, and storage capacity and flow are unbalanced, so that the utilization efficiency of storage space in the database backup process is low, and the storage space is wasted.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a database data backup method and system based on third-party assistance.
The purpose of the invention is realized by the following technical scheme:
a database data backup method based on third party assistance is characterized by comprising the following steps:
the method comprises the following steps: starting an I/O thread from the slave server, and configuring user information to be connected with the main server;
step two: reading the binary log of the master server from the slave server, and writing the read binary log into a local relay log;
step three: and starting the SQL thread from the server to check the relay log at regular time, if a new binary log is found in the relay log, immediately executing new binary content by the local database, and backing up the new binary content to the local database.
Specifically, the third step further includes that each slave server records a binary coordinate when executing new binary content, and the binary log coordinate includes a binary file name and a read and processed position of the current slave server in the binary file of the master server.
Specifically, the step two of backing up the new binary content to the local database further includes compressing the new binary log content as a backup result.
A third-party-assistance-based database data backup system, comprising: a master server and a slave server, wherein:
the main server is used for maintaining database data and storing all maintenance information into a binary log of the main server;
and the slave server is used for reading the binary log of the master server, executing backup in the local database according to the read content, and compressing the backed-up data to be used as a backup result to be stored in the local data.
Specifically, the slave server includes:
the backup module is used for reading the binary log of the main server and executing backup in the local database according to the read content;
and the compression module is used for compressing, packaging and numbering the executed binary logs as a backup result and storing the backup result into a local database.
The invention has the beneficial effects that: the invention can support the own intellectual property database by a newly-built information system, and simultaneously needs to provide the capability of implementing backup without excessively depending on foreign databases for the established information system, thereby ensuring the safety and the availability of data.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic block diagram of the apparatus of the present invention.
Fig. 3 is a gateway mode architecture diagram of the present invention.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings.
In this embodiment, as shown in fig. 1, a database data backup method based on third party assistance is characterized by including the following steps:
the method comprises the following steps: starting an I/O thread from the slave server, and configuring user information to be connected with the main server;
step two: reading the binary log of the master server from the slave server, and writing the read binary log into a local relay log;
step three: and starting the SQL thread from the server to check the relay log at regular time, if a new binary log is found in the relay log, immediately executing new binary content by the local database, and backing up the new binary content to the local database.
Specifically, the third step further includes that each slave server records a binary coordinate when executing new binary content, and the binary log coordinate includes a binary file name and a read and processed position of the current slave server in the binary file of the master server.
Specifically, the step two of backing up the new binary content to the local database further includes compressing the new binary log content as a backup result.
In the present invention, as shown in fig. 2, a database data backup system based on third party assistance includes: a master server and a slave server, wherein:
the main server is used for maintaining database data and storing all maintenance information into a binary log of the main server;
and the slave server is used for reading the binary log of the master server, executing backup in the local database according to the read content, and compressing the backed-up data to be used as a backup result to be stored in the local data.
Specifically, the slave server includes:
the backup module is used for reading the binary log of the main server and executing backup in the local database according to the read content;
and the compression module is used for compressing, packaging and numbering the executed binary logs as a backup result and storing the backup result into a local database.
The operation flow of the database data backup system comprises the following steps;
1. any modifications on the master server are kept in the Binary log by its own I/O threads.
2. And starting an I/O thread from the server, connecting to the main server through the configured user name and password to request to read the binary log, and writing the read binary log into a local read log (relay log).
3. And simultaneously opening an SQL thread timing check real log (the file is also binary) from the server, and immediately executing the updated content once on the local database if the update is found.
Each slave server receives a copy of the entire contents of the master server binary log. The slave server device is responsible for deciding which statements in the binary log should be executed. All events in the master-slave binary log are executed on the slave unless otherwise specified. Furthermore, the slave server may be configured to handle events for only some specific databases or tables. Each slave station (slave server) will record binary log coordinates including: the file name and the location in the file that the slave server has read and processed from the master site. Since each slave server records the position in the binary log currently processed by the slave server, the slave server can be disconnected, reconnected and then resumed to continue processing.
A gateway is an intermediary that decouples peers that cooperate with each other on a network and allows them to interact without directly relying on each other. According to the data integrity requirement of the database, the concept of a gateway mode is provided for realizing the data integrity of each node in the database.
As shown in fig. 3, the gateway mode mainly encapsulates the following three modes to ensure the integrity of data of each node in the network. An unblocked Buffered I/O mode that decouples the input mechanism from the output mechanism to properly and reliably route data without unduly blocking application processing. Active Object (Active Object) this mode decouples method execution from method calls to enhance concurrency and simplify synchronous access to objects residing in their own thread control. Ensuring that blocking read and write operations on a node do not reduce the transmission efficiency of other nodes. Receiver-connector (Acceptor-Co nnector) that decouples connection establishment and service initialization from service processing in the network system. The gateway model combines the components already provided by multiple ACEs, the important ones of which are briefly described below. Map Manager efficiently maps external ids (such as correspondent routing addresses) to internal ids (such as Consumer Handler). The gateway model uses the Map Manager to implement Routing tables that internally handle the demultiplexing and Routing of messages to the gateway. Message Queue component a Message Queue provides a common queuing mechanism used by gateways to buffer messages in a Consumer Handler while they wait to be routed to a peer.
When the gateway mode is used for realizing the communication module of the DB, multithreading is adopted for realizing the active object gateway, and the non-blocking buffer type I/O mode is used for improving the transmission efficiency of the updated data between the primary copy nodes. Meanwhile, the Message Queue in the Message queuing component is realized by adopting a Monitor Object mode, so that the communication module can run efficiently and robustly in a multi-thread environment.
(1) Suppler Handler, responsible for routing update data passed in the DB as incoming external messages to their destinations (i.e. the respective replica node associated with the primary replica node). When the Scheduler in the active object detects an event on the connected communication endpoint, it notifies the super Handler. After the provider Handler receives the complete Routing message from the endpoint, it queries the Routing Table to determine the Consumer Handler destination set for the message, and then requests the selected Consumer Handler to forward the message to the appropriate peer destination.
(2) Consumer Handler is responsible for reliably delivering routing messages (update data passed in the DB) to their destinations (i.e., the respective replica nodes associated with the primary replica node). The Consumer Handler is implemented using a non-blocking buffered I/O mode to prevent blocking from occurring when communicating with a node and affecting the overall system communication. Message Queue is used internally by the components to concatenate data that has not been sent due to temporary congestion on the communication line, in the order in which these messages must be delivered as allowed by the flow control mechanism. After the flow controlled connection is reopened, the ACE framework passes through the Consumer Handler, which begins to drain the Message Queue by sending a Message to the peer.
(3) The Consumer Handler is realized by adopting an active object mode with a multithread structure, and the same active object mode based on the multithread structure is also adopted in the supervisor Handler.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (5)
1. A database data backup method based on third party assistance is characterized by comprising the following steps:
the method comprises the following steps: starting an I/O thread from the slave server, and configuring user information to be connected with the main server;
step two: reading the binary log of the master server from the slave server, and writing the read binary log into a local relay log;
step three: and starting the SQL thread from the server to check the relay log at regular time, if a new binary log is found in the relay log, immediately executing new binary content by the local database, and backing up the new binary content to the local database.
2. The third-party-assisted database data backup method according to claim 1, wherein the third step further comprises recording binary coordinates when each slave server executes new binary content, the binary log coordinates comprising binary file name and read and processed position of current slave server in the master server binary file.
3. The database data backup method based on third party assistance as claimed in claim 1, wherein the step two of backing up the new binary contents into the local database further comprises compressing the new binary log contents as a backup result.
4. A third-party-assistance-based database data backup system, comprising:
the main server is used for maintaining database data and storing all maintenance information into a binary log of the main server;
and the slave server is used for reading the binary log of the master server, executing backup in the local database according to the read content, and compressing the backed-up data to be used as a backup result to be stored in the local data.
5. The third-party-assisted-based database data backup system according to claim 4, wherein the slave server comprises:
the backup module is used for reading the binary log of the main server and executing backup in the local database according to the read content;
and the compression module is used for compressing, packaging and numbering the executed binary logs as a backup result and storing the backup result into a local database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011510557.1A CN112559247A (en) | 2020-12-18 | 2020-12-18 | Database data backup method and system based on third-party assistance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011510557.1A CN112559247A (en) | 2020-12-18 | 2020-12-18 | Database data backup method and system based on third-party assistance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112559247A true CN112559247A (en) | 2021-03-26 |
Family
ID=75030465
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011510557.1A Pending CN112559247A (en) | 2020-12-18 | 2020-12-18 | Database data backup method and system based on third-party assistance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112559247A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110004586A1 (en) * | 2009-07-15 | 2011-01-06 | Lon Jones Cherryholmes | System, method, and computer program product for creating a virtual database |
| CN104657382A (en) * | 2013-11-21 | 2015-05-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting consistency of data of MySQL master and slave servers |
| CN106407356A (en) * | 2016-09-07 | 2017-02-15 | 网易（杭州）网络有限公司 | Data backup method and device |
| CN106933703A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method of database data backup, device and electronic equipment |
| CN112084161A (en) * | 2020-09-16 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Database-based data processing method and device and readable storage medium |
-
2020
- 2020-12-18 CN CN202011510557.1A patent/CN112559247A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110004586A1 (en) * | 2009-07-15 | 2011-01-06 | Lon Jones Cherryholmes | System, method, and computer program product for creating a virtual database |
| CN104657382A (en) * | 2013-11-21 | 2015-05-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting consistency of data of MySQL master and slave servers |
| CN106933703A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method of database data backup, device and electronic equipment |
| CN106407356A (en) * | 2016-09-07 | 2017-02-15 | 网易（杭州）网络有限公司 | Data backup method and device |
| CN112084161A (en) * | 2020-09-16 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Database-based data processing method and device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8676760B2 (en) | Maintaining data integrity in data servers across data centers | |
| CA2543493C (en) | System and method for failover | |
| US8886796B2 (en) | Load balancing when replicating account data | |
| US6735717B1 (en) | Distributed computing system clustering model providing soft real-time responsiveness and continuous availability | |
| CN102411639B (en) | Multi-copy storage management method and system of metadata | |
| US20150074052A1 (en) | Method and system of stateless data replication in a distributed database system | |
| JP2002522845A (en) | Fault tolerant computer system | |
| CN105493474B (en) | System and method for supporting partition level logging for synchronizing data in a distributed data grid | |
| US20070118840A1 (en) | Remote copy storage device system and a remote copy method | |
| WO2019210580A1 (en) | Access request processing method, apparatus, computer device, and storage medium | |
| CN101079896A (en) | A multi-availability mechanism coexistence framework of concurrent storage system | |
| CN110113406B (en) | Distributed computing service cluster system | |
| WO2019042174A1 (en) | Method for synchronization between primary database and standby database, database system and device | |
| US7093163B2 (en) | Processing takeover method in multiple computer system | |
| US8301750B2 (en) | Apparatus, system, and method for facilitating communication between an enterprise information system and a client | |
| CN112181723A (en) | Financial disaster recovery method and device, storage medium and electronic equipment | |
| CN112559247A (en) | Database data backup method and system based on third-party assistance | |
| CN101471955A (en) | Method for writing equipment data in dual-controller network storage circumstance | |
| CN111629111B (en) | Call management system and method | |
| JP2003318897A (en) | Communication system, connection management server device and program | |
| CN113641511B (en) | Message communication method and device | |
| WO2023280127A1 (en) | Message communication method and apparatus | |
| JP2002149439A (en) | Method for switching server and server device in distributed processing system | |
| CN117857572A (en) | High-availability architecture supporting second-level accurate push-forbidden | |
| CN201491023U (en) | Redundancy server structure based on RAC |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |