CN112099996A - Database cluster multi-node redo log recovery method based on page update sequence number - Google Patents
Database cluster multi-node redo log recovery method based on page update sequence number Download PDFInfo
- Publication number
- CN112099996A CN112099996A CN202010993792.2A CN202010993792A CN112099996A CN 112099996 A CN112099996 A CN 112099996A CN 202010993792 A CN202010993792 A CN 202010993792A CN 112099996 A CN112099996 A CN 112099996A
- Authority
- CN
- China
- Prior art keywords
- page
- redo log
- sequence number
- node
- scanning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Abstract
The invention relates to a database cluster multi-node redo log recovery method based on page update sequence numbers, which comprises the following steps: allocating space at the head of the page as a page updating sequence number; when all nodes in the cluster update the data page, updating the page updating sequence number and the redo log of the node; restarting the database cluster, wherein the node started first becomes a main node, and the main node performs instance recovery; the main node loads a control file from the shared disk, reads redo log information of each node from the control file as a scanning handle, and stores the redo log information in a scanning handle array; traversing all the scanning handles and recovering the redo log; and after all the scanning handles are scanned, recovering the redo log. When the redo log is newly added and the instance is recovered, the method judges whether the pages are continuous or not by using the page updating sequence number, and recovers the modification of the same page among different nodes in sequence, thereby ensuring the consistency of data and improving the reliability of system operation.
Description
Technical Field
The invention belongs to the technical field of databases, relates to database recovery, and particularly relates to a database cluster multi-node redo log recovery method based on page update sequence numbers.
Background
When a certain transaction wants to modify a certain row of data in the database, the database reads the related data page from the disk into the memory for modification. At this time, data is modified in memory, creating a difference compared to the page contents in disk, and this differentiated data page is called a dirty page.
The database processes the dirty page instead of refreshing the dirty page back to the disk every time the dirty page is generated, because a large number of random IO operations are generated during the process, the processing performance of the database is seriously affected. The database has a special page back-brushing thread, the data page in the memory can be regularly back-brushed to the disk, and the page becomes a clean page after being back-brushed. During the period from the generation of the dirty page to the back-flushing of the dirty page to the change of the clean page, if the database is accidentally down due to power failure, system failure and process crash, data errors can be caused, and the modification of the user is lost, so that the durability of the transaction cannot be ensured.
The database solves the above problem by redo log, ensuring transaction durability. When a transaction needs to modify a data page, the modified content of the transaction is recorded in a redo log file. When the database is restarted after being down, the database can be restored to a correct state by restoring the redo log.
In the shared storage cluster, each database node can read and write data files on the shared storage and independently provide services for the outside. If the nodes share one redo log file, competition is inevitably generated, and the performance of the database is influenced. Therefore, each database node in the cluster has its own redo log file. In the operation process, each node only accesses the redo log file of the node and does not access the redo log files of other nodes. When a node generates a redo log, the redo log is only written into the redo log belonging to the node, and the redo log writing is completely irrelevant among different nodes. By the method, interaction and waiting among the nodes are reduced, and the overall performance of the cluster is improved.
Under the environment of a single-machine database, the redo log is recovered only by sequentially recovering the log LSNs (note: the logical serial number of the log is increased along with the writing of the log) from small to large. Different from a single-machine environment, a plurality of nodes of a database cluster have redo logs of the nodes, and the LSNs among the logs cannot be directly compared in size, so that problems can be encountered when the redo logs are recovered. Examples are as follows:
as shown in FIG. 1a, consider the case when redo log recovery is performed, where one data page is modified by multiple instances, and two nodes in the graph alternate page modification.
For the same data page, if the redo log of one node is restored first and then another node is restored, the restoration is not performed according to the modification order of the page, resulting in that the last restored data is erroneous data, as shown in fig. 1 b.
For the same data page, the log of the modified page may be distributed in different redo log files, and the recovery must be performed according to the modification order of the page, so that the recovery is correct, as shown in fig. 1 c.
In summary, how to recover the modification of the same page between different nodes in order to ensure the consistency of data is a problem that needs to be solved urgently at present.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provides a database cluster multi-node redo log recovery method based on page update sequence numbers, which is reasonable in design, high in efficiency and capable of effectively guaranteeing the consistency of a cluster database.
The technical problem to be solved by the invention is realized by adopting the following technical scheme:
a database cluster multi-node redo log recovery method based on page update sequence numbers comprises the following steps:
and 6, finishing scanning all the scanning handles and finishing the recovery of the redo log.
Further, the step 1 allocates 8 byte spaces at the head of the page as the page update sequence number.
Further, the specific implementation method of step 2 is as follows: when the data page is modified, a redo log is added, the modified content and the current page update sequence number are written into the redo log, and then the page update sequence number value is increased by one.
Further, the control file loaded by the master node from the shared disk is a binary file defining the physical state of the database cluster.
Further, the specific implementation method of step 5 includes the following steps:
traversing the scanning handles, if all the scanning handles are scanned completely, jumping out of a cycle, and turning to the step 6;
secondly, if the current scanning handle is scanned completely, turning to the step 5, and starting the next redo log scanning;
and thirdly, circularly reading, analyzing and recovering each redo log from the current scanning handle.
6. The method for recovering the database cluster multi-node redo log based on the page update sequence number according to claim 5, characterized in that: the concrete implementation method of the step three is as follows:
if the end of the redo log is scanned, the current redo log is recovered, the current scanning handle is marked to be scanned completely, and the step 5 is carried out to start the next redo log scanning;
secondly, acquiring a redo log from the current scanning position of the scanning handle, acquiring a page updating sequence number from the redo log, comparing the page updating sequence number with the updating sequence number of the current page, and performing the following processing:
if the serial number recorded in the redo log is smaller than the update serial number of the current page, ignoring the log and continuing to scan the next log;
if the sequence number recorded in the redo log is equal to the update sequence number of the current page, applying the modification of the log to the data page, then adding one to the update sequence number value of the current page, and continuing to scan the next log;
and if the sequence number recorded in the redo log is greater than the update sequence number of the current page and the pages are discontinuous, sequentially updating the pages and recording the sequentially updated pages in the redo logs of other nodes, turning to the step 5, and starting scanning of the next redo log.
The invention has the advantages and positive effects that:
the invention has reasonable design, sets a page updating sequence number for marking the page updating sequence for each page, maintains the page updating sequence number when adding redo logs and recovering the instance, judges whether the pages are continuous or not by traversing each redo log and utilizing the page updating sequence number, and recovers the modification of the same page among different nodes in sequence, thereby ensuring the consistency of data and improving the reliability of system operation.
Drawings
FIG. 1a is a schematic diagram of a process in which a data page is modified by multiple instances;
FIG. 1b is a schematic diagram of a recovery process using a node sequence;
FIG. 1c is a schematic diagram of a recovery process using a page modification sequence;
FIG. 2 is a flow diagram of a write redo log of the present invention;
FIG. 3 is a flow chart of recovering redo logs of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The design idea of the invention is as follows: in order to be able to mark the update order of the pages, an update sequence number needs to be maintained for each page. Therefore, 8 bytes are allocated at the head of the page as a page update sequence number, representing the modification order on the page. After each data page update, the page update sequence number value is incremented by one. When the redo log is newly added, the current update sequence number of the page is also recorded into the redo log. When the instance is recovered, the redo log of one node is selected for recovery, and if the current page to be recovered is found to be discontinuous, the redo log is switched to the next redo log for recovery. And repeating the process until all the redo logs are recovered. By traversing each redo log and judging whether the pages are continuous or not by using the page update sequence number, the modification of the same page among different nodes can be restored in sequence, and the consistency of data is ensured.
Based on the design idea, the invention provides a database cluster multi-node redo log recovery method based on page update sequence numbers, which comprises the following steps:
In this embodiment, 8 byte spaces are allocated at the head of the page as the page update sequence number. And when a redo log is newly added, writing the modified content and the current page update sequence number into the redo log.
And 2, when all the nodes in the cluster update the data page, updating the page update sequence number and the redo log of the node.
As shown in fig. 1, the specific implementation method of this step is: when the data page is modified, a redo log is added, the modified content and the current page update sequence number are written into the redo log, and then the page update sequence number value is increased by one.
And 3, restarting the database cluster, wherein the node started firstly becomes a main node, and the main node performs instance recovery.
And 4, loading a control file (note: the control file is a binary file defining the physical state of the database cluster) from the shared disk by the main node, reading redo log information of each node from the control file as a scanning handle, and storing the redo log information in a scanning handle array.
And 5, traversing all the scanning handles and recovering the redo log.
As shown in fig. 3, the specific implementation method of this step includes the following steps:
if all the scanning handles are scanned completely, the process jumps out of the cycle and goes to step 6.
Secondly, if the current scanning handle is scanned completely, the next redo log scanning is started by going to the step 5.
Thirdly, circularly reading, analyzing and recovering each redo log from the current scanning handle, wherein the method comprises the following steps:
if the end of the redo log is scanned, the current redo log is recovered and the current scanning handle is marked to be scanned completely, and then the step 5 is carried out to start the next redo log scanning.
And acquiring a redo log from the current scanning position of the scanning handle. Acquiring a page updating sequence number from the redo log, comparing the page updating sequence number with the updating sequence number of the current page, and performing the following processing:
if the sequence number recorded in the redo log is smaller than the update sequence number of the current page, the redo log is indicated that the content of the redo log is returned to the page, the redo log is ignored, and the next log is continuously scanned.
If the sequence number recorded in the redo log is equal to the update sequence number of the current page, it indicates that the redo log is not refreshed to the page yet, and the redo log is updated in sequence, and the log modification is applied to the data page. And then adding one to the current page update sequence number value for advancing redo log recovery. The next log scan is continued.
If the serial number recorded in the redo log is greater than the update serial number of the current page, the page is discontinuous, which indicates that the redo log is not updated in sequence, the sequential update of the page is recorded in the redo logs of other nodes, and the step 5 is switched to, and the next redo log scanning is started.
And 6, finishing scanning all the scanning handles at the moment, and finishing the recovery of all the redo logs.
It should be emphasized that the embodiments described herein are illustrative rather than restrictive, and thus the present invention is not limited to the embodiments described in the detailed description, but also includes other embodiments that can be derived from the technical solutions of the present invention by those skilled in the art.
Claims (6)
1. A database cluster multi-node redo log recovery method based on page update sequence numbers is characterized by comprising the following steps: the method comprises the following steps:
step 1, allocating space at the head of a page as a page updating sequence number;
step 2, when all nodes in the cluster update the data page, updating the page updating sequence number and the redo log of the node;
step 3, restarting the database cluster, wherein the node started first becomes a main node, and the main node performs instance recovery;
step 4, the main node loads a control file from the shared disk, reads redo log information of each node from the control file as a scanning handle, and stores the redo log information in a scanning handle array;
step 5, traversing all scanning handles and recovering redo logs;
and 6, finishing scanning all the scanning handles and finishing the recovery of the redo log.
2. The method for recovering the database cluster multi-node redo log based on the page update sequence number according to claim 1, characterized in that: in the step 1, 8 byte spaces are allocated at the head of the page as the page updating sequence number.
3. The method for recovering the database cluster multi-node redo log based on the page update sequence number according to claim 1, characterized in that: the specific implementation method of the step 2 comprises the following steps: when the data page is modified, a redo log is added, the modified content and the current page update sequence number are written into the redo log, and then the page update sequence number value is increased by one.
4. The method for recovering the database cluster multi-node redo log based on the page update sequence number according to claim 1, characterized in that: and the control file loaded from the shared disk by the main node is a binary file defining the physical state of the database cluster.
5. The method for recovering the database cluster multi-node redo log based on the page update sequence number according to claim 1, characterized in that: the specific implementation method of the step 5 comprises the following steps:
traversing the scanning handles, if all the scanning handles are scanned completely, jumping out of a cycle, and turning to the step 6;
secondly, if the current scanning handle is scanned completely, turning to the step 5, and starting the next redo log scanning;
and thirdly, circularly reading, analyzing and recovering each redo log from the current scanning handle.
6. The method for recovering the database cluster multi-node redo log based on the page update sequence number according to claim 5, characterized in that: the concrete implementation method of the step three is as follows:
if the end of the redo log is scanned, the current redo log is recovered, the current scanning handle is marked to be scanned completely, and the step 5 is carried out to start the next redo log scanning;
secondly, acquiring a redo log from the current scanning position of the scanning handle, acquiring a page updating sequence number from the redo log, comparing the page updating sequence number with the updating sequence number of the current page, and performing the following processing:
if the serial number recorded in the redo log is smaller than the update serial number of the current page, ignoring the log and continuing to scan the next log;
if the sequence number recorded in the redo log is equal to the update sequence number of the current page, applying the modification of the log to the data page, then adding one to the update sequence number value of the current page, and continuing to scan the next log;
and if the sequence number recorded in the redo log is greater than the update sequence number of the current page and the pages are discontinuous, sequentially updating the pages and recording the sequentially updated pages in the redo logs of other nodes, turning to the step 5, and starting scanning of the next redo log.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010993792.2A CN112099996B (en) | 2020-09-21 | 2020-09-21 | Database cluster multi-node redo log recovery method based on page update sequence number |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010993792.2A CN112099996B (en) | 2020-09-21 | 2020-09-21 | Database cluster multi-node redo log recovery method based on page update sequence number |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112099996A true CN112099996A (en) | 2020-12-18 |
| CN112099996B CN112099996B (en) | 2022-04-05 |
Family
ID=73754733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010993792.2A Active CN112099996B (en) | 2020-09-21 | 2020-09-21 | Database cluster multi-node redo log recovery method based on page update sequence number |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112099996B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114020357A (en) * | 2021-11-04 | 2022-02-08 | 湖北美和易思教育科技有限公司 | Method, device, system and medium for starting namenode node |
| CN115550145A (en) * | 2022-12-01 | 2022-12-30 | 问策师信息科技南京有限公司 | Industrial Internet safety dynamic protection method and protection system |
| CN117131069A (en) * | 2023-10-26 | 2023-11-28 | 天津神舟通用数据技术有限公司 | Database list grouping scanning method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060117074A1 (en) * | 2004-11-30 | 2006-06-01 | Ezzat Ahmed K | Method and apparatus for database cluster recovery |
| CN102629250A (en) * | 2012-02-28 | 2012-08-08 | 杭州丰城信息技术有限公司 | Recovery method of redo log files for main memory database |
| CN102999564A (en) * | 2012-11-06 | 2013-03-27 | 华为技术有限公司 | Method, device and equipment for writing in data |
| CN105279045A (en) * | 2015-10-28 | 2016-01-27 | 浪潮(北京)电子信息产业有限公司 | Method and system for recovering data of database |
| CN108874588A (en) * | 2018-06-08 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of database instance restoration methods and device |
| CN110807064A (en) * | 2019-10-28 | 2020-02-18 | 北京优炫软件股份有限公司 | Data recovery device in RAC distributed database cluster system |
| CN111046024A (en) * | 2019-12-16 | 2020-04-21 | 上海达梦数据库有限公司 | Data processing method, device, equipment and medium for sharing storage database |
-
2020
- 2020-09-21 CN CN202010993792.2A patent/CN112099996B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060117074A1 (en) * | 2004-11-30 | 2006-06-01 | Ezzat Ahmed K | Method and apparatus for database cluster recovery |
| CN102629250A (en) * | 2012-02-28 | 2012-08-08 | 杭州丰城信息技术有限公司 | Recovery method of redo log files for main memory database |
| CN102999564A (en) * | 2012-11-06 | 2013-03-27 | 华为技术有限公司 | Method, device and equipment for writing in data |
| CN105279045A (en) * | 2015-10-28 | 2016-01-27 | 浪潮(北京)电子信息产业有限公司 | Method and system for recovering data of database |
| CN108874588A (en) * | 2018-06-08 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of database instance restoration methods and device |
| CN110807064A (en) * | 2019-10-28 | 2020-02-18 | 北京优炫软件股份有限公司 | Data recovery device in RAC distributed database cluster system |
| CN111046024A (en) * | 2019-12-16 | 2020-04-21 | 上海达梦数据库有限公司 | Data processing method, device, equipment and medium for sharing storage database |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114020357A (en) * | 2021-11-04 | 2022-02-08 | 湖北美和易思教育科技有限公司 | Method, device, system and medium for starting namenode node |
| CN114020357B (en) * | 2021-11-04 | 2024-05-03 | 湖北美和易思教育科技有限公司 | Starting method, device, system and medium of naminode node |
| CN115550145A (en) * | 2022-12-01 | 2022-12-30 | 问策师信息科技南京有限公司 | Industrial Internet safety dynamic protection method and protection system |
| CN117131069A (en) * | 2023-10-26 | 2023-11-28 | 天津神舟通用数据技术有限公司 | Database list grouping scanning method |
| CN117131069B (en) * | 2023-10-26 | 2023-12-22 | 天津神舟通用数据技术有限公司 | Database list grouping scanning method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112099996B (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112099996B (en) | Database cluster multi-node redo log recovery method based on page update sequence number | |
| US9817588B2 (en) | Memory device and operating method of same | |
| US8341497B2 (en) | Semiconductor storage | |
| EP3441890A1 (en) | Transaction recovery method in database system and database management system | |
| CN103577121A (en) | High-reliability linear file access method based on nand flash | |
| CN102929750B (en) | Nonvolatile media dirty region tracking | |
| CN101419564B (en) | Method and device for recovering data by employing snapshot | |
| CN107665219B (en) | Log management method and device | |
| CN103577574A (en) | High-reliability linear file system based on nand flash | |
| CN115291815A (en) | Memory and control method thereof and memory system | |
| JP2007011872A (en) | Memory card and control method therefor | |
| CN112463880B (en) | Block chain data storage method and related device | |
| CN111966531B (en) | Data snapshot method and device, computer equipment and storage medium | |
| CN116578246B (en) | Storage device and control method thereof | |
| KR100725415B1 (en) | Log compaction method and apparatus of database | |
| WO2024077863A1 (en) | Recovery method for all-flash storage system, and related apparatus | |
| CN109240941B (en) | Garbage recovery method of storage system and related device | |
| CN101788934A (en) | Data manipulation method of logical volume manager | |
| CN115587050A (en) | Mapping table garbage recovery efficiency method and device, computer equipment and medium | |
| JPS62245348A (en) | Method and device for updating data base | |
| CN110134548B (en) | Quick restarting method of distributed memory big data processing system | |
| JPS63133240A (en) | Contents assurance system for residence table | |
| CN110351386B (en) | Increment synchronization method and device between different copies | |
| CN113821485A (en) | Data change method, device, equipment and computer readable storage medium | |
| CN115391276A (en) | High-resolution remote sensing image distributed processing system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |