CN107220145B - Method for recovering data of flash memory database - Google Patents
Method for recovering data of flash memory database Download PDFInfo
- Publication number
- CN107220145B CN107220145B CN201710358273.7A CN201710358273A CN107220145B CN 107220145 B CN107220145 B CN 107220145B CN 201710358273 A CN201710358273 A CN 201710358273A CN 107220145 B CN107220145 B CN 107220145B
- Authority
- CN
- China
- Prior art keywords
- log
- data
- transaction
- record
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for recovering data of a flash memory database, which comprises the following steps: scanning the log file, and determining whether a still valid transaction submission log record or a transaction rollback log exists in the log file; and respectively processing according to the current state of the log file: if the effective transaction submitting log record exists, all changes related to the transaction are not completely written into the data file, and the data file is rewritten according to the log record; if the effective rollback log record exists, searching all log records according to the transaction ID, and performing rollback operation; if no effective transaction submitting log record exists, no effective rollback log exists, but a log record which is still effective exists, which indicates that the transaction is not completed yet, whether partial log records are written into the data file or not and partial log records are not written into the data file or not are checked, and if yes, rollback operation is carried out.
Description
Technical Field
The present invention relates to a data storage technology of a flash memory, and in particular, to a method for recovering data in a flash memory database.
Background
Log-based recovery techniques are widely used in conventional disk database management systems. Under different protocols, the design of log records, management of log/data buffers, checkpointing mechanisms, logging and recovery processes are all different. Taking the undo log as an example, when the transaction T needs to change the value of the database element X (with the original value v) to v1, the undo log will form a log record such as < T, X, v1> and store the log record on the disk. When a rollback operation is required for transaction T, then a log record such as < T, X, v > is formed, resetting the value of element X to v.
Flash Memory (Flash Memory) is a long-life non-volatile Memory. Compared with a magnetic disk, the flash memory has the advantages of high read-write speed, shock resistance, electricity saving, small size and the like. The undo log recovery process is reproduced in the flash memory:
(1) the initial state of the data table is shown in table 1(a), the table comprises A, B, C elements, the initial values are va, vb and vc, at this time, the flag bits are all set to 1, and the values of the three elements are all valid;
(2) when the transaction T modifies the value A in the data table from va to va', a new record is inserted, for example, the last line in the table 1(b), and at this time, the flag bit of the value A in the first line is set to 0, which indicates that the value of A is changed and the original value is no longer valid;
(3) if transaction T needs to be rolled back, a new record is written again, restoring the A value to va, as shown in Table 1 (c).
TABLE 1(a)
| Value taking | Marker bit | |
| A | va | 1 |
| B | vb | 1 |
| C | vc | 1 |
| … | … | … |
TABLE 1(b)
| Value taking | Marker bit | |
| A | va | 0 |
| B | vb | 1 |
| C | vc | 1 |
| A | va’ | 1 |
| … | … | … |
TABLE 1(c)
| Value taking | Marker bit | |
| A | va | 0 |
| B | vb | 1 |
| C | vc | 1 |
| A | va’ | 0 |
| A | va | 1 |
| … | … | … |
As can be seen from table 1(c), the last record is identical to the first record, that is to say the last line is redundant in this case. Thus, there is typically a large amount of historical versions of data in flash memory, and conventional data recovery methods continually write existing data records, which wastes both space and time.
Disclosure of Invention
The present invention is directed to a method for recovering data in a flash database, which is used to solve the above-mentioned problems of the prior art.
The invention discloses a method for recovering data of a flash memory database, which comprises the following steps: scanning the log file, and determining whether a still valid transaction submission log record or a transaction rollback log exists in the log file; according to the current state of the log file, the following processes are respectively carried out: if the effective transaction submitting log record exists, all changes related to the transaction are not completely written into the data file, and the data file is rewritten according to the log record; if the effective rollback log record exists, searching all log records according to the transaction ID, and performing rollback operation; if no effective transaction submitting log record exists, no effective rollback log exists, but a log record which is still effective exists, which indicates that the transaction is not completed yet, whether partial log records are written into the data file or not and partial log records are not written into the data file or not are checked, and if yes, rollback operation is carried out.
In an embodiment of the method for data recovery of a flash database according to the present invention, the rollback operation includes: firstly, finding out all log records which are still valid relative to the affair corresponding to the affair ID from a log file according to the affair ID, then sequentially judging log record operation types, and finishing the following operations according to different operation types: if the data element is inserted, deleting the currently inserted data element; if the operation is deletion, reinserting the deleted data elements; if so, restoring the value of the data element to the initial state.
An embodiment of the method for recovering data in a flash database according to the present invention further includes: the chain log comprises a chain table and a data table, wherein the chain table sequentially stores log records of different data operations generated by each transaction, the data table comprises different version data of each data member in the log record, and the log record of the chain table comprises an address of a specified data table; the log record in the data table comprises four fields of a transaction ID, a data member, a front member and a data address table, wherein the transaction ID is the ID number of the transaction to which the current log record belongs; the data member is the name of the data member pointed by the current data operation; the front member is the front member which generates the change in the current transaction; the data address table is the address of all version data of the data member, each version data is connected by a linked list according to the time sequence, and the data address field is set to be null after the transaction is submitted or rolled back.
According to an embodiment of the method for recovering data in a flash memory database, the method for constructing a log file by log records comprises the following steps: when a new transaction is started and a first data change is generated, inserting a log record for a first updated data member; when other data member changes are generated in the transaction, a new log record is inserted, and the former member is set as the data record of the previous change; for insert and delete operations, multiple identifiers are used in the data address table to distinguish different versions of data; if the transaction is committed or rolled back, inserting a commit or roll back log record, and setting the former member as the member updated before; all the changed data members and the specific data operations thereof in the database are recorded and uniformly stored in a log file.
An embodiment of the method for recovering data in a flash database according to the present invention further includes: writing a check point in the log file, wherein the current position is the check point; scanning the logs one by taking the log blocks as units, confirming the validity of all log records before a check point when the check point is met, and writing the still valid log records into a new idle block; after scanning a log block, carrying out erasing operation on the log block, and transferring to the next unscanned log block; and when the free block is full, applying for a new free block and writing a new effective record into the new free block.
An embodiment of the method for recovering data in a flash database according to the present invention further includes: the method for transferring the effective record in the current log block to the new log block comprises the following steps: setting a dump flag bit to be 1, and creating a new log block; acquiring a first log record, judging whether the current unloading zone bit is 1, if not, erasing an old log block and finishing; if the log record is 1, judging whether the current log record is valid, if the current log record is still valid, copying the current log record to a new log block, and acquiring the next log record as the current log record; if the log record is invalid, directly acquiring the next log record as the current log record; if the new log block is full, acquiring a next free block as a new log block; and if the acquired current log record is empty, setting the unloading flag bit to be 0, indicating that the unloading operation is finished.
According to an embodiment of the method for data recovery of a flash database of the present invention, the time interval of the check point is dynamically adjusted according to the application characteristics.
According to an embodiment of the method for recovering data in a flash memory database of the present invention, the data storage method of the chained log comprises: firstly, the log record is written into the memory or the cache, the log record is written into the log file, the data change is written into the data file, and the corresponding log record in the log file is invalidated.
The invention provides a data recovery method based on a flash memory database, which can save storage space and improve data recovery efficiency. The chain type log system performs log operation by using historical version information existing in a flash memory database, and adopts a chain table structure to replace a sequence structure to store log records so as to improve the data recovery efficiency. The failed log records are removed through the periodical checkpoint strategy, the flash memory space is saved, and meanwhile, the length of the log file is effectively reduced.
Drawings
FIG. 1 is a schematic diagram of a chained journal system architecture;
FIG. 2 is a diagram illustrating a log record format;
FIG. 3 is a flowchart illustrating the unloading step;
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
Fig. 1 is a schematic diagram of a chain log system architecture, and as shown in fig. 1, the present invention designs a new log system named chain log. In the chain log system, a log manager 1 is responsible for generating and uniformly managing log records; the log file 2 is a collection of log records and consists of one log record; the recovery manager 3 is responsible for recovery management of data when a system crashes or a transaction rolls back; the DBMS4(Database management system) is a Database management system, and is responsible for transaction management, log management, data file 5 management, and the like; the data file 5 is the physical storage location of the data record.
Fig. 2 is a diagram showing a log record format, and as shown in fig. 2, the chain log stores the log record by using a linked list structure, and the system maintains a linked list and a data table. The linked list stores logs of different data operations generated by each transaction in sequence, the data table comprises different version data of each data member in one log record, and the linked list log comprises addresses of a specified data table.
As shown in fig. 2, the chained log is used for rapidly recovering the flash memory database, and a new log record format is designed based on the characteristics of the flash memory, so that the operation information of the user on the database can be rapidly acquired through the log record, and the flash memory database can be rapidly recovered. The log record of the chained log comprises four fields of a transaction ID, a data member, a front member and a data address table, wherein the transaction ID refers to the ID number of the transaction to which the current log belongs; "data member" refers to the name of the data member to which the current data operation points; "front member" refers to the front member that has changed in the current transaction, and if the data member of the current operation is the first member of the transaction, the front member field is set to "Begin"; the data address table indicates the addresses of all version data of the data members, each version data is connected by a linked list according to the time sequence, and the data address field is set to be NULL after the transaction is submitted or rolled back.
As shown in fig. 1 and 2, the log file 2 is composed of one log record, and table 2 is a log file structure table of the chain log. The log record (log for short) is to record the operation information of the user to the database, the operation of the user to the database and the change of the data can be known through the log record, and when the system is crashed, the database can be recovered according to the log record; the log file is a file for storing log records and is composed of a plurality of log blocks; when the log file is stored, the log file is stored according to blocks, each block is called a log block, namely the log file is divided into a plurality of log blocks, and each log block stores a plurality of log records;
the invention relates to a new log system designed by a chained log for realizing quick recovery of a flash memory database, which is mainly characterized in that a new log record format is designed based on the characteristics of a flash memory, the operation information of a user on the database can be quickly acquired through log records, and the flash memory database can be quickly recovered
TABLE 2
| Serial number | Transaction ID | Data member | Front member | Data address table |
| 1 | T1 | X | Begin | P(X1)->P(X2)->P(X3) |
| 2 | T1 | Y | P(X) | P(Y1)->P(Y2) |
| 3 | T2 | A | Begin | P(A1)->P(A2) |
| 4 | T1 | Z | P(Y) | P(Z1)->P(Z2)-> |
| 5 | T2 | B | P(A) | NULL->P(B1) |
| 6 | T1 | Commit | P(Z) | NULL |
| 7 | T2 | Rollback | P(B) | NULL |
| 8 | … | … | … | … |
As shown in fig. 1 and 2, the log manager 1 manages the log records collectively. The algorithm and the process for constructing the log file 2 by the chain log records are as follows:
first, when a new transaction opens and a first data change occurs (an insert, modify, or delete operation causes a change to the database), the journal manager 1 inserts a journal record for the first updated data member and sets its predecessor member to "Begin". As shown in the first row of Table 2, transaction T1 opens and performs a data operation on data member X, sequentially updating its values (from X1 to X2 to X3), so the first log record is assigned the value: the transaction ID is set to T1, the data member is set to X, the previous member is set to "Begin", and the data address table is set to P (X1) - > P (X2) - > P (X3) (here, the addresses in the data table of X1, X2, X3).
Second, when other data member changes occur in the transaction, the log manager 1 inserts a new log record and sets its former member as the data record of the previous change. If the value of data member Y is changed in transaction T1 from Y1 to Y2, then the log record is assigned the value: the transaction ID is still set to T1, the data member is set to Y, the previous member is set to P (X), and the data address table is set to P (Y1) - > P (Y2), as shown in the second row of Table 2.
Third, for insert and delete operations, multiple identifiers are used in the data address table to distinguish between different versions of data. If an insert operation is performed in a transaction, the first address in its data address table is set to "NULL", and if a delete operation is performed in a transaction, the last address in the data address table is set to "NULL", as shown in Table 2 where the fourth row indicates delete data member Z in T1 and the fifth row indicates insert data member B in T2.
Fourth, if the transaction is committed or rolled back, the log manager 1 inserts a commit or roll back log record with the previous member set to the previously updated member and the address table set to "NULL". As in Table 2, the T1 transaction was committed (Commit) and the T2 transaction was rolled back (Rollback), so the sixth and seventh line log records were written, respectively.
Through the log algorithm, the log manager 1 can construct a complete log record, record all changed data members in the database and specific data operations thereof, and store the data members in the log file 2 in a unified manner to form the log file 2.
As shown in fig. 1, to avoid generating a log file 2 that is too long, resulting in a reduction in system efficiency, the present invention uses a checkpoint strategy to ensure that only log records that are still valid are transferred. The checkpoint setting method is to write checkpoint into the log file 2, indicating that the current position is a checkpoint.
After using the checkpoint policy in the chained log, the operation flow is roughly as follows:
first, the logs are scanned one by one in units of log blocks. When a checkpoint is encountered, the validity of all log records prior to the checkpoint is confirmed and the still valid log records are written to a new free block (free log block, i.e., log block to which no log record was written).
Second, when a log block is scanned, it can be erased and transferred to the next unscanned log block.
Thirdly, when the free block is full, a new free block is applied, and a new valid record (a valid log record having a flag indicating whether the current log record is valid) is written into the new free block.
If a log block can store 100 log records, 10 of which are still valid, 10 log records are stored into a new log block, and the old block can be erased.
Fig. 3 is a flowchart illustrating a procedure of unloading, and as shown in fig. 1 and fig. 3, the procedure of unloading includes:
(1) setting the unloading zone bit to 1 and creating a new log block; the unloading flag bit is a flag for judging whether the current log still has an effective record, if so, it indicates that the effective record exists, and if so, it indicates that the effective record does not exist, and the log block can be erased.
(2) And acquiring a first log record, and judging whether the current 'unloading zone bit' is 1 or not. If not, erasing the old log block and ending; if the log record is 1, judging whether the current log record is valid, if the current log record is still valid, copying the current log record to a new log block, and acquiring the next log record as the current log record; if the log record is invalid, directly acquiring the next log record as the current log record;
(3) if the new log block is full, acquiring a next free block as a new log block;
(4) and if the acquired current log record is empty, setting the unloading zone bit to be 0, which indicates that the unloading operation is finished.
The excessively long checkpoint interval can cause the generation of a large log file 2, and the small checkpoint interval can shorten the life cycle of the flash memory, so that the checkpoint interval of the invention can be dynamically adjusted according to the application characteristics, and meanwhile, a periodic checkpoint strategy is adopted.
In a chained log system, the process of data storage normally involves a database management system (DBMS4) and a log manager 1, where DBMS4 is responsible for normal transactional operations of the database and controls the generation of data files 5, and log manager 1 is responsible for the management of log records and the generation of log files 2. The general flow of normal data storage of the chained log is as follows:
first, the DBMS4 performs normal transaction operations, transacts transactions on the resulting database changes (involving data insertion, deletion, and modification operations) and passes them to the log manager 1, wherein a transaction is required when a series of changes to the source data are saved or cancelled as a whole. The transaction processing comprises opening a transaction, submitting the transaction and rolling back the transaction, updating all operations when the transaction is submitted, and canceling all operations when the transaction is rolled back. Only insert, delete and change operations need to use the transaction because they generate updates to the data in the database. If it is a read operation only, then no transaction need be used.
Secondly, the log manager 1 generates a log record according to the requirement of the chain log format, and forms a log file 2 to complete the complete record of the transaction operation.
Third, the transaction commits normally, at which point the DBMS4 writes the data records to data file 5 (secondary storage medium).
Fourthly, the log manager 1 invalidates the log records related to the committed transaction, and the whole transaction committing process is completed.
As can be seen from the data storage process, the general flow of data storage is as follows: firstly, the log record is written into a memory or a cache- > the log record is written into a log file 2- > the data change is written into a data file 5- > the corresponding log record in the log file 2 is invalidated. Data recovery must be assisted by log recording, i.e. a crash occurs before a log record is written into the log file 2, recovery is not needed because there is no log record currently; if a crash occurs after the log record has been invalidated, recovery is not needed because the current data has been written to the data file 5. In summary, data recovery only occurs after log file 2 is generated and before data file 5 is completed.
Referring to fig. 1, the data recovery process of the method for recovering data in a flash memory database of the present invention includes:
(1) the recovery manager 3 confirms whether there is a still valid transaction commit log (commit record) or transaction rollback log (rollback record) in the log file 2 by scanning the log file 2. The following processes are performed according to the current state of the log file 2.
(2) If there is a valid transaction commit log record (commit record) indicating that all changes related to the transaction have not been completely written to the data file 5, then the overwriting of the data file 5 is completed according to the log record. I.e. already committed transactions, regardless of whether the update is completely fed back into the data file 5, the data records in the data file 5 are guaranteed to be in the state at the time of normal transaction commit as long as all log records relating to the transaction are redone once. As in Table 2, transaction T1 has committed, but does not know whether it has been completely written to data file 5. If crash occurs, the recovery manager 3 first finds all valid records about T1 from the log file 2 according to the transaction ID (the first, second, and fourth rows of table 2), then writes the data changes involved in the log records into the data file 5 in sequence, for example, in table 2, the first row modifies the value of the data member X to P (X1) - > P (X2) - > P (X3), the second row modifies the value of the data member Y to P (Y1) - > P (Y2), the fourth row modifies the value of the data element Z to P (Z1) - > P (Z2), and then deletes the data element Z, the data recovery process only needs to write the final state of the data element in the transaction T1 into the data file 5 (i.e., modify the X value to X3, modify the Y value to Y2, delete Z), and the intermediate process changes can be omitted, thus, the efficiency of log recovery is greatly improved.
(3) If there is a valid rollback log record (rollback log), it is also necessary to search all log records according to the transaction ID and perform a rollback operation. The rollback process is that all the log records still valid about the transaction are found out from the log file 2 according to the transaction ID, then the log record operation types are sequentially judged, and the following operations are completed according to different operation types: if the data is inserted (namely the log records the first data address as NULL), deleting the currently inserted data element; if the operation is deleting (the last data address is NULL), the deleted data element is reinserted; if it is an update operation (except for the two cases described above), the value of the data element is restored to the original state (i.e., the value of the first data address). If transaction T2 in table 2 has a rollback log, then a log record that is still valid is found according to the transaction ID (third and fifth rows in table 2), and then rollback is performed: that is, the value of the data element a is set to the initial value a1, and the data element B is deleted.
(4) If no effective transaction commit log exists, no effective rollback log exists, but a still effective log record exists, which indicates that the transaction is not completed yet, a situation that a part of log records are written into the data file 5 and a part of log records are not written may exist, and at this time, a transaction is inserted for rollback which needs to be displayed. And then the rollback operation is performed according to the third step described above. As shown in Table 2, assuming a crash after the fifth row data record is filled, at this time, the transaction T1 does not write a commit log record, and at this time, all operations of T1 need to be rolled back, i.e., a roll-back record is first inserted into the log file 2, as shown in Table 3.
Table 3:
| T1 | Rollback | P(Z) | NULL |
the value of data element X is then restored to P (X1), the value of data element Y is restored to P (Y1), data element Z is inserted, and its value is set to P (Z1).
The method for restoring the data of the flash memory database designs a new log record format, and stores the changed data members and the data addresses of all different versions of the changed data members into the same log record, so that the effective data record can be quickly found according to the transaction ID when the log is rolled back or submitted, and the efficiency of data restoration is improved.
Meanwhile, the invention follows the WAL (write Ahead logging) rule to decide when to refresh the log, that is, when the local database is changed, there must be a corresponding log record, otherwise, this is not necessarily the case. When a system conflict occurs, the update will only be stored by log file 2 and will not be passed to the database. Therefore, when the recovery operation is executed, the mechanism can ensure the consistency and the integrity of the database.
The method for restoring the data of the flash memory database utilizes the historical version information in the flash memory database of the chain type log system to carry out log operation, and adopts a chain table structure to replace a sequential structure to store log records so as to improve the data restoring efficiency. The failed log records are removed through the periodical checkpoint strategy, the flash memory space is saved, and meanwhile, the length of the log file is effectively reduced.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (5)
1. A method for recovering data in a flash memory database, comprising:
scanning a log file, and determining whether a still-effective transaction submission log record or a transaction rollback log exists in the log file, wherein the log file is a chain log;
according to the current state of the log file, the following processes are respectively carried out:
if the effective transaction submitting log record exists, all changes related to the transaction are not completely written into the data file, and the data file is rewritten according to the log record;
if the effective rollback log record exists, searching all log records according to the transaction ID, and performing rollback operation;
if no effective transaction submitting log record exists, no effective rollback log exists, but a still effective log record exists, which indicates that the transaction is not completed yet, checking whether a part of log records are written into a data file, and a part of log records are not written into the data file, if so, performing rollback operation;
the rollback operation includes: firstly, finding out all log records which are still valid relative to the affair corresponding to the affair ID from a log file according to the affair ID, then sequentially judging log record operation types, and finishing the following operations according to different operation types: if the data element is inserted, deleting the currently inserted data element; if the operation is deletion, reinserting the deleted data elements; if the data element is updated, restoring the value of the data element to the initial state;
the chain log comprises a chain table and a data table, wherein the chain table sequentially stores log records of different data operations generated by each transaction, the data table comprises different version data of each data member in the log record, and the log record of the chain table comprises an address of a specified data table; the log record in the data table comprises four fields of a transaction ID, a data member, a front member and a data address table, wherein the transaction ID is the ID number of the transaction to which the current log record belongs; the data member is the name of the data member pointed by the current data operation; the former member is the address of the data member which changes in the current transaction; the data address table is the address of all version data of the data member, each version data is connected by a linked list according to the time sequence, and the field of the data address table is set to be null after the transaction is submitted or rolled back;
the method for constructing the log file by the chain log records comprises the following steps:
when a new transaction is started and a first data change is generated, inserting a log record for a first updated data member;
when other data member changes are generated in the transaction, a new log record is inserted, and the former member is set as the data record of the previous change;
for insert and delete operations, multiple identifiers are used in the data address table to distinguish different versions of data;
if the transaction is committed or rolled back, inserting a commit or roll back log record, the previous member being set to the address of the previously updated data member;
all the changed data members and the specific data operations thereof in the database are recorded and uniformly stored in a log file.
2. The method for flash database data recovery as set forth in claim 1, further comprising:
writing a check point in the log file, wherein the current position is the check point;
scanning the log records one by taking the log block as a unit, confirming the validity of all the log records before a check point when the check point is met, and writing the still valid log records into a new idle block;
after scanning a log block, carrying out erasing operation on the log block, and transferring to the next unscanned log block;
and when the free block is full, applying for a new free block and writing a new effective record into the new free block.
3. The method for flash database data recovery as set forth in claim 2, further comprising: the method for transferring the effective record in the current log block to the new log block comprises the following steps:
setting a dump flag bit to be 1, and creating a new log block;
acquiring a first log record, judging whether the current unloading zone bit is 1, if not, erasing an old log block and finishing; if the log record is 1, judging whether the current log record is valid, if the current log record is still valid, copying the current log record to a new log block, and acquiring the next log record as the current log record; if the log record is invalid, directly acquiring the next log record as the current log record;
if the new log block is full, acquiring a next free block as a new log block;
and if the acquired current log record is empty, setting the unloading flag bit to be 0, indicating that the unloading operation is finished.
4. The method of claim 3, wherein the time interval of the checkpoints is dynamically adjusted according to application characteristics.
5. The method for data recovery of a flash database of claim 1, wherein the data storage method of the chained log:
firstly, the log record is written into the memory or the cache, the log record is written into the log file, the data change is written into the data file, and the corresponding log record in the log file is invalidated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710358273.7A CN107220145B (en) | 2017-05-19 | 2017-05-19 | Method for recovering data of flash memory database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710358273.7A CN107220145B (en) | 2017-05-19 | 2017-05-19 | Method for recovering data of flash memory database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107220145A CN107220145A (en) | 2017-09-29 |
| CN107220145B true CN107220145B (en) | 2021-06-01 |
Family
ID=59945356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710358273.7A Active CN107220145B (en) | 2017-05-19 | 2017-05-19 | Method for recovering data of flash memory database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107220145B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109960601B (en) * | 2017-12-26 | 2023-03-24 | 航天信息股份有限公司 | Service data recovery method and device and electronic equipment |
| CN109101368B (en) * | 2018-08-20 | 2022-04-22 | 郑州云海信息技术有限公司 | Data processing method and device |
| CN111240891A (en) * | 2018-11-29 | 2020-06-05 | 中国移动通信集团内蒙古有限公司 | Data recovery method and device based on data consistency among multiple tables of database |
| CN109885422B (en) * | 2018-12-29 | 2024-01-12 | 贝壳技术有限公司 | Data rollback method and device for configuration platform in real estate transaction system |
| CN109684292A (en) * | 2019-01-10 | 2019-04-26 | 江苏华存电子科技有限公司 | A kind of method that flash memory database quickly carries out data recovery |
| CN110096389A (en) * | 2019-04-30 | 2019-08-06 | 上海达梦数据库有限公司 | A kind of starting method, apparatus, equipment and the storage medium of database |
| CN111026772B (en) * | 2019-11-26 | 2020-11-27 | 中科驭数（北京）科技有限公司 | Method and device for ensuring permanence of database modification |
| CN111694893B (en) * | 2020-04-23 | 2023-04-25 | 武汉达梦数据库股份有限公司 | Partial rollback analysis method and data synchronization system based on log analysis |
| CN111654453B (en) * | 2020-04-23 | 2023-04-18 | 平安国际智慧城市科技股份有限公司 | Form data offline caching method and device, terminal and storage medium |
| CN112612648B (en) * | 2020-12-29 | 2022-11-08 | 厦门市美亚柏科信息股份有限公司 | SQL Server database recovery method, terminal equipment and storage medium |
| CN114064664A (en) * | 2022-01-17 | 2022-02-18 | 北京奥星贝斯科技有限公司 | Method and device for inquiring transaction modification content in database |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101470629A (en) * | 2007-12-30 | 2009-07-01 | 英特尔公司 | Mechanism for strong atomicity in a transactional memory system |
| US8037112B2 (en) * | 2007-04-23 | 2011-10-11 | Microsoft Corporation | Efficient access of flash databases |
| CN105786955A (en) * | 2015-01-08 | 2016-07-20 | 国际商业机器公司 | Data replication in a database management system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7409587B2 (en) * | 2004-08-24 | 2008-08-05 | Symantec Operating Corporation | Recovering from storage transaction failures using checkpoints |
| CN101286127B (en) * | 2008-05-08 | 2010-06-02 | 华中科技大学 | Multi-fork diary memory continuous data protecting and restoration method |
| US10430298B2 (en) * | 2010-10-28 | 2019-10-01 | Microsoft Technology Licensing, Llc | Versatile in-memory database recovery using logical log records |
| CN102156720A (en) * | 2011-03-28 | 2011-08-17 | 中国人民解放军国防科学技术大学 | Method, device and system for restoring data |
| US8880786B2 (en) * | 2012-09-28 | 2014-11-04 | Apple Inc. | Flash translation layer (FTL) database journaling schemes |
| EP4089540A1 (en) * | 2013-12-12 | 2022-11-16 | Huawei Technologies Co., Ltd. | Data replication method and storage system |
| CN105930500A (en) * | 2016-05-06 | 2016-09-07 | 华为技术有限公司 | Transaction recovery method in database system, and database management system |
-
2017
- 2017-05-19 CN CN201710358273.7A patent/CN107220145B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8037112B2 (en) * | 2007-04-23 | 2011-10-11 | Microsoft Corporation | Efficient access of flash databases |
| CN101470629A (en) * | 2007-12-30 | 2009-07-01 | 英特尔公司 | Mechanism for strong atomicity in a transactional memory system |
| CN105786955A (en) * | 2015-01-08 | 2016-07-20 | 国际商业机器公司 | Data replication in a database management system |
Non-Patent Citations (1)
| Title |
|---|
| 一种高效的闪存数据库故障恢复方法MMR;王曼丽等;《计算机技术与发展》;20120131;第22卷(第1期);第40-44页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107220145A (en) | 2017-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107220145B (en) | Method for recovering data of flash memory database | |
| US10261869B2 (en) | Transaction processing using torn write detection | |
| US9183236B2 (en) | Low level object version tracking using non-volatile memory write generations | |
| KR100862661B1 (en) | Method for deferred logging and apparatus thereof | |
| US9798630B2 (en) | Hardware-supported memory temporal copy | |
| US20190073277A1 (en) | Transaction Recovery Method in Database System, and Database Management System | |
| CN107992269B (en) | Transaction writing method based on deduplication SSD | |
| US20040181560A1 (en) | Method and apparatus for interrupting updates to a database to provide read-only access | |
| US20060224639A1 (en) | Backup system, program and backup method | |
| US11841801B2 (en) | Metadata management in non-volatile memory devices using in-memory journal | |
| WO2020143240A1 (en) | Method for quickly recovering data in flash memory database | |
| US8019953B2 (en) | Method for providing atomicity for host write input/outputs (I/Os) in a continuous data protection (CDP)-enabled volume using intent log | |
| US20170185493A1 (en) | Database transaction replay | |
| KR20110046118A (en) | Adaptive logging apparatus and method | |
| US20230044942A1 (en) | Conditional update, delayed lookup | |
| US20110113017A1 (en) | Supporting Internal Consistency Checking with Consistency Coded Journal File Entries | |
| CN110795282B (en) | Data rollback method, device, equipment and storage medium | |
| CN116595012A (en) | Time sequence database log storage method and system based on nonvolatile memory | |
| Agrawal et al. | Recovery algorithms for database machines with non-volatile main memory | |
| KR101946135B1 (en) | Database management system and method thereof using a non-volatile memory | |
| KR100501414B1 (en) | Method of and apparatus for logging and restoring the meta data in file system | |
| KR20100088325A (en) | Method and apparatus for recovering data using bit logging | |
| Lee et al. | RMSS: an efficient recovery management scheme on NAND flash memory based solid state disk | |
| CN113986974A (en) | Database transaction management method and device and database transaction recovery method and device | |
| CN115437550A (en) | Writing method of storage system and writing method of distributed storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |