CN117271221A - Database data recovery method, storage medium and device - Google Patents
Database data recovery method, storage medium and device Download PDFInfo
- Publication number
- CN117271221A CN117271221A CN202311335503.XA CN202311335503A CN117271221A CN 117271221 A CN117271221 A CN 117271221A CN 202311335503 A CN202311335503 A CN 202311335503A CN 117271221 A CN117271221 A CN 117271221A
- Authority
- CN
- China
- Prior art keywords
- target
- log
- block
- data block
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims description 20
- 238000012795 verification Methods 0.000 claims description 18
- 230000002159 abnormal effect Effects 0.000 claims description 8
- 230000006835 compression Effects 0.000 claims description 8
- 238000007906 compression Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 9
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000008014 freezing Effects 0.000 description 2
- 238000007710 freezing Methods 0.000 description 2
- 238000011065 in-situ storage Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1415—Saving, restoring, recovering or retrying at system level
- G06F11/1435—Saving, restoring, recovering or retrying at system level using file system or storage system metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1405—Saving, restoring, recovering or retrying at machine instruction level
- G06F11/141—Saving, restoring, recovering or retrying at machine instruction level for bus or memory accesses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1727—Details of free space management performed by the file system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database data recovery method, a storage medium and a device. Wherein the method comprises the following steps: determining target information corresponding to a target data block to be restored; acquiring a corresponding target full-page write log in a log file according to the target information, wherein the full-page write log is used for recording basic data information of a data block after a certain check point; and recovering the target data block according to the target full page write log. According to the method, the target full-page write log corresponding to the target data block can be directly found in the log file, so that the basic data of the target data block at a certain check point is recovered according to the record in the target full-page write log, and then updating recovery is continuously carried out according to the subsequent log record. The method directly jumps to the target data block to perform data recovery at the record start of a certain check point, and the data recovery is not required to be performed from zero, so that the speed of data block recovery is improved.
Description
Technical Field
The present invention relates to the field of databases, and in particular, to a method, a storage medium, and an apparatus for recovering database data.
Background
In a normal running primary-backup database system, which generally includes a primary database, a backup database, and a database backup, when a failure occurs, the primary database switches the primary database to the backup database, in this process, data loss occurs in some scenes, and in some scenes of disk damage, normal data files of the database are damaged, which requires the database to provide a data recovery technique.
Common data recovery techniques are classified into two types, data recovery by data backup and data recovery by log files. That is, the whole of the data file and the log file of the existing database is archived, or the sql execution statement is restored from the log file, and then the sql statement is re-executed to restore the data. Most databases provide archiving and recovery functions, and parse into sql based on journals, and then replay the sql for data recovery.
Disclosure of Invention
It is an object of the invention to increase the speed of data block recovery.
It is a further object of the present invention to directly find the corresponding location in the log file for recovery.
In particular, the present invention provides a method for recovering database data, comprising:
determining target information corresponding to a target data block to be restored;
acquiring a corresponding target full-page write log in a log file according to the target information, wherein the full-page write log is used for recording basic data information of a data block at a certain check point;
and recovering the target data block according to the target full page write log.
Optionally, the step of obtaining the corresponding target full page write log in the log file according to the target information includes:
analyzing the log file and reading the log file into a cache to obtain a log reading cache;
judging whether the log reading cache is a full page writing log or not;
if yes, acquiring a target full page write log from the log read cache according to the target information.
Optionally, the target information includes an identification of the target data block;
the step of obtaining the target full page write log in the log read cache according to the target information comprises the following steps:
determining an index number of a target full page write log according to the identification of the target data block;
determining whether a target full page write log exists in the log read cache according to the index number;
if yes, acquiring a target full page write log.
Optionally, the target information includes a size of a target data block;
the step of recovering the target data block according to the target full page write log includes:
calculating the offset of a target log block in a target full-page write log, which is required for recovering the target data block, according to the target information, wherein the target log block is used for recording the basic data information of the target data block in a certain check point;
determining the starting point of the target log block in the target full page write log according to the offset;
copying log blocks with the same size from the starting point into the memory according to the size of the target data block to serve as target log blocks;
performing verification processing on the target log block;
under the condition that the target log block passes the verification processing, carrying out data recovery on the target data block according to the data in the target log block;
and updating the target data block according to the record after the log is written on the whole page in the log file.
Optionally, the step of performing the verification process on the target log block includes:
judging whether the target log block is subjected to compression processing or not;
if yes, judging whether a memory hole exists in the target log block;
and under the condition that the target log block does not have a memory hole, decompressing the target log block, and judging that the target log block passes the verification processing.
Optionally, the step of determining whether the target log block has a memory hole further includes:
zero-filling the memory hole under the condition that the memory hole exists in the target log block;
and judging that the filled target log block passes the verification processing.
Optionally, the step of determining target information corresponding to the target data block to be recovered includes:
detecting the database in real time through a preset detection tool;
judging whether abnormal data appear in the database;
if yes, determining a target data block corresponding to the abnormal data;
and acquiring target information of the target data block.
Optionally, the step of determining target information corresponding to the target data block to be recovered further includes:
and receiving the target data block designated by the client and the corresponding target information thereof.
According to yet another aspect of the present invention, there is also provided a machine-readable storage medium having stored thereon a machine-executable program which, when executed by a processor, implements a method of recovering database data of any of the above.
According to still another aspect of the present invention, there is also provided a computer apparatus including a memory, a processor, and a machine executable program stored on the memory and running on the processor, and the processor implementing a method of recovering database data of any one of the above when executing the machine executable program.
In the scheme of the invention, the target information corresponding to the target data block to be recovered is determined; acquiring a corresponding target full-page write log in a log file according to the target information, wherein the full-page write log is used for recording basic data information of a data block after a certain check point; and recovering the target data block according to the target full page write log. According to the method, the target full-page write log corresponding to the target data block can be directly found in the log file, so that the basic data of the target data block at a certain check point is recovered according to the record in the target full-page write log, and then updating recovery is continuously carried out according to the subsequent log record. The method directly jumps to the target data block to perform data recovery at the record start of a certain check point, and the data recovery is not required to be performed from zero, so that the speed of data block recovery is improved.
The above, as well as additional objectives, advantages, and features of the present invention will become apparent to those skilled in the art from the following detailed description of a specific embodiment of the present invention when read in conjunction with the accompanying drawings.
Drawings
Some specific embodiments of the invention will be described in detail hereinafter by way of example and not by way of limitation with reference to the accompanying drawings. The same reference numbers will be used throughout the drawings to refer to the same or like parts or portions. It will be appreciated by those skilled in the art that the drawings are not necessarily drawn to scale. In the accompanying drawings:
FIG. 1 is a flow diagram of a method of recovering database data according to one embodiment of the invention;
FIG. 2 is a flow chart of a method of recovering database data according to another embodiment of the present invention;
FIG. 3 is an effect diagram of a method of recovering database data according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of a machine-readable storage medium in a method of recovering database data according to one embodiment of the invention; and
FIG. 5 is a schematic diagram of a database data restoration method computer device according to one embodiment of the invention.
Detailed Description
It should be understood by those skilled in the art that the embodiments described below are only some embodiments of the present invention, but not all embodiments of the present invention, and the some embodiments are intended to explain the technical principles of the present invention and are not intended to limit the scope of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive effort, based on the embodiments provided by the present invention, shall still fall within the scope of protection of the present invention.
FIG. 1 is a flow diagram of a method for recovering database data according to one embodiment of the invention. In some alternative embodiments, when the database needs to perform data recovery, an alternative way is to perform offline recovery by means of a log or perform recovery by means of backup data, and the recovery process is generally to perform a table building operation on the data to be recovered from zero in the log, and then re-execute all subsequent sql statements, thereby realizing data recovery. The problem with this approach is that each sql statement is replayed through the corresponding log during data recovery and then fetched for execution at a relatively slow rate.
To solve the above problem, the flow of this embodiment may generally include:
step S101, determining target information corresponding to the target data block to be restored. In some alternative embodiments, the data block is a unit of storage space in the management data file in the database, and is a unit of I/O used in the database, and the size of the unit may be different from the standard I/O block size of the operating system, that is, a storage manner in the data file, and typically, a data file contains at least one data block. The definition of a data block and its corresponding data content can be determined by a person skilled in the art according to the actual situation. The target information corresponding to the data block may generally include an identifier of the data block in the log file, that is, name information of the data block in the log file, and information such as a space size corresponding to the data block.
Step S102, obtaining a corresponding target full page write log in the log file according to the target information. In some alternative embodiments, this step may generally include: and searching and acquiring a corresponding target full page write log in the log file according to the target information corresponding to the target data block in the step S101. The log file is used for recording execution operation information of all data in the data block, such as a table construction statement of a data table and a subsequent operation statement. The Full Page Write log is a recording manner in a pre-Write log, wherein the pre-Write log is a technology for providing atomicity and durability in a database system, and in the system of the pre-Write log, there is a technology of Full Page Write (FPW) to record the content of a file block of a data file in the log completely, that is, the Full Page Write log, which records basic data information of a target data block at a certain check point.
And step S103, recovering the target data block according to the target full page write log. In some alternative embodiments, the content of the target full page write log is directly obtained in the log file according to step S102, so that the target full page write log is directly jumped to begin recovering the target data block.
By the method, in the process of data recovery, the operation is not gradually executed from beginning according to the log file, but the basic data is directly built according to the target full page write log, and then the data recovery is carried out according to the operation statement recorded subsequently. Thereby improving the speed of data recovery, reducing the resources consumed by data recovery and improving the performance of the database.
In some alternative embodiments, the step of obtaining the corresponding target full page write log in the log file according to the target information may generally include: analyzing the log file and reading the log file into a cache to obtain a log reading cache; judging whether the log reading cache is a full page writing log or not; if yes, acquiring a target full page write log from the log read cache according to the target information. The log file is read into the cache after being analyzed, and the reading speed is higher than the reading speed in the disk in the memory, so that the execution speed of the subsequent step can be increased, and the data recovery of a plurality of subsequent data blocks can be facilitated after one-time reading. In this step, since a plurality of types of log files are stored in the log read cache, it is necessary to first determine whether or not there is a full page write log therein, thereby ensuring the progress of the subsequent functions. One optional identification method for the full page write log is as follows: by marking the log type field and the flag resource type field in the log file. Specific examples are: when the log resource type is judged to be a first heap type, namely, the log generated after adding, deleting and modifying any logged (recorded) table, and the log type is added, deleted, modified, hot modified, confirmed, locked, in-situ modified and the like, the required full page write log is judged, so that corresponding recovery operation is respectively carried out later; when the log resource type is the second heap type, namely the pre-written log record generated by modifying the heap by the database, and the corresponding log type is the types of cleaning, page freezing, visibility, lock modification, batch insertion and the like, the required full page write log is judged, so that corresponding recovery operation is respectively carried out later; when the log resource type is the transaction type, namely the pre-written log record generated by the transaction submitting related operation, and the corresponding log type is the type of submitting, giving up and the like, the required full page write log is judged, so that the corresponding recovery operation is respectively carried out later. The log resource type and the log type are operation modes used in actual production, and a person skilled in the art can determine the log resource type and the log type according to actual conditions. Meanwhile, a person skilled in the art can determine the specific analysis step of the log file, the acquisition of the log reading cache and the specific judgment mode of the full page write log according to actual conditions.
The step of obtaining the target full page write log in the log read cache according to the target information may generally include: determining an index number of a target full page write log according to the identification of the target data block; determining whether a target full page write log exists in the log read cache according to the index number; if yes, acquiring a target full page write log. In some alternative embodiments, the target information may generally include an identification of the target data block. One example of the identification of the target data block may be a number or a specific name corresponding thereto, or the like, which can be distinguished from other data blocks. The person skilled in the art can determine the specific expression form of the identification of the target data block according to the actual situation.
The target information includes a size of a target data block. The step of recovering the target data block from the target full page write log may generally include: calculating the offset of a target log block in a target full-page write log, which is required for recovering the target data block, according to the target information, wherein the target log block is used for recording the basic data information of the target data block in a certain check point; determining the starting point of the target log block in the target full page write log according to the offset; copying log blocks with the same size from the starting point into the memory according to the size of the target data block to serve as target log blocks; performing verification processing on the target log block; under the condition that the target log block passes the verification processing, carrying out data recovery on the target data block according to the data in the target log block; and updating the target data block according to the record after the log is written on the whole page in the log file. In this embodiment, since the service of the database is usually at the server, and the corresponding size is set for the data block at the server, in actual production, the user often subdivides the data block at the client due to the actual service requirement, so as to divide a block of data block into a plurality of blocks. Therefore, in this step, after determining the target full-page write log corresponding to the target data block, determining the offset of the required recovery data in the target full-page write log according to the actual size and the actual content of the target data block, then finding the corresponding starting point according to the offset, and copying the content of the data block with the same size into the memory to serve as the target log block, wherein the data recorded in the target log block at this time is the basic data of the target data block at the check point. And after verification, after the basic data of the target data block at the check point is recovered, continuing updating the target data block according to the subsequent operation record in the log file, and finally recovering the target data block.
In some alternative embodiments, the step of performing a verification process on the target log block may generally include: judging whether the target log block is subjected to compression processing or not; if yes, judging whether a memory hole exists in the target log block; and under the condition that the target log block does not have a memory hole, decompressing the target log block, and judging that the target log block passes the verification processing. In actual production, during the process of storing the log file, due to the storage space, a special part in the log file is usually simplified, so that the purpose of compression is achieved, and further, the space is saved. An example of a special case is a memory hole, which is a memory that is applied and released, and cannot be returned to the system because it is not located at the heap top, but is still reserved for the process itself, so that the memory does not store information therein, but still occupies storage space. The restoration mode of the compression space can be determined by a person skilled in the art according to the actual situation.
In other optional embodiments, the step of determining whether the target log block has a memory hole may further generally include: zero-filling the memory hole under the condition that the memory hole exists in the target log block; and judging that the filled target log block passes the verification processing. When the target log block has a memory hole, the database can select to compress the space during storage, but the accuracy of the target log block needs to be ensured during the data recovery process, so that the target log block is restored in the step, and an optional restoring mode is zero-filling of the memory hole. The restoration mode of the memory cavity can be determined by a person skilled in the art according to actual conditions.
In this embodiment, the step of determining the target information corresponding to the target data block to be restored may generally include: detecting the database in real time through a preset detection tool; judging whether abnormal data appear in the database; if yes, determining a target data block corresponding to the abnormal data; and acquiring target information of the target data block.
In other alternative embodiments, the step of determining the target information corresponding to the target data block to be restored may generally further include: and receiving the target data block designated by the client and the corresponding target information thereof. That is, when the user needs to recover the specific target data block due to service requirement or other special conditions, the method can also be used for recovering.
Fig. 2 is a flow chart of a method for recovering database data according to another embodiment of the present invention. In some alternative embodiments, the present process may generally include:
step S201, determining target information corresponding to the target data block to be restored. In some alternative embodiments, this step may generally include: detecting the database in real time through a preset detection tool; judging whether abnormal data appear in the database; if yes, determining a target data block corresponding to the abnormal data; and acquiring target information of the target data block. In other alternative embodiments, the step may generally further comprise: and receiving the target data block designated by the client and the corresponding target information thereof. That is, when the user needs to recover the specific target data block due to service requirement or other special conditions, the method can also be used for recovering. The data block is a unit of storage space in a management data file in the database, and is a unit of I/O used in the database, and the size of the data block may be different from the standard I/O block size of an operating system, that is, a storage mode in the data file, and typically, a data file contains at least one data block. The definition of a data block and its corresponding data content can be determined by a person skilled in the art according to the actual situation. The target information corresponding to the data block may generally include an identifier of the data block in the log file, that is, name information of the data block in the log file, and information such as a space size corresponding to the data block.
Step S202, analyzing the log file and reading the log file into a cache to obtain a log reading cache. Since the read speed is higher than the read speed in the disk in the memory, in this embodiment, the read-to-cache can accelerate the execution speed of the subsequent step, and after one read, the data recovery of the subsequent data blocks can be facilitated. The specific analysis step of the log file and the acquisition mode of the log reading cache can be determined by a person skilled in the art according to actual conditions.
Step S203, judging whether the log read cache is a full page write log. In this embodiment, the Full Page Write log is a recording manner in a pre-Write log, where the pre-Write log is a technology for providing atomicity and durability in a database system, and in the system of the pre-Write log, there is a Full Page Write (FPW) technology, in which the content of a file block of a data file is completely recorded in the log, that is, the Full Page Write log, which records basic data information of a target data block at a certain checkpoint. In this step, since the log read cache stores a plurality of types of log files, it is necessary to determine whether the log is a full page write log, so as to ensure the proceeding of the subsequent functions. One optional identification method for the full page write log is as follows: by marking the log type field and the flag resource type field in the log file. Specific examples are: when the log resource type is judged to be a first heap type, namely, the log generated after adding, deleting and modifying any logged (recorded) table, and the log type is added, deleted, modified, hot modified, confirmed, locked, in-situ modified and the like, the required full page write log is judged, so that corresponding recovery is respectively carried out later; when the log resource type is the second heap type, namely the pre-written log record generated by modifying the heap by the database, and the corresponding log type is the types of cleaning, page freezing, visibility, lock modification, batch insertion and the like, the required full page write log is judged, so that corresponding recovery is respectively carried out later; when the log resource type is the transaction type, namely the pre-written log record generated by the transaction submitting related operation, and the corresponding log type is the type of submitting, giving up and the like, the required full page write log is judged, so that corresponding recovery is carried out respectively in the follow-up. The log resource type and the log type are operation modes used in actual production, and a person skilled in the art can determine the log resource type and the log type according to actual conditions.
Step S204, when the determination in step S203 is yes, the index number of the target full page write log is determined according to the identification of the target data block. In some alternative embodiments, the target information may generally include an identification of the target data block. One example of the identification of the target data block may be a number or a specific name corresponding thereto, or the like, which can be distinguished from other data blocks. The person skilled in the art can determine the specific expression form of the identification of the target data block according to the actual situation.
Step S205, judging whether a target full page write log exists in the log read cache. In some alternative embodiments, this step may generally include: and searching in the log reading cache according to the index number in the step S204, so as to judge whether the corresponding target full page write log exists.
In step S206, if the determination in step S205 is yes, the target full page write log is acquired.
Step S207, calculating the offset of the target log block in the target full page write log, which is required for recovering the target data block, according to the target information. In this embodiment, since the service of the database is usually at the server, and the corresponding size is set for the data block at the server, in actual production, the user often subdivides the data block at the client due to the actual service requirement, so as to divide a block of data block into a plurality of blocks. Therefore, in this step, after the target full-page write log corresponding to the target data block is determined, the offset of the required recovery data in the target full-page write log is determined according to the actual size and the actual content of the target data block, then the corresponding starting point is found according to the offset, and then the content of one data block is copied into the memory. Thereby ensuring that the target data block can be recovered directly.
Step S208, determining the starting point of the target log block in the target full page write log according to the offset.
In step S209, the log blocks with the same size are copied from the starting point to the memory according to the size of the target data block, and the log blocks are used as target log blocks. In this embodiment, the data recorded in the target log block at this time is the base data of the target data block at the checkpoint.
Step S210, checking the target log block. In some alternative embodiments, this step may generally include: judging whether the target log block is subjected to compression processing or not; if yes, judging whether a memory hole exists in the target log block; and under the condition that the target log block does not have a memory hole, decompressing the target log block, and judging that the target log block passes the verification processing. In actual production, during the process of storing the log file, due to the storage space, a special part in the log file is usually simplified, so that the purpose of compression is achieved, and further, the space is saved. An example of a special case is a memory hole, which is a memory that is applied and released, and cannot be returned to the system because it is not located at the heap top, but is still reserved for the process itself, so that the memory does not store information therein, but still occupies storage space. The restoration mode of the compression space can be determined by a person skilled in the art according to the actual situation.
In other alternative embodiments, in the event that a memory hole exists in the target log block, zero padding is performed on the memory hole; and judging that the filled target log block passes the verification processing. When the target log block has a memory hole, the database can select to compress the space during storage, but the accuracy of the target log block needs to be ensured during the data recovery process, so that the target log block is restored in the step, and an optional restoring mode is zero-filling of the memory hole. The restoration mode of the memory cavity can be determined by a person skilled in the art according to actual conditions.
Step S211, in the case that the verification is passed in step S210, the target data block is restored according to the data in the target log block. In this step, the recovery process is performed by copying the base data recorded in the target log block into the target data block, thereby obtaining the base data of the target data block at the check point.
And S212, updating and recovering the target data block according to the record after the log is written on the whole page in the log file. In some alternative embodiments, since the log file continues to record subsequent operation information after a certain checkpoint, after determining the base data of the target data block at the checkpoint, the update recovery operation is continued on the target data block according to the record in the log file after writing the log on the full page, and finally the information in the target data block is completely recovered.
According to the method, the target full-page write log corresponding to the target data block can be directly found in the log file, so that the basic data of the target data block at a certain check point is recovered according to the record in the target full-page write log, and then updating recovery is continuously carried out according to the subsequent log record. The method directly jumps to the target data block to perform data recovery at the record start of a certain check point, and the data recovery is not required to be performed from zero, so that the speed of data block recovery is improved.
Fig. 3 is an effect diagram of a method of recovering database data according to an embodiment of the present invention. In some alternative embodiments, the present schematic may generally include:
a time line 30 for representing the time spent in the data recovery process. For the conventional method of data recovery, each operation sentence in the log file is usually executed step by step from the init (start) point of the log file, so as to achieve the purpose of data recovery, and therefore, the consumed time is shown in a time line 301. After the method is used, a corresponding starting point can be found according to the target data block to start recovery, because the recording time of the check points is different, that is, the recording time of the full page write log is different, the starting point may be Fpw point 1, fpw point 2 and Fpw point 3, the time consumed by each Fpw point is different, for example, fpw point 1 corresponds to time line 302, fpw point 2 corresponds to time line 303 and Fpw point 3 corresponds to time line 304, and as can be seen from the figure, the time line 302, time line 303 and time line 304 are smaller than time line 301, that is, the consumed time is lower than recovery from init (starting point).
Further, in the actual use process, the time consumed for the whole table recovery is shown in table 1:
TABLE 1
The time spent in the actual use of the method is shown in table 2:
TABLE 2
As can be seen from the above examples, in the data recovery process, the method is able to jump directly to the starting point of the target full page write log to perform data recovery instead of gradually performing operations from the beginning according to the log file, thereby improving the speed of data recovery, reducing the resources consumed by data recovery, and improving the performance of the database.
The present implementation also provides a machine-readable storage medium and a computer device. Fig. 4 is a schematic diagram of a machine-readable storage medium 401 according to one embodiment of the invention, and fig. 5 is a schematic diagram of a computer device 503 according to one embodiment of the invention.
The machine-readable storage medium 401 has stored thereon a machine-executable program 402, which when executed by a processor, implements the database data restoration method of any of the embodiments described above.
The computer device 503 may include a memory 501, a processor 502, and a machine executable program 402 stored on the memory 501 and running on the processor 502, and the processor 502 implements the database data restoration method of any of the embodiments described above when executing the machine executable program 402.
It should be noted that the logic and/or steps represented in the flow diagrams or otherwise described herein, e.g., data recovery, may be embodied in any machine-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
For the purposes of this description of embodiments, a machine-readable storage medium 401 may be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system.
The computer device 503 may be a type of in-vehicle device such as an in-vehicle computer. The computer device 503 may be described in the general context of computer-system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer device 503 may be implemented in a distributed cloud computing environment where remote processing devices coupled via a communications network perform tasks. In a distributed cloud computing environment, program modules may be located in both local and remote computing system storage media including memory storage devices such as an automobile central processing unit.
The computer device 503 may comprise a processor 502 adapted to execute stored instructions, a memory 501 providing temporary storage for the operation of the instructions during operation. The processor 502 may be a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. Memory 501 may include Random Access Memory (RAM), read only memory, flash memory, or any other suitable storage system.
The processor 502 may be connected through a system interconnect (e.g., PCI-Express, etc.) to an I/O interface (input/output interface) adapted to connect the computer device 503 to one or more I/O devices (input/output devices). The I/O devices may include, for example, a keyboard and a pointing device, which may include a touch pad or touch screen, among others. The I/O device may be a built-in component of the computer device 503 or may be a device externally connected to the computing device.
The processor 502 may also be linked through a system interconnect to a display interface adapted to connect the computer device 503 to a display device. The display device may include a display screen as a built-in component of the computer device 503. The display device may also include a computer monitor, television, projector, or the like, which is externally connected to the computer device 503. Further, a network interface controller (network interface controller, NIC) may be adapted to connect the computer device 503 to a network through a system interconnect. In some embodiments, the NIC may use any suitable interface or protocol (such as an internet small computer system interface, etc.) to transfer data. The network may be a cellular network, a radio network, a Wide Area Network (WAN), a Local Area Network (LAN), or the internet, among others. The remote device may be connected to the computing device through a network.
The flowcharts provided by this embodiment are not intended to indicate that the operations of the method are to be performed in any particular order, or that all of the operations of the method are included in all of each case. Furthermore, the method may include additional operations. Additional variations may be made to the above-described methods within the scope of the technical ideas provided by the methods of the present embodiments.
By now it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been shown and described herein in detail, many other variations or modifications of the invention consistent with the principles of the invention may be directly ascertained or inferred from the present disclosure without departing from the spirit and scope of the invention. Accordingly, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.
Claims (10)
1. A method of recovering database data, comprising:
determining target information corresponding to a target data block to be restored;
acquiring a corresponding target full-page write log in a log file according to the target information, wherein the full-page write log is used for recording basic data information of a data block at a certain check point;
and recovering the target data block according to the target full page write log.
2. The method for recovering database data according to claim 1, wherein,
the step of obtaining the corresponding target full page write log in the log file according to the target information comprises the following steps:
analyzing the log file and reading the log file into a cache to obtain a log reading cache;
judging whether the log reading cache is a full page writing log or not;
if yes, acquiring the target full page write log from the log read cache according to the target information.
3. The method for recovering database data according to claim 2, wherein,
the target information comprises an identification of the target data block;
the step of obtaining the target full page write log in the log read cache according to the target information comprises the following steps:
determining an index number of the target full page write log according to the identification of the target data block;
determining whether the target full page write log exists in the log read cache according to the index number;
if yes, acquiring the target full page write log.
4. The method for recovering database data according to claim 1, wherein,
the target information includes a size of the target data block;
the step of recovering the target data block according to the target full page write log includes:
calculating the offset of a target log block in the target full page write log, which is required for recovering the target data block, according to the target information, wherein the target log block is used for recording the basic data information of the target data block at a certain check point;
determining the starting point of the target log block in the target full page write log according to the offset;
copying log blocks with the same size from the starting point to a memory according to the size of the target data block to serve as the target log block;
performing verification processing on the target log block;
under the condition that the target log block passes the verification processing, carrying out data recovery on the target data block according to the data in the target log block;
and updating the target data block according to the record after the full page write log in the log file.
5. The method for recovering database data according to claim 4, wherein,
the step of verifying the target log block comprises the following steps:
judging whether the target log block is subjected to compression processing or not;
if yes, judging whether the target log block has a memory hole or not;
and under the condition that the target log block does not have a memory hole, decompressing the target log block, and judging that the target log block passes the verification processing.
6. The method for recovering database data according to claim 5, wherein,
the step of judging whether the target log block has a memory hole further comprises the following steps:
zero-filling the memory hole under the condition that the memory hole exists in the target log block;
and judging that the filled target log block passes the verification processing.
7. The method for recovering database data according to claim 1, wherein,
the step of determining the target information corresponding to the target data block to be recovered comprises the following steps:
detecting the database in real time through a preset detection tool;
judging whether abnormal data appear in the database;
if yes, determining a target data block corresponding to the abnormal data;
and acquiring target information of the target data block.
8. The method for recovering database data according to claim 1, wherein,
the step of determining the target information corresponding to the target data block to be recovered further comprises:
and receiving the target data block designated by the client and the corresponding target information thereof.
9. A machine-readable storage medium having stored thereon a machine-executable program which when executed by a processor implements a method of recovering database data according to any one of claims 1 to 8.
10. A computer device comprising a memory, a processor and a machine executable program stored on the memory and running on the processor, and the processor implementing a method of recovering database data according to any one of claims 1 to 8 when executing the machine executable program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311335503.XA CN117271221A (en) | 2023-10-16 | 2023-10-16 | Database data recovery method, storage medium and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311335503.XA CN117271221A (en) | 2023-10-16 | 2023-10-16 | Database data recovery method, storage medium and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117271221A true CN117271221A (en) | 2023-12-22 |
Family
ID=89214259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311335503.XA Pending CN117271221A (en) | 2023-10-16 | 2023-10-16 | Database data recovery method, storage medium and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117271221A (en) |
-
2023
- 2023-10-16 CN CN202311335503.XA patent/CN117271221A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108664359B (en) | Database recovery method, device, equipment and storage medium | |
| CN106951345B (en) | Consistency test method and device for disk data of virtual machine | |
| US20180203606A1 (en) | Method and device for writing data and acquiring data in a distributed storage system | |
| US9262283B2 (en) | Method for reading kernel log upon kernel panic in operating system | |
| CN109542682B (en) | Data backup method, device, equipment and storage medium | |
| JP2006065845A (en) | Generation of optimized restoration plan | |
| CN105573859A (en) | Data recovery method and device of database | |
| CN109471851B (en) | Data processing method, device, server and storage medium | |
| US10402279B2 (en) | Optimization to permit block based incremental backup across system reboot or crash | |
| CN110795287A (en) | Data recovery method, system, electronic device and computer storage medium | |
| CN111258666A (en) | Reading method and device of computer file, computer system and storage medium | |
| KR20170054767A (en) | Database management system and method for modifying and recovering data the same | |
| CN113076220A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| US20140279943A1 (en) | File system verification method and information processing apparatus | |
| US8370815B2 (en) | Electronic device and method for debugging programs | |
| CN114138192A (en) | Storage node online upgrading method, device, system and storage medium | |
| US9015116B2 (en) | Consistent replication of transactional updates | |
| WO2015015502A1 (en) | Writing to files and file meta-data | |
| CN117112522A (en) | Concurrent process log management method, device, equipment and storage medium | |
| CN112000623A (en) | Metadata access method and device and computer readable storage medium | |
| CN111625407A (en) | SSD performance test method and related components | |
| CN117271221A (en) | Database data recovery method, storage medium and device | |
| CN111984472B (en) | Data snapshot method, device and related equipment | |
| CN113791934A (en) | Data recovery method, computing device and storage medium | |
| CN114281798A (en) | Data migration method and device for heterogeneous database and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |