CN115576494A - Data storage method and computing device - Google Patents
Data storage method and computing device Download PDFInfo
- Publication number
- CN115576494A CN115576494A CN202211343759.0A CN202211343759A CN115576494A CN 115576494 A CN115576494 A CN 115576494A CN 202211343759 A CN202211343759 A CN 202211343759A CN 115576494 A CN115576494 A CN 115576494A
- Authority
- CN
- China
- Prior art keywords
- data
- log
- transaction
- modification
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000013500 data storage Methods 0.000 title claims abstract description 32
- 238000012986 modification Methods 0.000 claims abstract description 158
- 230000004048 modification Effects 0.000 claims abstract description 158
- 238000011010 flushing procedure Methods 0.000 claims description 26
- 238000012217 deletion Methods 0.000 claims description 10
- 230000037430 deletion Effects 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 13
- 238000007726 management method Methods 0.000 description 12
- 238000011084 recovery Methods 0.000 description 11
- 230000000694 effects Effects 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000001680 brushing effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000002955 isolation Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data storage method and computing equipment, when a transaction is submitted, a data page is flushed, then a post-write log is flushed, the post-write log comprises a data modification log, and the data modification log comprises a data page modification position and a modification operation type. Because the position and the modification type of the modified data are recorded in the data modification log, the data before modification and the data after modification do not need to be recorded, and the data volume corresponding to the position and the modification type of the modified data is small, thereby reducing the log volume and the IO volume during data modification. And because the data page disk refreshing is finished before the post-write log disk refreshing, the problem of half-page writing can not occur when the post-write log disk refreshing is finished, so that a full-page writing mechanism is not required to be written, and the data volume of the log is further reduced.
Description
Technical Field
The present invention relates to the field of data technologies, and in particular, to a data storage method and a computing device.
Background
In the process of executing the transaction processing by the database, a transaction log needs to be recorded, so that the durability of the transaction processing is ensured. The current method for recording transaction logs is mainly WAL (Write-ahead logging).
Under the condition of a traditional mechanical disk storage medium, the capability of random IO is far behind sequential IO, so that in order to solve the problem of random IO submitted by each transaction on the premise of ensuring the consistency of databases such as Oracle, mySQL, postgreSQL and the like, the WAL technology is adopted to write data modification contents contained in random dirty pages into sequential logs. When the transaction is submitted, the disk is flushed, and the dirty page is not flushed.
However, since data modification related to the transaction needs to be written into the log before the transaction is committed, the IO volume of the log is increased dramatically, and the situation that the database stops working due to the full disk caused by log expansion is easy to occur.
Therefore, how to reduce the log amount is an important issue at present.
Disclosure of Invention
The embodiment of the application provides a data storage method and computing equipment, which are used for reducing the amount of logs and achieving the purpose of improving the transaction processing performance of a database.
In a first aspect, the present application provides a data storage method, including:
when the transaction begins to submit, the data page is flushed;
and flushing a disk of the post-write log, wherein the post-write log comprises a data modification log, and the data modification log records the position and modification type of modification data corresponding to modification operation when the data modification log carries out modification operation on a data page in a buffer area of a memory.
According to the data storage method provided by the embodiment of the application, when the transaction is started to be submitted, the data page is flushed, and the post-write log is flushed after the data page is flushed. The post-write log comprises a data modification log, and when the data modification log records modification operation on a data page in a buffer area of the memory, the position and the modification type of the modified data corresponding to the modification operation do not need to record the data before modification and the data after modification, so that the log quantity of the data modification log is reduced, and the log quantity of the post-write log is further reduced.
Optionally, the transaction includes a plurality of modification operations performed on a data page in a buffer of the memory;
when the transaction is started to be submitted, the flushing of the data page specifically includes:
and when the first modification operation in the plurality of modification operations in the transaction starts to be written into a buffer area of a disk, writing a data page in the buffer area of the disk into the disk.
In the data storage method provided in this embodiment, when a first modification operation in a plurality of modification operations in a transaction starts to be written into a buffer area of a disk, a data page in the buffer area of the disk is written into the disk, so that when the transaction starts to commit, a data page is flushed.
Optionally, the flushing the write-after log specifically includes:
when a data page in a buffer area of the memory is modified, writing the position and the modification type of modified data corresponding to the modification operation into a log buffer area of the memory to generate a data modification log;
and flushing the data modification log.
In the data storage method provided in this embodiment, when a data page in a buffer area of a memory is modified, a location and a modification type of modified data corresponding to the modification operation are written into a log buffer area of the memory to generate a data modification log, so as to determine modified data according to the data modification log in a disk.
Optionally, the modification operation includes at least one of an update operation, a delete operation, and an add operation, and the data modification log includes at least one of a transaction update log, a transaction delete log, and a transaction add log;
when the data page in the buffer area of the memory is modified, writing the position of the modified data corresponding to the modification into the log buffer area of the memory to generate a data modification log, which specifically includes:
when updating operation is carried out on a data page in a buffer area of a memory, writing the position of updating data corresponding to the updating operation into a log buffer area of the memory to generate a transaction updating log;
when a data page in a buffer area of the memory is deleted, writing the position of deleted data corresponding to the deletion operation into a log buffer area of the memory to generate a transaction deletion log;
when the data page in the buffer area of the memory is subjected to the adding operation, the position of the added data corresponding to the adding operation is written into the log buffer area of the memory, and a transaction adding log is generated.
In the data storage method provided by this embodiment, the modification operation includes at least one of an update operation, a delete operation, and an add operation, when the update operation is performed, update data corresponding to the update operation is written into a log buffer area of the memory to generate a transaction update log, when the delete operation is performed, delete data corresponding to the delete operation is written into the log buffer area of the memory to generate a transaction delete log, and when the add operation is performed, add data corresponding to the add operation is written into the log buffer area of the memory to generate a transaction add log, so that a log corresponding to each operation is obtained.
Optionally, before the transaction starts to commit, the method further includes:
when a data page in a buffer area of an internal memory is modified, calculating the position of modified data corresponding to the modification operation;
and locking the position of the modified data corresponding to the modification operation.
In the data storage method provided by this embodiment, after the position of the modified data corresponding to the modification operation is obtained through calculation, the position corresponding to the modified data is locked, so that the amount of the locked data is reduced, and further, the performance consumption caused by locking is reduced.
Optionally, when performing a modification operation on a data page in a buffer of the memory, calculating a position of modified data corresponding to the modification operation, specifically including:
when the data page in the buffer area of the memory is modified, the position of the modified data is calculated according to the end position of the last data of the modified data corresponding to the modification operation and the disk space occupied by the modified data.
In the data storage method provided in this embodiment, when a data page in a buffer of a memory is modified, a location of modified data is calculated according to an end location of last data of the modified data corresponding to the modification and a disk space occupied by the modified data, so that the location of the modified data is accurately obtained.
Optionally, when the data page in the buffer of the memory is modified, the calculating the position of the modified data according to the end position of the last data of the modified data corresponding to the modification operation and the disk space occupied by the modified data specifically includes:
when updating operation is carried out on a data page in a buffer area of the memory, calculating the position of the updated data according to the ending position of the last data of the updated data corresponding to the updating operation and the disk space occupied by the updated data;
when a data page in a buffer area of the memory is deleted, calculating the position of deleted data according to the ending position of the last data of the deleted data corresponding to the deletion and the disk space occupied by the deleted data;
when the data page in the buffer area of the memory is subjected to the adding operation, the position of the added data is calculated according to the end position of the last data of the added data corresponding to the adding operation and the disk space occupied by the added data.
In the data storage method provided in this embodiment, when performing an update operation on a data page in a buffer of an internal memory, a position of update data is calculated according to an end position of previous data of the update data corresponding to the update operation and a disk space occupied by the update data, when performing a delete operation on a data page in the buffer of the internal memory, a position of the update data is calculated according to an end position of previous data of delete data corresponding to the delete operation and a disk space occupied by the delete data, and when performing an add operation on a data page in the buffer of the internal memory, a position of add data is calculated according to an end position of previous data corresponding to the add operation and a disk space occupied by the add data, so as to obtain positions of data corresponding to the update operation, the delete operation, and the add operation.
Optionally, the post-write log includes a transaction commit log or a transaction rollback log, where the transaction commit log records a data page committed to a buffer of a disk in a buffer of a memory, and the transaction rollback log records a data page on which a modification operation has been performed before a failure operation;
the flushing the disk of the post-write log specifically comprises:
after the data page in the buffer area of the memory is flushed, writing the data page submitted to the buffer area in the disk in the buffer area of the memory into a log buffer area of the memory, generating a transaction submission log, and flushing the transaction submission log;
or writing a data page which is subjected to modification operation before the failure operation in the buffer area of the memory into a log buffer area of the memory, generating a transaction rollback log, and flushing the transaction rollback log.
The data storage method provided in this embodiment may further write, after flushing the disk of the data page in the buffer area of the memory, the data page submitted to the buffer area in the disk in the buffer area of the memory into the log buffer area of the memory, generate a transaction submission log, write, in the log buffer area of the memory, the data page on which the modification operation has been performed before the failed operation in the buffer area of the memory, and generate a transaction rollback log, where the post-write log may further include the transaction submission log or the transaction rollback log, so that the transaction submission record and the transaction rollback record may be obtained according to the post-write log.
Optionally, after the disk of the write-after log is flushed, the method further includes:
acquiring a target log from a magnetic disk, playing back the target log, and judging whether the target log is the last log in a write-back log;
if not, continuously acquiring the target log from the disk until the target log is the last log in the post-write log;
if yes, the control computer receives a request for modifying the data in the disk.
The data storage method provided by this embodiment may further acquire the target log from the disk, and play back the target log, so as to obtain the modification record of the data in the disk according to the target log.
In a second aspect, the present application provides a computing device comprising a processor configured to perform the data storage method described above.
According to the data storage method, when the transaction is submitted, the data page is flushed, then the post-write log is flushed, the post-write log comprises the data modification log, and the data modification log comprises the data page modification position and the modification operation type. Because the position and the modification type of the modified data are recorded in the data modification log, the data before modification and the data after modification do not need to be recorded, and the data volume corresponding to the position and the modification type of the modified data is small, thereby reducing the log volume and the IO volume during data modification. And because the data page disk refreshing is finished before the post-write log disk refreshing, the problem of half-page writing can not occur when the post-write log disk refreshing is finished, so that a full-page writing mechanism is not required to be written, and the data volume of the log is further reduced.
Drawings
In order to clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIGS. 1-3 are flow diagrams of various embodiments of a data storage method;
FIG. 4 is an architecture diagram of a data storage system.
Detailed Description
The terminology used in the description of the embodiments section of the present application is for the purpose of describing particular embodiments of the present application only and is not intended to be limiting of the present application.
For ease of understanding, related art terms referred to in the embodiments of the present application are explained and illustrated first.
Data tuple (tuple): from a user perspective, in the database, each table includes multiple rows of data; from the database kernel implementation, each row of data, plus metadata that needs to be added to manage the row of data (e.g., whether the row of data is expired), constitutes a data tuple.
Transaction (Transaction): generally to what is to be done or what is to be done. In computer terminology, refers to one or more program execution units (units) that access and possibly update data items in a database. Transactions are typically caused by the execution of user programs written in a database manipulation language or programming language (e.g., SQL, C + +, or Java), and are delimited in the form of begin transaction and end transaction statements (or function calls), among others. A transaction consists of an ensemble of operations performed between a transaction start (begin transaction) and a transaction end (end transaction).
ACID (Atomicity, consistency, isolation, durability): the characteristics that the transaction has; atomicity means that the transaction is either successful or failed, and after the transaction is failed, the system rolls back to a state before the transaction is started, so that the situation that one part of the transaction is successful and influences that the transaction is omitted in the system and the other part of the transaction is failed does not exist; the consistency means that after the transaction is successfully executed, the system enters the next consistency state from the consistency before execution; the isolation is that the transaction is not influenced by other transactions when being executed, and the influence generated in the transaction execution process is not perceived by other transactions; the persistence means that after the transaction is successfully executed, the effect of the transaction is continuously present in the system, and the effect of the transaction is not lost or damaged due to system failure (such as server restart, database process crash, and the like).
Redo log (Redo log): after the transaction is successfully executed, the transaction modifies the data, and the modification effect of the transaction may disappear (the modification is not yet done on the dirty page in the memory) due to a failure (such as server restart, database process crash, etc.). To avoid this problem, a redo log is introduced, recording the transaction modify operation type (modify, delete, add, etc.), the data tuple ID involved in the modification, the new data tuple itself (the delete type does not have this data), etc. According to the redo log, the transaction modification operation can be redone, and the modification effect can be reproduced.
Undo log (rollback log): in the event that a transaction does not succeed, the effects of the transaction's modification, if ever, may remain in the system as a result of the operating system flushing pages. In order to eliminate the retention effect, the undo log is introduced, and the modification made by the failed transaction can be cleared by reversely executing the undo log.
Clog log (submit log): a transaction commit history log, which records the status log of all transaction commits (or rollbacks) that have been performed in the system.
WAL: (Write-ahead loading, pre-Write log): the WAL refers to that when a transaction is submitted, a transaction log is flushed first, and then a modified data page is flushed, namely the transaction log is flushed before.
WBL: (Write-before loading, write-after log): WBL refers to flushing the modified data page and then the transaction log when the transaction is committed, i.e., flushing the transaction log later.
Out-place update (not updated in place): when writing a data page, a new data page is written to another location instead of the original page.
Brushing dirty pages: the data modification process of the operating system comprises the following steps: (1) Reading data into a memory from a disk file according to a page (the size is generally 4K); (2) Modifying a data page in the memory, wherein the data page is a dirty page; and (3) writing the dirty page back to the disk file. And (3) brushing dirty pages.
4K Page atom write: when data is read into the memory from the disk file and the disk file is written out from the memory, 4K is taken as a reading-writing unit, namely a page. The operating system ensures that the data in the page is complete and consistent data at a certain moment when the whole 4K page is read and written every time, the situation that the previous part of data in the 4K page is more updated or older than other data does not exist, and the characteristic of the data writing operation is called 4K page atomic writing.
8K page half write: the size of a data page of the database is 8K, the data page consists of two operating system pages, the operating system can ensure 4K page atomic write, but does not ensure that one 8K page is written out in a 2 4K page atomic write mode, and before the next page is written out, the operating system fails, so that half of data in the 8K pages of the database, namely one 4K operating system page, is stored with new data, and the other half of the data is stored with old data.
Full Page Write: when the database system modifies the 8K data page for the first time (before, the data page is just read from a disk file to a memory, or a dirty page is executed on the data page), the whole data page is written into a database transaction log before modification, the log can store a clean data page, and if half-page writing problem occurs subsequently, the clean and consistent data page can be recovered from the log. Full page write increases the amount of database logs IO.
Checkpoint (Checkpoint or manufacturing Checkpoint): the database (calling an operating system API) performs a dirty page flushing operation on all modified data pages, after the dirty page flushing operation is completed, the operation is recorded in a database transaction log, and information indicating the current processing progress of the database is written in a default position, and the whole process is called a manufacturing check point (at this time, the checkpoint is a verb). The manufacturing checkpoint operation information and the current processing progress information of writing the appointment position into the database are recorded in the log and called as the checkpoint (in this case, checkpoint is a noun). If the system crashes, the state of the database can be restored to the processing progress of the database corresponding to the check point by the disk data file and the check point data.
RTO (Recovery Time Objective, system Recovery Time): the database system is restarted due to crash of some reasons or restarted according to a plan due to maintenance, and the database system is restored to a state before restarting through replaying the redo log recording operation after restarting. And restarting the database to a state of finishing database recovery, namely the recovery time of the system. During this time, the database system cannot provide service to the outside, and generally the shorter the system recovery time, the better.
Random IO: the read and write contents are scattered at different positions of the disk and need to be searched back and forth.
And (4) sequential IO: the read and write contents are stored in a block of the disk in a centralized manner and are read sequentially from front to back.
Dirty pages: when the memory data page is inconsistent with the disk data page, the memory page is called a dirty page.
Brushing dirty pages: the operation of writing a page of memory data to disk is called flushing a page.
Transaction logging: it is a file separate from the database file that stores all changes made to the database and records all insertions, updates, deletions, submissions, backoffs, and database schema changes.
In the process of executing the transaction processing by the database, a transaction log needs to be recorded, so that the durability of the transaction processing is ensured. There are two types of methods for logging transactions: WAL and WBL.
Under the condition of a traditional mechanical disk storage medium, the capacity of random IO is far behind sequential IO, so that databases such as Oracle, mySQL, postgreSQL and the like adopt the WAL technology to write random dirty pages into sequential logs on the premise of ensuring the consistency of the databases so as to solve the problem of random IO submitted by each transaction, and the logs are used for flushing a disk, so that the dirty pages are not needed to be flushed when the transactions are submitted.
However, since data related to a transaction needs to be written into a log before the transaction is committed, the IO volume of the log increases dramatically, and the database is likely to stop working due to the full disk caused by the expansion of the log.
And because the operating system can only guarantee 4K Page atomic Write, in order to prevent the 8K Page half-Page Write problem, the Full Page Write mechanism is written, and although the 8K Page half-Page Write problem is solved by the mechanism, the log data volume is larger.
In addition, when the logs are written, the logs need to be written in the sequence of generating the logs by the database, but a plurality of transactions are executed in the database at the same time, and in order to force the logs to be arranged in the sequence of generating the logs by the database, a mutual exclusion lock needs to be added to the logs so that the log writing is serial, so that the concurrency is not advantageous.
For the WBL, when the data page is modified, the modified data page is not covered on the data page before modification, but the modified data page is written to other positions, and the data of the modified data page is inconsistent in an abnormal scene. When the database is restarted and recovered, in order to ensure that the data page modification is consistent with the data modification recorded in the log, the part of the data page modification which is more than the data modification recorded in the log needs to be rolled back, and thus, the transaction batch submission needs to be performed with more roll-back operations in an abnormal scene, and the system RTO can be reduced.
Based on this, the application provides a data storage method, when a transaction is started to commit, a data page is flushed, and then a WBL log is flushed, wherein the WBL log includes a data modification log, and the data modification log includes a data page modification position and a modification operation type. The existing WAL log comprises data before modification and data after modification, and the second log records the content of the transaction before modifying the database and the content of the transaction after modifying the database, namely records the old value and the new value. Therefore, when recovery is encountered, if the previous writing is not finished, the recovery can be realized by finding a new value and finishing the writing; if half of the data is written before and fails, the old value can be recovered after being found, the data modification log comprises the position and the modification type of the modified data, the data volume of the old value and the new value in the WAL log is large, and the data volume corresponding to the position and the modification type of the modified data in the data modification log is small, so that the log volume is reduced, and the IO volume during data modification is reduced. And because the data page disk refreshing is finished before the WBL log disk refreshing, the half-page writing problem can not occur when the WBL log disk refreshing is finished, so that a full-page writing mechanism is not required to be written, and the log data volume is further reduced.
The data storage method provided by the embodiment of the present application is described in detail below through a specific implementation manner.
Fig. 1 is a flowchart of a data storage method according to an embodiment of the present disclosure. Referring to fig. 1, a data storage method provided in an embodiment of the present application includes:
s101, when the transaction is started to be submitted, the data page is flushed.
A transaction includes all operations performed between the beginning of the transaction and the end of the transaction, and the transaction may include one or more operations, which may include adding, updating, deleting, etc. data in memory. When a transaction includes one operation, the transaction start may be the time when the operation starts to be executed, the time when the transaction starts may be the time when the operation starts to be executed, when the transaction includes a plurality of operations, the transaction start may be the time when the first operation starts to be executed, and the time when the transaction starts may be the time when the first operation starts to be executed.
For example, a transaction may include one or more DML statements (Data management statement), and the overall operations performed between the start of the transaction and the end of the transaction may be the execution of one or more DML statements in the transaction. When the transaction includes a DML statement, the time when the processor starts executing the DML statement may be the time when the transaction starts; when the transaction includes a plurality of DML statements, the time at which the processor starts executing the first DML statement is taken as the time at which the transaction starts. The transaction may further include a DDL statement (Data Definition statement), and the overall operation performed from the beginning of the transaction to the end of the transaction may be to execute the DDL statement, and the time when the processor starts executing the DDL statement is the time when the transaction starts. The transaction may further include a DCL statement (Data Control statement), and the overall operation from the start of the transaction to the execution of the transaction may be the execution of the DCL statement, and the time when the processor starts to execute the DCL statement is the time when the transaction starts.
The transaction end comprises a normal transaction end and an abnormal transaction end, wherein the normal transaction end means that all operations in the transaction are normally completed, and the transaction end means that a certain operation in the transaction fails to be executed.
It should be noted that, when all operations in a transaction are normally completed, a transaction is committed, where the transaction is committed and all operations in a buffer of a memory are written into a buffer in a disk, and when the transaction is committed, the end of the transaction can be determined, and the time when the transaction starts to commit can be used as the time when the transaction ends. When a transaction includes one operation, the time when the transaction starts to commit may be the time when the operation starts to write to the buffer of the disk, and when the transaction includes multiple operations, the time when the transaction starts to commit may be the time when the first operation in the transaction starts to write to the buffer of the disk.
And when the execution of a certain operation in the transaction fails, performing transaction rollback, wherein the transaction rollback means that a fault occurs in the execution process of the certain operation and all completed operations in the transaction are cancelled. Then when a transaction rollback occurs, the end of the transaction may be determined, and the time at which the transaction starts rolling back may be taken as the time at which the transaction ends. The transaction includes multiple operations, and partial operations have been executed before a certain operation fails, the rollback of the transaction needs to completely withdraw the executed partial operations, and the time when the transaction ends may be the time when the rollback starts for the first operation in the executed partial operations.
For example, the processor may complete execution of all operations in the transaction and commit the transaction after receiving a commit instruction sent by the user, may fail execution of a certain operation in the transaction and rollback the transaction after receiving a rollback instruction sent by the user, may automatically commit the transaction after completion of execution of all operations in the transaction, or may automatically rollback the transaction after failure of execution of a certain operation in the transaction. For example, when a transaction includes one or more DML statements, the transaction COMMIT may occur after receiving a COMMIT statement sent by a user, or the transaction ROLLBACK may occur after receiving a ROLLBACK statement sent by a user. When the transaction comprises a DDL statement or a DCL statement, the transaction is automatically submitted or rolled back after the DDL statement or the DCL statement is executed.
When the data page in the disk needs to be updated, the data page in the disk is not directly updated, but the data page in the disk is read into a buffer area of a memory, and then the data page in the buffer area of the memory is updated, wherein the updating of the data page in the buffer area of the memory can be the updating of partial data or all data in the data page. The data page in the buffer area of the memory is written into the buffer area of the magnetic disk because the data page is inconsistent with the content of the data page in the magnetic disk due to being updated, the read-write times of the magnetic disk are reduced, and then the data page in the buffer area of the magnetic disk is written into the magnetic disk, so that the data page in the memory is consistent with the data page in the magnetic disk, and the disk refreshing of the data page is realized.
For example, when a user needs to perform data addition on data in a disk, an increase instruction is sent to a processor, the processor reads a data page corresponding to the increase instruction from the disk, reads the data page corresponding to the increase instruction into a buffer pool of a memory, and then the processor increases the data in the data page in the buffer pool of the memory and writes the data page with the data added into the disk. When a user needs to update data in a magnetic disk, an update instruction is sent to a processor, the processor reads a data page corresponding to the update instruction from the magnetic disk, the data page corresponding to the update instruction is read into a buffer pool of a memory, then the processor updates the data in the data page in the buffer pool of the memory, and writes the data page after updating the data into the magnetic disk. When a user needs to delete data in the magnetic disk, a deletion instruction is sent to the processor, the processor reads a data page corresponding to the deletion instruction from the magnetic disk, the data page corresponding to the deletion instruction is read into a buffer pool of the memory, then the processor deletes the data in the data page in the buffer pool of the memory, and writes the data page with the data deleted into the magnetic disk.
The transaction start committing means that the operation on the buffer area of the memory is started to be written into the buffer area of the disk, for example, when data updating is performed on a data page in the buffer area of the memory, the data page in the buffer area of the memory, on which the data updating is performed, is written into the buffer area of the disk; when data increase is carried out on the data page in the buffer area of the memory, the data page which is subjected to the data increase in the buffer area of the memory is written into the buffer area of the magnetic disk; when data page in the buffer area of the memory is deleted, the data page in the buffer area of the memory, which is deleted, is written into the buffer area of the disk. For example, the fsync function may be used to synchronize the data page in the buffer of the memory to the buffer of the disk.
When the transaction begins to commit, the data page in the buffer area of the disk is written into the disk, so that when the transaction begins to commit, the data page in the buffer area of the memory is written into the disk, and when the transaction begins to commit, the disk refreshing of the data page is carried out.
S102, after the data page is flushed, the WBL log is flushed.
When the data page in the buffer pool of the memory is operated, all operations on the data page are recorded in the log buffer area of the memory, so as to generate a WBL log, and the WBL log records all operations such as updating, deleting, adding, submitting, rolling back and the like on the data page in the buffer area of the memory. The WBL log includes one or more of a data modification log, a transaction commit log, and a transaction rollback log. The data modification log may record a location of modified data in a data page in the buffer area of the memory, for example, when data in the data page in the buffer area of the memory is updated, the data modification log may record a location of the updated data; when deleting data in a data page in a buffer area of the memory, the data modification log can record the position of the deleted data; when the data page in the buffer area of the memory is subjected to the adding operation, the data modification log can record the position of the added data. The data modification log may also record the type of operations, such as update operations, delete operations, update operations, etc., on the data pages in the buffer of the memory. The data modification log may also record the ID of the data page where the modified data is located.
Illustratively, the data modification log may include at least one of a transaction update log, a transaction delete log, and a transaction add log. The data updating log records the updating operation of the data page in the buffer area of the memory and the position of the updated data, the data deleting log records the deleting operation of the data page in the buffer area of the memory and the position of the deleted data, and the transaction adding log records the adding operation of the data page in the buffer area of the memory and the position of the added data.
The transaction commit log records data pages in the buffer committed to disk in the memory's buffer. The transaction rollback log records at least one of a data page on which an update operation has been performed before a certain operation fails to be performed, a data page on which a delete operation has been performed before a certain operation fails to be performed, and a data page on which an add operation has been performed before a certain operation fails to be performed in a buffer area of the memory.
In some embodiments, when an update operation is performed on a data page in a buffer of the memory, the update operation and a location of update data corresponding to the update operation may be written into a log buffer of the memory, so as to generate a transaction update log. When a data page in the buffer area of the memory is deleted, the delete operation and the location of the delete data corresponding to the delete operation may be written into a log buffer area of the memory, so as to generate a transaction delete log. When the add operation is performed on the data page in the buffer area of the memory, the add operation and the location of the add data corresponding to the add operation may be written into the log buffer area of the memory, so as to generate a transaction add log. After the data page in the buffer area of the memory is flushed, the data page submitted to the buffer area in the disk in the buffer area of the memory may be written into a log buffer area of the memory, and a transaction submission log is generated. And writing the data page subjected to transaction rollback in the buffer area of the memory into a log buffer area of the memory to generate a transaction rollback log.
Because the WBL log comprises the data modification log, and for the operations of updating, deleting, adding and the like of the data page, the data modification log records the position and the modification type of the modified data in the data page, the data before modification and the data after modification do not need to be recorded, and the log amount of the data modification log is reduced, thereby reducing the log amount of the WBL log. And because the data page disk refreshing is finished before the WBL log disk refreshing, the half-page writing problem can not occur when the WBL log disk refreshing is finished, so that a full-page writing mechanism is not required to be written, and the log data volume is further reduced.
At present, in order to improve the security of data, a primary and secondary storage manner is usually adopted to store data, for example, data may be stored in disks of two computers, for convenience of description, the two computers are referred to as a primary computer and a secondary computer, data page flushing and WBL log flushing may be completed in the primary computer, and then data pages and WBL logs are stored in the disk of the primary computer. When a WBL log in a disk of a host computer is copied to a disk of a standby computer, an existing active/standby copy method needs to use 8K as a minimum unit to avoid the problem of half-page write of 8K pages. Because the data page disk refreshing is already finished before the WBL log disk refreshing in the application, the problem of half-page writing can not occur when the WBL log disk refreshing is finished, and the WBL log in the host computer can be sent to the standby computer by taking 4K as the minimum unit.
After receiving the main and standby synchronous instruction sent by the user, the main computer sends the WBL log in the disk of the main computer to the standby computer, and the standby computer analyzes the WBL log and modifies the data page in the disk of the standby computer according to the WBL log.
As an implementation manner, the data modification log may record a position where last data of the modified data ends and a disk space occupied by the modified data, and then the position of the modified data may be calculated according to the position where last data of the modified data ends and the disk space occupied by the modified data.
For example, when data in a data page in a buffer area of a memory is updated, a data modification log may record an end position of last data of the updated data and a disk space occupied by the updated data, and a position of the updated data may be determined according to the end position of last data of the updated data and the disk space occupied by the updated data. When data in a data page in a buffer area of a memory is deleted, a data modification log can record the end position of the last data of the deleted data and the disk space occupied by the deleted data, and then the position of the deleted data can be determined according to the end position of the last data of the deleted data and the disk space occupied by the deleted data. When the data page in the buffer area of the memory is subjected to the adding operation, the data modification log can record the end position of the last data of the added data and the disk space occupied by the added data, and the position of the added data can be determined according to the end position of the last data of the added data and the disk space occupied by the added data.
In some embodiments, when a data page in a buffer pool of a memory is modified, a position of modified data to be modified in the data page may be locked, so as to reduce performance consumption caused by large concurrent locking conflicts. Illustratively, the minimum granularity of locking may be byte level locking.
For example, when an update operation is performed on a data page in a buffer pool of the memory, a position of update data corresponding to the update operation in the data page may be locked. When the data page in the buffer pool of the memory is subjected to the adding operation, the position of the added data corresponding to the adding operation in the data page can be locked.
According to the data storage method, the data page is firstly flushed, and then the WBL log is flushed, and because the WBL log comprises the data modification log, and the data modification log records the position and modification type of the modified data in the data page, the data before modification and the data after modification do not need to be recorded, the log amount of the data modification log is reduced, and therefore the log amount of the WBL log is reduced.
Fig. 2 is a flowchart of a data storage method according to an embodiment of the present disclosure. Referring to fig. 2, a data storage method provided in an embodiment of the present application includes:
s201, when the data page in the buffer pool of the memory is modified, the position of the modified data in the data page is calculated.
When a data page in a buffer pool of the memory is modified, the end position of the last data of the modified data in the data page and the disk space occupied by the modified data are obtained, and the position of the modified data is calculated according to the end position of the last data of the modified data and the disk space occupied by the modified data. The modified data comprises at least one of updated data, deleted data and added data, wherein the updated data refers to the updating operation of the data in the data page, the deleted data refers to the deleting operation of the data in the data page, and the added data refers to the adding operation of the data page.
For example, when updating data in a data page in a buffer area of a memory, a location of the updated data is calculated according to an end location of last data of the updated data and a disk space occupied by the updated data. When deleting data in a data page in a buffer area of the memory, the position of the deleted data can be calculated according to the end position of the last data of the deleted data and the disk space occupied by the deleted data. When the data page in the buffer area of the memory is added, the position of the added data can be calculated according to the end position of the last data of the added data and the disk space occupied by the added data.
S202, locking the position of the modified data in the data page.
Because the data in the disk is a shared resource, access by multiple users or multiple applications is supported, and the access processes are independent, when multiple users concurrently access the data in the disk, a situation where multiple transactions access the same data at the same time may occur in the disk. Incorrect data may be read and stored without control over concurrent operations, destroying database consistency. When a user needs to modify a data page in a disk, a request is sent to a processor first to lock the data page. After the processor locks the data page, the processor does not modify the data page according to modification instructions sent by other users before the data page releases the lock. And the position of the modified data in the data page is locked, so that the performance consumption is reduced.
S203, when the transaction is started to be submitted, the data page is flushed.
After the data page in the buffer area of the memory is updated, when the transaction begins to commit, the data page which is updated in the buffer area of the memory is written into the buffer area of the disk, and is written into the disk from the buffer area of the disk. After the data page in the buffer area of the memory is subjected to the increasing operation, the data page subjected to the increasing operation in the buffer area of the memory is written into the buffer area of the disk, and is written into the disk from the buffer area of the disk. When the data page in the buffer area of the memory is deleted, the data page which is deleted in the buffer area of the memory is written into the buffer area of the disk, and the data page is written into the disk from the buffer area of the disk.
When the transaction begins to commit, the data page in the buffer area of the disk is written into the disk, so that when the transaction begins to commit, the data page in the buffer area of the memory is written into the disk, and when the transaction begins to commit, the disk refreshing of the data page is carried out.
And S204, after the data page is flushed, flushing the WBL log.
The WBL log includes one or more of a data modification log, a transaction commit log, and a transaction rollback log. The data modification log may include at least one of a transaction update log, a transaction delete log, and a transaction add log.
When the data page in the buffer area of the memory is updated, the update operation and the position of the update data corresponding to the update operation may be written into the log buffer area of the memory, so as to generate a transaction update log. When a data page in the buffer area of the memory is deleted, the delete operation and the location of the deleted data corresponding to the delete operation may be written into a log buffer area of the memory, so as to generate a transaction delete log. When the data page in the buffer area of the memory is subjected to the adding operation, the adding operation and the position of the added data corresponding to the adding operation can be written into the log buffer area of the memory, and a transaction adding log is generated. After the data page in the buffer area of the memory is flushed, the data page submitted to the buffer area in the disk in the buffer area of the memory may be written into a log buffer area of the memory, and a transaction submission log is generated. And writing the data page subjected to transaction rollback in the buffer area of the memory into a log buffer area of the memory to generate a transaction rollback log. I.e., to generate WBL logs.
After the data page is flashed, the WBL log is flashed. For example, the data modification log in the log buffer area of the memory is written into the disk, the transaction commit log in the log buffer area of the memory is written into the disk, and the transaction rollback log in the log buffer area of the memory is written into the disk. It should be noted that, the transaction commit log and the transaction rollback log may be referred to as a commit log, and writing the transaction commit log and the transaction rollback log to the disk may be referred to as a log landing.
According to the data storage method, when the data page in the buffer pool of the memory is modified, the position of the modified data in the data page is calculated, then the position of the modified data in the data page is locked, the locked data volume is reduced, and further the performance consumption caused by locking is reduced. And then after the data page is flushed, the WBL log is flushed, and because the WBL log comprises the data modification log which records the position and the modification type of the modified data, the data before modification and the data after modification do not need to be recorded, and the log amount of the WBL log is reduced. And because the data page disk refreshing is finished before the WBL log disk refreshing, the problem of half-page writing does not occur in the WBL log disk refreshing process, a full-page writing mechanism is not needed, and the log data volume is further reduced. In addition, because the server needs to perform corresponding processing according to the WBL log when restarting and recovering, and because the amount of the WBL log is reduced, the actual operation can be reduced when restarting and recovering the server, and the system recovery time is reduced.
Fig. 3 is a flowchart of a method for recovering from a server restart according to an embodiment of the present application. Referring to fig. 3, a method for recovering a server restart provided in an embodiment of the present application includes:
s301, when the transaction is started to be submitted, the data page is flushed.
S302, after the data page is flushed, the WBL log is flushed.
S303, acquiring the target log from the disk of the server.
And restarting the server after the operating system of the server is down, wherein operations recorded by WBL logs need to be played back in the process of restarting the server. The data page is restored to the state before the fault occurs by replaying the operation recorded in the transaction rollback log in the WBL log, and after the server is started, the processor acquires a target log from a disk of the server, wherein the target log can be a data modification log, a transaction commit log or a transaction rollback log in the WBL log.
S304, replaying the target log and judging whether the target log is the last log in the WBL log.
After a log is fetched, the next log is continuously fetched, if the next log is not fetched, the currently fetched log is determined to be the last log in the WBL log, and if the next log is fetched, the currently fetched log is determined to be not the last log in the WBL log. The starting log entry for a WBL log replay may be the first log entry after the last full checkpoint.
If so, step S305 is executed, otherwise, step S306 is executed.
S305, the control server receives a request for modifying the data in the disk.
When the fetched log is the last log in the WBL logs, the WBL logs are indicated to be played back completely. The server may receive a user request to modify data in the disk.
S306, judging whether the target log is a transaction modification log.
The type of the target log may be determined according to information recorded in the target log. For example, if the target log records the location of the update data, the target data is determined to be the transaction update log. And if the target log records the position of the deleted data, determining that the target data is a transaction deleted log. And if the target log records the position of the added data, determining the target data as the transaction added log.
If so, step S303 is executed, otherwise, step S307 is executed.
S307, judging whether the target log is a transaction submission log.
And judging whether the target log records a data page submitted to a buffer area of a disk in the buffer area of the memory, if so, determining that the target log is a transaction submission log, and if not, determining that the target log is a non-transaction submission log.
If yes, go to step S303, otherwise go to step S308.
When the data page is submitted to the buffer of the disk from the buffer of the memory, the data page submitted to the buffer of the disk from the buffer of the memory is recorded in the clog log. The log includes a transaction commit log and a transaction rollback log, and a data page committed from the memory buffer to the disk buffer may be recorded in the transaction commit log.
S308, determining the target log as a transaction rollback log.
And then step S303 is performed.
According to the server restart recovery method, the WBL logs in the disk are played back, and due to the reduction of the number of the WBL logs, actual operation can be reduced during the server restart recovery, and the system recovery time is reduced.
An embodiment of the present application further provides a data storage system 10, and as shown in fig. 4, the data storage system 10 includes: the transaction processing module 101, the transaction processing module 101 includes a transaction management module 1011 and a log management module 1012;
the transaction management module 1011 is configured to flush the data page when the transaction starts to commit;
the log management module 1012 is configured to flush WBL logs, including PCM logs that include data page modification locations and modification operation types.
The transaction processing module 101 may guarantee the transaction ACID characteristic.
In some embodiments, the transaction processing module 101 may further include a concurrency control module 1013 and a lock manager 1014, the concurrency control module being configured to ensure that when multiple transactions access the same data in the database at the same time, the isolation and consistency of the transactions and the consistency of the database are not destroyed. The lock manager 1014 is configured to lock the data modification location to reduce performance consumption caused by large concurrent locking conflicts.
An embodiment of the present application further provides a computing device, where the computing device includes a processor, and the processor is configured to execute the data storage method.
The computer device may include a mobile phone, a tablet computer, a notebook computer, a palm top computer, a Personal Digital Assistant (PDA), a server, and other computing devices.
In the description of the present application, it should be noted that unless otherwise specifically stated or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, and may for example be fixed or indirectly connected through intervening media, or may be interconnected between two elements or may be in the interactive relationship between two elements. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as the case may be.
It is expressly intended that all such additional apparatus or elements be included within this description or this summary, and be constructed and operative in a particular orientation, and not limited to the specific embodiments disclosed herein. In the description of the present application, "a plurality" means two or more unless specifically stated otherwise.
The terms used in the description and claims of the present application and in the accompanying drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A method of storing data, comprising:
when the transaction begins to submit, the data page is flushed;
and flushing a disk of the post-write log, wherein the post-write log comprises a data modification log, and the data modification log records the position and modification type of modification data corresponding to modification operation when the data modification log carries out modification operation on a data page in a buffer area of a memory.
2. The method of claim 1, wherein the transaction comprises a plurality of modification operations to a data page in a buffer of memory;
when the transaction begins to commit, the flushing of the data page includes:
and when the first modification operation in the plurality of modification operations in the transaction starts to be written into a buffer area of a disk, writing a data page in the buffer area of the disk into the disk.
3. The method according to claim 1, wherein the flushing the write-after log to disk specifically comprises:
when a data page in a buffer area of the memory is modified, writing the position and the modification type of modified data corresponding to the modification operation into a log buffer area of the memory to generate a data modification log;
and flushing the data modification log.
4. The method of claim 3, wherein the modification operation comprises at least one of an update operation, a delete operation, and an add operation, and wherein the data modification log comprises at least one of a transaction update log, a transaction delete log, and a transaction add log;
when the data page in the buffer area of the memory is modified, writing the position of the modified data corresponding to the modification into the log buffer area of the memory to generate a data modification log, which specifically includes:
when a data page in a buffer area of the memory is updated, writing the position of updated data corresponding to the update operation into a log buffer area of the memory to generate a transaction update log;
when a data page in a buffer area of the memory is deleted, writing the position of deleted data corresponding to the deletion operation into a log buffer area of the memory to generate a transaction deletion log;
when the data page in the buffer area of the memory is subjected to the adding operation, the position of the added data corresponding to the adding operation is written into the log buffer area of the memory, and a transaction adding log is generated.
5. The method of any of claims 1-4, wherein before the transaction begins to commit, the method further comprises:
when a data page in a buffer area of an internal memory is modified, calculating the position of modified data corresponding to the modification operation;
and locking the position of the modified data corresponding to the modification operation.
6. The method according to claim 5, wherein when performing a modification operation on a data page in a buffer of an internal memory, calculating a position of modification data corresponding to the modification operation includes:
when the data page in the buffer area of the memory is modified, the position of the modified data is calculated according to the end position of the last data of the modified data corresponding to the modification operation and the disk space occupied by the modified data.
7. The method according to claim 6, wherein when performing a modification operation on a data page in a buffer of the memory, calculating a position of the modified data according to an end position of a previous piece of modified data corresponding to the modification operation and a disk space occupied by the modified data, specifically comprising:
when a data page in a buffer area of the memory is updated, calculating the position of the updated data according to the end position of the last data of the updated data corresponding to the updating operation and the disk space occupied by the updated data;
when a data page in a buffer area of the memory is deleted, calculating the position of deleted data according to the end position of the last data of the deleted data corresponding to the deletion operation and the disk space occupied by the deleted data;
when the data page in the buffer area of the memory is subjected to the adding operation, the position of the added data is calculated according to the end position of the last data of the added data corresponding to the adding operation and the disk space occupied by the added data.
8. The method of any one of claims 1-4, wherein the post-write log comprises a transaction commit log or a transaction rollback log, wherein the transaction commit log records data pages in a buffer of the memory committed to disk, and wherein the transaction rollback log records data pages on which modification operations have been performed before a failed operation;
the flushing the disk of the post-write log specifically comprises:
after the data page in the buffer area of the memory is flushed, writing the data page submitted to the buffer area in the disk in the buffer area of the memory into a log buffer area of the memory, generating a transaction submission log, and flushing the transaction submission log;
or writing the data page which has executed the modification operation before the failure operation in the buffer area of the memory into a log buffer area of the memory, generating a transaction rollback log, and flushing the transaction rollback log.
9. The method of any of claims 1-4, wherein after flushing the write-after log, the method further comprises:
acquiring a target log from a disk, playing back the target log, and judging whether the target log is the last log in a post-write log;
if not, continuously acquiring the target log from the disk until the target log is the last log in the post-write log;
if so, the control computer receives a request for modifying the data in the disk.
10. A computing device comprising a processor configured to perform the data storage method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211343759.0A CN115576494B (en) | 2022-10-31 | 2022-10-31 | Data storage method and computing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211343759.0A CN115576494B (en) | 2022-10-31 | 2022-10-31 | Data storage method and computing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115576494A true CN115576494A (en) | 2023-01-06 |
| CN115576494B CN115576494B (en) | 2023-11-03 |
Family
ID=84587481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211343759.0A Active CN115576494B (en) | 2022-10-31 | 2022-10-31 | Data storage method and computing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115576494B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030061232A1 (en) * | 2001-09-21 | 2003-03-27 | Dun & Bradstreet Inc. | Method and system for processing business data |
| CN112597254A (en) * | 2020-12-07 | 2021-04-02 | 中国科学院计算技术研究所 | Hybrid DRAM-NVM (dynamic random Access memory-non volatile memory) main memory oriented online transactional database system |
| US20220253409A1 (en) * | 2021-02-05 | 2022-08-11 | International Business Machines Corporation | Cleaning compensated change records in transaction logs |
-
2022
- 2022-10-31 CN CN202211343759.0A patent/CN115576494B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030061232A1 (en) * | 2001-09-21 | 2003-03-27 | Dun & Bradstreet Inc. | Method and system for processing business data |
| CN112597254A (en) * | 2020-12-07 | 2021-04-02 | 中国科学院计算技术研究所 | Hybrid DRAM-NVM (dynamic random Access memory-non volatile memory) main memory oriented online transactional database system |
| US20220253409A1 (en) * | 2021-02-05 | 2022-08-11 | International Business Machines Corporation | Cleaning compensated change records in transaction logs |
Non-Patent Citations (3)
| Title |
|---|
| ARULRAJ JOY: "Write-Behind Logging", 《PROCEEDINGS OF THE VLDB ENDOWMENT》, pages 1 - 12 * |
| 朱阅岸: ""透过新硬件坏境下的存储技术，看未来数据库系统崛起"", pages 7, Retrieved from the Internet <URL:https://www.51cto.com/article/560284.html> * |
| 西门仙忍: ""Write-Behind-Logging"", pages 1 - 4, Retrieved from the Internet <URL:https://blog.csdn.net/duxingxia356/article/details/102908855> * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115576494B (en) | 2023-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11360863B2 (en) | Key-value store on persistent memory | |
| US7933927B2 (en) | Method and apparatus for building index of source data | |
| US9183236B2 (en) | Low level object version tracking using non-volatile memory write generations | |
| US5317731A (en) | Intelligent page store for concurrent and consistent access to a database by a transaction processor and a query processor | |
| US11132350B2 (en) | Replicable differential store data structure | |
| US7587429B2 (en) | Method for checkpointing a main-memory database | |
| US10754854B2 (en) | Consistent query of local indexes | |
| Graefe | A survey of B-tree logging and recovery techniques | |
| US9576038B1 (en) | Consistent query of local indexes | |
| JPH0212460A (en) | Parallel accessing to index tree | |
| US11567921B2 (en) | Rowgroup consolidation with global delta accumulation and versioning in distributed systems | |
| CN109871386A (en) | Multi version concurrency control (MVCC) in nonvolatile memory | |
| Graefe et al. | Instant recovery with write-ahead logging | |
| EP4170509A1 (en) | Method for playing back log on data node, data node, and system | |
| US20230297510A1 (en) | Write-behind optimization of covering cache | |
| CN115576494B (en) | Data storage method and computing device | |
| KR101419428B1 (en) | Apparatus for logging and recovering transactions in database installed in a mobile environment and method thereof | |
| CN116089359A (en) | Database snapshot generation method and device, electronic equipment and medium | |
| CN114816224A (en) | Data management method and data management device | |
| JP4139642B2 (en) | Database management method and system | |
| Kuhn et al. | Concurrency and Multiversioning | |
| WO2022212026A1 (en) | Rowgroup consolidation with global delta accumulation and versioning in distributed systems | |
| Matsuura | Designing a persistent-memory-native storage engine for SQL database systems | |
| CN117687807A (en) | Data processing method, device, electronic equipment and storage medium | |
| Ozsu | Notes on Database System Reliability |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |