CN105335098B - A kind of log file system performance improvement method based on storage level memory - Google Patents
A kind of log file system performance improvement method based on storage level memory Download PDFInfo
- Publication number
- CN105335098B CN105335098B CN201510621004.6A CN201510621004A CN105335098B CN 105335098 B CN105335098 B CN 105335098B CN 201510621004 A CN201510621004 A CN 201510621004A CN 105335098 B CN105335098 B CN 105335098B
- Authority
- CN
- China
- Prior art keywords
- log
- transferred
- log recording
- recording
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Abstract
The log file system performance improvement method based on SCM that the invention discloses a kind of, file system metadata and log are stored using SCM as memory device, optimize the read-write of metadata, and it is write by distinguishing to cover to write and add, only overwriting data, which will be covered, is written to log, additional data of writing directly write to file system and control the sequence of update to guarantee file system consistency, reduce log expense, promote file system performance；The characteristic that byte modification is pressed using SCM is realized the update of log byte granularity, reduces log data stream by calculating the difference between new and old log blocks.The invention mainly comprises the building of storage system, write log, garbage reclamation, data write back, system restores this five operations, can be used for various log file system, suitable for constructing high-performance, high capacity and the large memory system of high reliability.Solves the problems such as journaling techniques overhead in existing log file system is big, metadata writes back frequently, resume speed is slow after delay machine.
Description
Technical field
The invention belongs to technical field of computer data storage, more particularly, to a kind of day based on storage level memory
Will file system performance improvement method.
Background technique
Storage level memory (Storage Class Memory, abbreviation SCM) is a kind of novel nonvolatile storage, is had both interior
It deposits (memory) and stores the characteristic of (storage).The byte addressing of SCM, quick random access characteristic close to memory, because
This SCM can directly carry on rambus, this also allows conventional memory device too far behind to catch up.It is based on these brilliant property
Can, the appearance of SCM can not only solve the I/O bottleneck and energy consumption problem at high-performance data center, while will also cause computer
The change of system structure.Therefore, SCM is not only the hot spot of currently stored area research, all must in the future for a long period of time
It will be the core of field of storage research.
Traditional journaling techniques are a kind of common methods for guaranteeing file system consistency.Change file system it
Before, first more new data records in log.Log is generally stored in disk partition or journal file.After system failure, benefit
The mode that log recording writes back disk is subjected to system recovery with sequence.Traditional journaling techniques are set for block grade storage equipment
Meter, takes the strategy traded space for time.The problem of facing " writing twice " due to log mechanism is first write a log, then is write
File system, log mechanism may bring bigger expense to file system.In addition, traditional logs technology is in system
Disk is write with not making any distinction between all when recovery, in this way, multiple log copies of the same data block will occupy more parts of logs
Space is recorded, system will also write back repeatedly when restoring, not occupy a large amount of log spaces, also extend system recovery time.
Again since space utilization rate is low, it is easy to cause the situation that log is full, traditional logs technology must not be infrequently internal storage data
Write back disk.
Metadata refers to the system data of the feature for describing a file, and metadata is stored in file system in blocks
In system, the update of part metadata but will lead to the read-write to entire block, to be exaggerated magnetic disc i/o.Data shows only
There is 21% request for file I/O request, and be more than 50% request is metadata operation.Although metadata is all small update,
But the random small letter poor performance of disk.So the frequent access of metadata will be greatly reduced file system with a large amount of small write operations
The performance of system.
Summary of the invention
Aiming at the above defects or improvement requirements of the prior art, the present invention proposes a kind of log text based on storage level memory
Part system performance improvement method, it is intended that by storing file system metadata and log for SCM as memory device,
The access for optimizing metadata, using simplified log mechanism combination garbage reclamation mechanism, solve existing journaling techniques expense it is big,
Metadata write back frequently, I/O bottleneck the problems such as.
To achieve the above object, the present invention provides a kind of log file system performance raising side based on storage level memory
Method, building including storage system, write log, garbage reclamation, data write back, system restores this five operations, in which:
The building of storage system operates: one piece of SCM carry is added on rambus to original file system, with memory
Shared drive bus；And SCM is divided into log area and meta-data region, log area is used to record metadata log and number is write in covering
According to log, meta-data region is used for persistence metadata；
It writes journalizing: data is write into the addition of memory transaction and write file system；And by the metadata of memory transaction
Log area is write with overwriting data is covered；
Garbage collection operation: garbage reclamation is carried out at regular intervals；In garbage reclamation interval, if log area space is not
Foot, also carries out garbage reclamation；Log Label invalid at effective Log Label and low address at high address is exchanged, thus will
All effective log recordings are concentrated in one section of continuous log area space from log initial position；
Data written-back operation: after completing garbage reclamation, when log area space is still insufficient, the log area SCM will be cached in
In the overwriting data that covers write back to disk, metadata writes back to meta-data region；
System resumes operation: after system failure, coherency state is restored the system to using the log recording in SCM；
After system is restored, the log recording in SCM all cancels.
In general, through the invention it is contemplated above technical scheme is compared with the prior art, can obtain down and show
Beneficial effect:
1, the present invention solves the problems, such as that metadata present in existing file system frequently accesses caused magnetic disc i/o: will
Metadata is permanently stored in SCM, and the magnetic disc i/o problem for eliminating frequently access metadata ensure that file in conjunction with journaling techniques
System conformance.
2, the present invention improves the write performance of file system.Simplify traditional logging mode, distinguish it is additional write data and
Overwriting data is covered, the method updated using byte granularity reduces log data stream, and accelerate system resume speed.
3, the present invention is by metadata granularity refinement, while using garbage reclamation mechanism, takes full advantage of storage level memory, mentions
High space utilization rate.
Detailed description of the invention
Fig. 1 is system architecture schematic diagram of the invention；
Fig. 2 is log recording structural schematic diagram of the invention；
Fig. 3 is log recording Hash table schematic diagram of the invention；
Fig. 4 is log superblock schematic diagram of the invention；
Fig. 5 writes journalizing flow chart for of the invention；
Fig. 6 is garbage collection operation flow chart of the invention；
Fig. 7 is data written-back operation flow chart of the invention；
Fig. 8 is system resumes operation flow chart of the invention；
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as in addition, technical characteristic involved in the various embodiments of the present invention described below
Not constituting a conflict with each other can be combined with each other.
Fig. 1 is that system of the invention constructs schematic diagram.One SCM disk conduct of carry on rambus is needed in hardware deployment
Log disk and metadata disk, with DRAM shared drive bus；Include four operations on software configuration: writing journalizing, rubbish returns
It brings drill to an end work, data written-back operation, system resumes operation.In addition, SCM log disk is equipped with log superblock, log recording Hash table
With three kinds of structures of log recording.
Therefore, realize that the method for the present invention needs to add one piece of SCM carry first in original file system in rambus
On, with memory sharing rambus.SCM is divided into log area and meta-data region, log area for record metadata log and
Data logging is write in covering, and meta-data region is used for persistence metadata.
Secondly, mainly establishing in SCM has three kinds of log superblock, log recording Hash table and log recording storage knots
Structure.Wherein, log superblock is for recording first available log blocks, log capacity, remaining space, effective log recording number
The information such as amount, log recording initial position, last affairs end position, Current transaction end position；Log recording Hash table
For recording the corresponding newest log recording of data block and time newest log recording；Log recording, including log blocks and log mark
Label, log blocks are submitted to the buffer contents of log area for deposit transaction, and Log Label is used to describe the correlation of log recording
Information, including the corresponding disk block number of log recording, log recording effective marker and metadata flags position etc..
Finally, in view of metadata is smaller, to avoid the problem that writing amplification and space waste, by number first in meta-data region
It is set as 128bytes according to granularity.
Fig. 2 is log recording structural schematic diagram of the invention.Log recording includes log blocks and Log Label two parts.Day
Will block writes data block and meta data block for storing file system covering, and Log Label is used to store the description information of log blocks.
Log Label is made of four fields: disk block number blknr, significance bit valid, metadata flags position meta, where log blocks
Position scmnr.The operations such as positioning, lookup, marked invalid to log recording are carried out by Log Label.
Fig. 3 is log recording Hash table schematic diagram of the invention.Log recording Hash table is made of three fields: disk block
Number blknr, newest log recording scmlog, secondary newest log recording old_scmlog.Each table of log recording Hash table
Item is used to the corresponding newest log recording scmlog of recording disc block number blknr and time newest log recording old_scmlog.
Fig. 4 is log superblock schematic diagram of the invention.Log superblock includes seven fields: magic number, indicates that log is super
The mark of grade block；Log recording size indicates the size of log；Log space can be used, log also how many free space indicated；
Next available log blocks indicate assignable piece of starting point of log；Effective log recording quantity, indicates the number of effective log recording
Mesh；Log initial position indicates the initial position of log area；A upper affairs end position indicates the end of a upper affairs,
It is simultaneously also the starting point of Current transaction sequential write；Current transaction end position, indicates the end of Current transaction, and Current transaction is complete
Ground is recorded between an affairs end position and Current transaction end position.Log superblock has recorded the basic letter of log
Breath, it is ensured that the atomicity of memory transaction.
Fig. 5 writes journalizing flow chart for of the invention, writes journalizing and is used to memory transaction being submitted to logging device,
By the consistency for guaranteeing the control for writing data flow file system.
One memory transaction is managed by two double-linked circular lists: one is that chained list is write in addition, for managing addition
Write data buffer zone content；The other is covering write with metadata chained list, for managing covering writing buffer and metadata data
Buffer contents.Writing journalizing mainly includes two processes:
1, data are write into the addition of memory transaction and writes file system；
2, it by the metadata of memory transaction and covers overwriting data and writes log area.
Additional data of writing can write sequence by control to guarantee file system consistency, i.e., first write data into file system
System, after metadata write into log, and meta-data region is write back to when suitable；And it metadata and covers overwriting data and then needs
Guarantee integrality by write-ahead log mode.
That writes journalizing focuses on process 2, i.e., the content in memory transaction is submitted to logging device, and need to protect
Write operation interrupts the consistency that will not all destroy data to card at any time.Same data block only retains newest day in log area
Will record and time newest log recording.
Exclusive or updating method is used in writing logging process, and it is different from time newest log recording to find out the log blocks that will be written
Byte, realize log byte granularity update.
Writing journalizing, specific step is as follows:
(1.1) start one new memory transaction of submission, additional will write the data in chained list and write file system, and inside
It deposits and caches the affairs in log buffer area, completed if this time operating the file system data write-in being related to, be transferred to step (1.2)；
(1.2) investigate covering write with log blocks to be committed in metadata chained list, in log recording Hash table search described in
The corresponding file system blocks of log blocks to be committed whether there is time newest log recording and be transferred to step (1.3) if it does not exist；If
In the presence of being transferred to step (1.4)；
(1.3) log blocks to be committed are sequentially write into the clear position in log area, and by the log to be committed
The information of block is added on log recording Hash table；Here in two kinds of situation: first, institute is not found on log recording Hash table
The information for stating log blocks to be committed illustrates that this is to update the log blocks to be committed for the first time, therefore it is labeled as newest log
Record；Second, newest log recording is had found on log recording Hash table and does not find time newest log recording but, then most by it
New log recording is updated to time newest log recording, and the log blocks to be committed are labeled as newest log recording；It is transferred to step
(1.5)；
(1.4) exclusive or updating method is used, it is different from next newest log recording B1 to find out the log blocks B3 to be committed
Byte, i.e. P=B1 ⊕ B3 (⊕ is exclusive or symbol), analyze each byte in P, if 0, then it represents that log blocks B3 and log blocks
The byte of B1 is identical, does not do and updates；If non-zero, then it represents that log blocks B3 is different from the byte of log blocks B1, by P and B1
In the byte do XOR operation and realize update to log blocks B1.Since B1 is time newest log recording, it is invalid to have been marked as,
Even if renewal process is interrupted, the consistency and integrality of log affairs will not be destroyed.Update log records Hash table, by B3 pairs
The newest log recording answered is labeled as time newest log recording, and exclusive or updates the B1 after completing and is labeled as newest log recording；
It is transferred to step (1.5)；
(1.5) if all log blocks to be committed in current memory affairs have all been completed to submit, step (1.6) are transferred to；
Otherwise, step (1.2) are transferred to；
(1.6) Current transaction end position is set by the position of the last item log recording.It is transferred to step (1.7)；
(1.7) the corresponding data block of current memory affairs in log buffer area is transferred to step (1.8) labeled as in vain；
(1.8) a upper affairs end position is set by Current transaction end position.Indicate current memory affairs
It completes, the submission of next memory transaction can be received.
Fig. 6 is garbage collection operation flow chart of the invention, and the purpose of garbage collection operation is to recycle nothing in log area
The log recording of effect, specific method use Tag switching method.
Due to there are a large amount of invalid log recordings in log area, it is therefore desirable to carry out garbage reclamation at regular intervals.
In garbage reclamation interval, if log area insufficient space, also will do it garbage reclamation.In the present invention the characteristics of garbage reclamation
In Log Label invalid at effective Log Label and low address at high address being exchanged, thus by all effective log recordings
It concentrates in one from log initial position section of continuous log area space.
Specific step is as follows for garbage collection operation:
(2.1) assume there is pointer first and pointer last, pointer first is directed toward first log recording, and pointer last refers to
To the last item log recording.Judge whether first pointer is overlapped with last pointer.If being overlapped, step (2.8) are transferred to, otherwise,
It is transferred to step (2.2)；
(2.2) whether the log recording for checking that first is directed toward is invalid, if effectively, first is directed toward next log recording,
It is transferred to step (2.3)；If invalid, it is transferred to step (2.4)；
(2.3) judge whether first pointer is overlapped with last pointer.If being overlapped, step (2.8) are transferred to, otherwise, are transferred to step
Suddenly (2.2)；
(2.4) whether the log recording for checking that last is directed toward is invalid, if effectively, being transferred to step (2.5), otherwise, is transferred to step
Suddenly (2.6)；
(2.5) the last log recording being directed toward is write into the log recording of first direction, and first pointer is directed toward
Log recording is labeled as effective.Assuming that the corresponding disk block number of log recording that first is directed toward is blknr, then log note is searched
Hash table is recorded, updating the corresponding newest log recording of disk block blknr is the log recording that last pointer is directed toward.It is transferred to step
(2.7)；
(2.6) last pointer is directed toward a upper log recording, is transferred to step (2.3)；
(2.7) first pointer is directed toward next log recording, is transferred to step (2.2)；
(2.8) it is Current transaction end position by the position mark of first pointer, is transferred to step (2.9)；
(2.9) finally, discharging to continuous invalid log recording space, garbage collection operation is completed.
Fig. 7 is data written-back operation flow chart of the invention, data written-back operation be when log area insufficient space, into
Row data written-back operation.
After completing garbage reclamation, when log area space is still insufficient, overwriting data is covered by being cached in the log area SCM
Disk is write back to, metadata writes back to meta-data region；Data write back tactful main consideration two o'clock: the 1, succession of write back data；
2, the cold and hot degree of write back data.
For the succession of write back data, preferential write back data log in the present invention.In general the succession of data is wanted
Higher than the succession of metadata, therefore preferential write back data, is conducive to improve data and writes back efficiency.For the cold and hot of write back data
Degree, due to the logical order of log area, from low address to the data distribution of high address, formed one " natural " from being as cold as
The distribution of heat: more the data of " heat ", the tag addresses where effective log recording are higher；The more data of " cold ", effectively
Log recording where tag addresses it is lower.
Therefore, when execution data write back every time, since first log recording of log area, from low address to highland
Location selects several effective log recordings to write back, that is, realizes daily record data from being as cold as writing back for heat.
Specific step is as follows for data written-back operation:
(3.1) since first log recording, whether effective log recording is checked, if in vain, being transferred to step (3.2),
Otherwise, step (3.3) are transferred to；
(3.2) check whether next log recording be effective, if in vain, being transferred to step (3.2) and being otherwise transferred to step
(3.3)；
(3.3) the metadata flags position for investigating the log recording is transferred to step (3.4) and is otherwise transferred to if metadata
Step (3.5)；
(3.4) metadata is write back into meta-data region, is transferred to step (3.6)；
(3.5) data block back disk is write into covering, is transferred to step (3.6)；
(3.6) assume that the corresponding disk block number of data block write back is blknr, log recording Hash table is searched, by disk
The corresponding log recording of block number blknr is labeled as invalid.Delete the corresponding Hash node of disk block number blknr.It is transferred to step
(3.7)；
(3.7) step (3.2) are repeated, until by the corresponding data block back of all effective log recordings to disk and member
Data block back is to meta-data region.
Fig. 8 is system resumes operation flow chart of the invention, after system failure, needs to utilize the log recording in SCM
Restore the system to coherency state.After system is restored, the log recording in SCM all cancels.
Specific step is as follows for system resumes operation:
(4.1) since first log recording, whether effective log recording is checked, if in vain, being transferred to step (4.2),
Otherwise, step (4.3) are transferred to；
(4.2) check whether next log recording be effective, if in vain, being transferred to step (4.2) and being otherwise transferred to step
(4.3)；
(4.3) assuming that current log records corresponding disk block number is blknr, and current log record is write back to disk phase
The position answered and be invalid by current log recording mark.It is transferred to step (4.4)；
(4.4) log recording Hash table is searched, the corresponding Hash section of disk block number blknr in log recording Hash table is deleted
Point.It is transferred to step (4.5)；
(4.5) step (4.2) and (4.4) are repeated, until all data log records are restored to disk and metadata day
Will record is restored to meta-data region.It is transferred to step (4.6)；
(4.6) a upper affairs end mark and Current transaction end mark log starting final position is placed in set.
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to
The limitation present invention, any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should all include
Within protection scope of the present invention.
Claims (6)
1. a kind of log file system performance improvement method based on storage level memory, which is characterized in that including storage system
It constructs, write log, garbage reclamation, data write back, system restores this five operations, in which:
The building of storage system operates: one piece of SCM carry is added on rambus to original file system, with memory sharing
Rambus；And SCM is divided into log area and meta-data region, log area is for recording metadata log and covering overwriting data day
Will, meta-data region are used for persistence metadata；
It writes journalizing: data is write into the addition of memory transaction and write file system；And it by the metadata of memory transaction and covers
Overwriting data writes log area；
Garbage collection operation: garbage reclamation is carried out at regular intervals；In garbage reclamation interval, if log area insufficient space,
Also garbage reclamation is carried out；Log Label invalid at effective Log Label and low address at high address is exchanged, thus by institute
Effective log recording is concentrated in one section of continuous log area space from log initial position；
Data written-back operation: it after completing garbage reclamation, when log area space is still insufficient, will be cached in the log area SCM
It covers overwriting data and writes back to disk, metadata writes back to meta-data region；
System resumes operation: after system failure, coherency state is restored the system to using the log recording in SCM；System
After recovery, the log recording in SCM all cancels；
In the building operation of storage system, mainly establishing in SCM has log superblock, log recording Hash table and log note
Record three kinds of storage organizations；Wherein, log superblock for record first available log blocks, log capacity, remaining space, effectively
Log recording quantity, log recording initial position, last affairs end position, Current transaction end position；Log recording is breathed out
Uncommon table is for recording the corresponding newest log recording of data block and time newest log recording；Log recording, including log blocks and day
Will label, wherein log blocks are submitted to the buffer contents of log area for deposit transaction, and Log Label is for describing log note
The relevant information of record, including the corresponding disk block number of log recording, log recording effective marker and metadata flags position；
Write in journalizing described, a memory transaction is managed by two double-linked circular lists: one is that chain is write in addition
Table writes data buffer zone content for managing addition；The other is covering write with metadata chained list, for manage covering Write post
Area and metadata data buffer contents；The journalizing of writing specifically includes:
(1.1) a new memory transaction is submitted, the additional data write in chained list are write into file system, and slow in memory log
It rushes in area and caches the affairs, completed if this time operating the file system data write-in being related to, be transferred to step (1.2)；
(1.2) investigate covering write with log blocks to be committed in metadata chained list, searched in log recording Hash table described wait mention
It hands over the corresponding file system blocks of log blocks to whether there is time newest log recording and is transferred to step (1.3) if it does not exist；If it exists,
It is transferred to step (1.4)；
(1.3) log blocks to be committed are sequentially write into the clear position in log area, and by the log blocks to be committed
Information is added on log recording Hash table；It is transferred to step (1.5)；
(1.4) exclusive or updating method is used, the log blocks B3 to be committed byte different from next newest log recording B1 is found out,
That is P=B1 ⊕ B3, wherein ⊕ is exclusive or symbol, each byte in P is analyzed, if 0, then it represents that log blocks B3 and log blocks B1
The byte it is identical, do not do and update；If non-zero, then it represents that log blocks B3 is different from the byte of log blocks B1, will be in P and B1
The byte does update of the XOR operation realization to log blocks B1；By the corresponding newest log recording of B3 labeled as time newest log note
Record, exclusive or update the B1 after completing and are labeled as newest log recording；It is transferred to step (1.5)；
(1.5) if all log blocks to be committed in current memory affairs have all been completed to submit, step (1.6) are transferred to；Otherwise,
It is transferred to step (1.2)；
(1.6) Current transaction end position is set by the position of the last item log recording；It is transferred to step (1.7)；
(1.7) the corresponding data block of current memory affairs in log buffer area is transferred to step (1.8) labeled as in vain；
(1.8) a upper affairs end position is set by Current transaction end position；Indicate that current memory affairs have been completed,
It can receive the submission of next memory transaction.
2. the method as described in claim 1, which is characterized in that the log recording Hash table is made of three fields: disk
Block number blknr, newest log recording scmlog, secondary newest log recording old_scmlog；The each of log recording Hash table
List item is used to the corresponding newest log recording scmlog of recording disc block number blknr and time newest log recording old_scmlog.
3. the method as described in claim 1, which is characterized in that the step (1.3) in two kinds of situation: first, log recording
The information that the log blocks to be committed are not found on Hash table illustrates that this is to update the log blocks to be committed for the first time, will
It is labeled as newest log recording；Second, have found on log recording Hash table newest log recording do not find but it is time newest
Log recording, then be updated to time newest log recording for its newest log recording, and the log blocks to be committed are labeled as newest day
Will record.
4. method according to claim 1 or 2, which is characterized in that specific step is as follows for the garbage collection operation:
(2.1) it is equipped with pointer first and pointer last, pointer first is directed toward first log recording, and pointer last is directed toward last
One log recording；Judge whether first pointer is overlapped with last pointer；If being overlapped, step (2.8) are transferred to, otherwise, are transferred to step
Suddenly (2.2)；
(2.2) whether the log recording for checking that first is directed toward is invalid, if effectively, first is directed toward next log recording, is transferred to
Step (2.3)；If invalid, it is transferred to step (2.4)；
(2.3) judge whether first pointer is overlapped with last pointer；If being overlapped, it is transferred to step (2.8) and is otherwise transferred to step
(2.2)；
(2.4) whether the log recording for checking that last is directed toward is invalid, if effectively, being transferred to step (2.5) and being otherwise transferred to step
(2.6)；
(2.5) the last log recording being directed toward is write to the log recording of first direction, and the log that first pointer is directed toward
Recording mark is effective；Assuming that the corresponding disk block number of log recording that first is directed toward is blknr, then log recording Kazakhstan is searched
Uncommon table, updating the corresponding newest log recording of disk block blknr is the log recording that last pointer is directed toward；It is transferred to step (2.7)；
(2.6) last pointer is directed toward a upper log recording；It is transferred to step (2.3)；
(2.7) first pointer is directed toward next log recording, is transferred to step (2.2)；
It (2.8) is Current transaction end position by the position mark of first pointer；It is transferred to step (2.9)；
(2.9) continuous invalid log recording space is discharged, completes garbage collection operation.
5. method according to claim 1 or 2, which is characterized in that specific step is as follows for the data written-back operation:
(3.1) since first log recording, whether effective log recording is checked, if in vain, being transferred to step (3.2), otherwise,
It is transferred to step (3.3)；
(3.2) check whether next log recording be effective, if in vain, being transferred to step (3.2), otherwise, be transferred to step (3.3)；
(3.3) the metadata flags position for investigating the log recording is transferred to step (3.4) and is otherwise transferred to step if metadata
(3.5)；
(3.4) metadata is write back into meta-data region, is transferred to step (3.6)；
(3.5) data block back disk is write into covering, is transferred to step (3.6)；
(3.6) assume that the corresponding disk block number of data block write back is blknr, log recording Hash table is searched, by disk block number
The corresponding log recording of blknr is labeled as invalid；Delete the corresponding Hash node of disk block number blknr；It is transferred to step (3.7)；
(3.7) step (3.2) are repeated, until by the corresponding data block back of all effective log recordings to disk and metadata
Block writes back to meta-data region.
6. method according to claim 1 or 2, which is characterized in that specific step is as follows for the system resumes operation:
(4.1) since first log recording, whether effective log recording is checked, if in vain, being transferred to step (4.2), otherwise,
It is transferred to step (4.3)；
(4.2) check whether next log recording be effective, if in vain, being transferred to step (4.2), otherwise, be transferred to step (4.3)；
(4.3) assuming that current log records corresponding disk block number is blknr, and it is corresponding that current log record is write back to disk
Position and be invalid by current log recording mark；It is transferred to step (4.4)；
(4.4) log recording Hash table is searched, the corresponding Hash node of disk block number blknr in log recording Hash table is deleted；
It is transferred to step (4.5)；
(4.5) step (4.2) and (4.4) are repeated, until all data log records are restored to disk and metadata log note
Record is restored to meta-data region；It is transferred to step (4.6)；
(4.6) a upper affairs end mark and Current transaction end mark log starting final position is placed in set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510621004.6A CN105335098B (en) | 2015-09-25 | 2015-09-25 | A kind of log file system performance improvement method based on storage level memory |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510621004.6A CN105335098B (en) | 2015-09-25 | 2015-09-25 | A kind of log file system performance improvement method based on storage level memory |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105335098A CN105335098A (en) | 2016-02-17 |
| CN105335098B true CN105335098B (en) | 2019-03-26 |
Family
ID=55285679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510621004.6A Active CN105335098B (en) | 2015-09-25 | 2015-09-25 | A kind of log file system performance improvement method based on storage level memory |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105335098B (en) |
Families Citing this family (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105786410A (en) * | 2016-03-01 | 2016-07-20 | 深圳市瑞驰信息技术有限公司 | Method for increasing processing speed of data storage system and data storage system |
| CN107479824B (en) * | 2016-06-08 | 2020-03-06 | 宜鼎国际股份有限公司 | Redundant disk array system and data storage method thereof |
| CN106202307B (en) * | 2016-07-01 | 2019-10-11 | 百势软件(北京)有限公司 | A kind of batch log preservation method and device |
| CN108334277B (en) * | 2017-05-10 | 2019-06-28 | 中兴通讯股份有限公司 | A kind of log write-in and synchronous method, device, system, computer storage medium |
| CN107220342A (en) * | 2017-05-26 | 2017-09-29 | 郑州云海信息技术有限公司 | The control method and system of a kind of distributed data base |
| US11177940B2 (en) | 2017-06-20 | 2021-11-16 | 707 Limited | Method of evidencing existence of digital documents and a system therefor |
| CN107291924B (en) * | 2017-06-29 | 2020-08-14 | 深信服科技股份有限公司 | Synchronous log replication control method and system for disaster recovery backup system |
| US11301433B2 (en) * | 2017-11-13 | 2022-04-12 | Weka.IO Ltd. | Metadata journal in a distributed storage system |
| CN107908370B (en) * | 2017-11-30 | 2021-07-06 | 新华三技术有限公司 | Data storage method and device |
| CN108121789B (en) * | 2017-12-19 | 2020-06-30 | 苏州精濑光电有限公司 | Log management method and system |
| CN110018784B (en) | 2018-01-09 | 2023-01-10 | 阿里巴巴集团控股有限公司 | Data processing method and device and computing equipment |
| CN108829345B (en) * | 2018-05-25 | 2020-02-21 | 华为技术有限公司 | Data processing method of log file and terminal equipment |
| CN109669632A (en) * | 2018-12-10 | 2019-04-23 | 浪潮电子信息产业股份有限公司 | Metadata wiring method, device and medium based on distributed objects storage system |
| CN110968269A (en) * | 2019-11-18 | 2020-04-07 | 华中科技大学 | SCM and SSD-based key value storage system and read-write request processing method |
| CN111104254A (en) * | 2019-11-29 | 2020-05-05 | 北京浪潮数据技术有限公司 | Storage system data flashing method, device, equipment and readable storage medium |
| CN111414320B (en) * | 2020-02-20 | 2023-06-06 | 上海交通大学 | Method and system for constructing disk cache based on nonvolatile memory of log file system |
| CN111708481B (en) * | 2020-04-24 | 2021-04-06 | 浙江大学 | Solid State Disk (SSD) double-area wear leveling method based on super block |
| CN114281762B (en) * | 2022-03-02 | 2022-06-03 | 苏州浪潮智能科技有限公司 | Log storage acceleration method, device, equipment and medium |
| CN114579529B (en) * | 2022-05-07 | 2022-08-05 | 深圳市杉岩数据技术有限公司 | Local storage method and system based on redirection and log mixing |
| CN115048046B (en) * | 2022-05-26 | 2022-11-25 | 北京华昱卓程软件有限公司 | Log file system and data management method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1454349A (en) * | 2000-06-07 | 2003-11-05 | 处理存储器有限公司 | A method and system for highly-parallel logging and recovery operation in main-memory transaction processing systems |
| CN102024021A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Method for logging metadata in logical file system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7305421B2 (en) * | 2001-07-16 | 2007-12-04 | Sap Ag | Parallelized redo-only logging and recovery for highly available main memory database systems |
-
2015
- 2015-09-25 CN CN201510621004.6A patent/CN105335098B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1454349A (en) * | 2000-06-07 | 2003-11-05 | 处理存储器有限公司 | A method and system for highly-parallel logging and recovery operation in main-memory transaction processing systems |
| CN102024021A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Method for logging metadata in logical file system |
Non-Patent Citations (4)
| Title |
|---|
| High Performance Database Logging using Storage Class Memory;Ru Fang等;《IEEE》;20111231;第1221-1231页 |
| Storage class memory;Lam C H.等;《Proceedings of the 10th IEEE International Conference on Solid-State and Integrated Circuit Technology》;20101231;第1-4页 |
| Wu X等.SCMFS;A file system for storage class memory.《Proceedings of the International Conference on High Performance Computing》.2011,第1-2页. |
| 基于相变存储器的存储技术研究综述;冒伟等;《计算机学报》;20150531;第38卷(第5期);第944-960页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105335098A (en) | 2016-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105335098B (en) | A kind of log file system performance improvement method based on storage level memory | |
| US7856522B2 (en) | Flash-aware storage optimized for mobile and embedded DBMS on NAND flash memory | |
| US11194696B2 (en) | Recording a trace of code execution using reserved cache lines in a cache | |
| JP7334163B2 (en) | Logging Influx to Cache with Requests to Higher Level Cache | |
| CN105574104B (en) | A kind of LogStructure storage system and its method for writing data based on ObjectStore | |
| CN103136121B (en) | Cache management method for solid-state disc | |
| US10324851B2 (en) | Facilitating recording a trace file of code execution using way-locking in a set-associative processor cache | |
| CN106662981A (en) | Storage device, program, and information processing method | |
| US11756618B1 (en) | System and method for atomic persistence in storage class memory | |
| CN106503051B (en) | A kind of greediness based on meta data category prefetches type data recovery system and restoration methods | |
| CN104881371A (en) | Persistent internal memory transaction processing cache management method and device | |
| CN103631536B (en) | A kind of method utilizing the invalid data of SSD to optimize RAID5/6 write performance | |
| CN107025070B (en) | Versioned stores device and method | |
| WO2013174305A1 (en) | Ssd-based key-value type local storage method and system | |
| CN110018790B (en) | Method and system for ensuring data breakdown consistency in persistent memory | |
| CN105955664B (en) | A kind of reading/writing method of watt record conversion layer based on segment structure | |
| US20160196063A1 (en) | Apparatus and method for managing buffer having three states on the basis of flash memory | |
| CN102667720A (en) | Consistency without ordering dependency | |
| CN110309233A (en) | Method, apparatus, server and the storage medium of data storage | |
| CN105260139B (en) | A kind of disk management method and system | |
| KR20110046118A (en) | Adaptive logging apparatus and method | |
| CN108431784B (en) | Access request processing method and device and computer system | |
| CN109815168A (en) | System and method for marking buffer to realize less | |
| Chen et al. | A unified framework for designing high performance in-memory and hybrid memory file systems | |
| CN112395260B (en) | Data storage method and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |