CN105183868A - Storage method and device for logs of file system - Google Patents
Storage method and device for logs of file system Download PDFInfo
- Publication number
- CN105183868A CN105183868A CN201510587595.XA CN201510587595A CN105183868A CN 105183868 A CN105183868 A CN 105183868A CN 201510587595 A CN201510587595 A CN 201510587595A CN 105183868 A CN105183868 A CN 105183868A
- Authority
- CN
- China
- Prior art keywords
- file
- daily record
- file system
- journal
- memory device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Abstract
The invention discloses a storage method for logs of a file system. The method includes the steps that a log file is connected into the file system; the log file is mounted in log storage equipment, and the log storage equipment is independent from data storage equipment in the file system; the log file to be stored currently is placed in a cache of the log storage equipment, and metadata patches in the log file to be stored currently are aggregated; the log file to be stored currently is stored in the log storage equipment according to preset delay time. Log information is stored in the independent storage equipment, time wasted for a magnetic head to seek tracks in a reciprocating mode is shortened, more time is used for reading and writing data, and the working efficiency of a magnetic disk is improved. In addition, the delay storage technology is added, so that the frequency of brushing and writing the log storage equipment by a large number of metadata patches is reduced, and the performance of the log storage equipment is improved.
Description
Technical field
The present invention relates to technical field of memory, particularly relate to a kind of storage means and device of file system journal.
Background technology
Journaling File System is a file system with failover capability, in this file system, because to the lastest imformation of catalogue and bitmap always before original Disk Logs is updated, write in a continuous print daily record on disk, so the integrality of data can be ensured.When generation systems mistake, full Journaling File System will ensure the state of date restoring before generation systems collapse on disk.Meanwhile, it also will cover the data of not preserving, and if be there is computing machine and do not collapsed the position that these data may lose, this is the characteristic very important concerning Mission critical applications.
Journaling File System is the amendment (simultaneously also can be applied to emergency to recover) that usage log carrys out buffer file system, can take different strategies according to the time of record from content.Wherein, three kinds of common strategies are: write-back (writeback), predetermined (ordered) and data (data).
In write-back mode, only have metadata to be recorded to daily record, data block is then written directly on Disk Locality.Can file system structure be preserved like this, prevent collapse, but likely after metadata record to daily record, before data block write disk, system crash.Preassigned pattern only by metadata record to daily record, but write data into disk before this.So just can ensure the consistance of data and file system after System recover.In data pattern, data be also recorded in daily record, metadata and data are all recorded in daily record.This pattern can prevent file system crash and loss of data to greatest extent, but (first write daily record twice because total data is all written with, write disk again, be called that secondary writes punishment: writetwicepenalty), system performance can reduce widely.
In prior art, disk head seek can be caused to increase writing when data are write with a brush dipped in Chinese ink in daily record again, make real data access time shorter, performance is reduced; And for the application scenarios of fritter IO, the IO number that same quantity of data produces is multiplied, and system will be caused frequently to write with a brush dipped in Chinese ink daily record, add bandwidth pressure to file system.
Summary of the invention
The object of this invention is to provide a kind of storage means and device of file system journal, object is the performance improving existing file system.
For solving the problems of the technologies described above, the invention provides a kind of storage means of file system journal, comprising:
Journal file is connected in file system;
Described journal file is mounted to daily record memory device, and the data storage device in described daily record memory device and file system is separate;
Current journal file to be stored is put into the buffer memory of described daily record memory device, and the fritter metadata in described current journal file to be stored is polymerized;
According to predetermined time delay, described current journal file to be stored is stored on described daily record memory device.
Alternatively, the described buffer memory current journal file to be stored being put into described daily record memory device, and polymerization is carried out to the fritter metadata in described current journal file to be stored comprise:
When system generation power down, be that described buffer memory is powered by standby power supply.
The invention provides a kind of memory storage of file system journal, comprising:
Link block, for being connected in file system by journal file;
Carry module, for described journal file is mounted to daily record memory device, the data storage device in described daily record memory device and file system is separate;
Aggregation module, for current journal file to be stored being put into the buffer memory of described daily record memory device, and is polymerized the fritter metadata in described current journal file to be stored;
Memory module, for according to predetermined time delay, is stored into described current journal file to be stored on described daily record memory device.
Alternatively, described daily record memory device adopts independently netting twine to connect.
Alternatively, described daily record memory device is the memory device adopting SerialAttachedSCSI hard disk.
Alternatively, described buffer memory is random access memory.
Alternatively, described aggregation module comprises further:
Standby power supply, for when system generation power down, for described buffer memory provides standby power supply.
The storage means of file system journal provided by the present invention and device, by journal file is connected to file system, be mounted on corresponding daily record memory device, then current journal file to be stored is put into the buffer memory of daily record memory device, and fritter metadata is wherein polymerized, according to predetermined time delay, current journal file to be stored is stored on daily record memory device.The storage means of file system journal provided by the present invention and device, log information and data file memory location are independent, are left on independent memory device by log information, decrease the time of magnetic head tracking waste back and forth, more time is used for reading and writing data, and improves the work efficiency of disk.Add delay memory technology in addition, reduce the frequency that a large amount of fritter metadata writes with a brush dipped in Chinese ink daily record memory device, improve the performance of daily record memory device.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of a kind of embodiment of the storage means of file system journal provided by the present invention;
Fig. 2 is the process flow diagram of the another kind of embodiment of the storage means of file system journal provided by the present invention;
Fig. 3 is the structured flowchart of a kind of embodiment of the memory storage of file system journal provided by the present invention.
Embodiment
In order to make those skilled in the art person understand the present invention program better, below in conjunction with the drawings and specific embodiments, the present invention is described in further detail.Obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
As described in Figure 1, the method comprises the process flow diagram of a kind of embodiment of the storage means of file system journal provided by the present invention:
Step S101: journal file is connected in file system;
Step S102: described journal file is mounted to daily record memory device, the data storage device in described daily record memory device and file system is separate;
Step S103: the buffer memory current journal file to be stored being put into described daily record memory device, and the fritter metadata in described current journal file to be stored is polymerized;
Step S104: according to predetermined time delay, is stored into described current journal file to be stored on described daily record memory device.
The storage means of file system journal provided by the present invention, by journal file is connected to file system, be mounted on corresponding daily record memory device, then current journal file to be stored is put into the buffer memory of daily record memory device, and fritter metadata is wherein polymerized, according to predetermined time delay, current journal file to be stored is stored on daily record memory device.The storage means of file system journal provided by the present invention, log information and data file memory location are independent, are left on independent memory device by log information, decrease the time of magnetic head tracking waste back and forth, more time is used for reading and writing data, and improves the work efficiency of disk.Add delay memory technology in addition, reduce the frequency that a large amount of fritter metadata writes with a brush dipped in Chinese ink daily record memory device, improve the performance of daily record source equipment.
It is pointed out that in the present embodiment, log information leaves on independent memory device, this storage device performance must be consistent with the IO characteristic of daily record data.IO characteristic due to daily record is mainly order IO, and because daily record periodically records metadata, and it is less to take up room, so periodic cycle uses memory device, this just requires equipment wear resistant, the scope that SSD that is high as price and don't wear-resistant is not just considering.Here need, according to the demand of user oneself, to select voluntarily after assessed cost.
Memory device can specifically adopt 15,0002.5-inchSerialAttachedSCSI or 15,0003.5-inchSerialAttachedSCSI, can certainly adopt other equipment, and this does not all affect realization of the present invention.
Before the data rule of prior art file system, need to write with a brush dipped in Chinese ink in advance log information, data, daily record shared system bandwidth, data bandwidth can be caused can not to take the maximum bandwidth of system completely.And because the data storage device in daily record memory device and file system is separate in the embodiment of the present invention, therefore independent netting twine can be used to connect, this mode solves the problems referred to above, and log information writes with a brush dipped in Chinese ink the bandwidth no longer taking system, has maximum bandwidth safeguard when data are write with a brush dipped in Chinese ink.
The process flow diagram of the another kind of embodiment of the storage means of file system journal provided by the present invention as shown in Figure 2, compared with a upper embodiment, in the present embodiment when system generation power down, be that buffer memory is powered by reserve battery, the method comprises:
Step S201: journal file is connected in file system;
Step S202: described journal file is mounted to daily record memory device, the data storage device in described daily record memory device and file system is separate;
Step S203: the buffer memory current journal file to be stored being put into described daily record memory device, and the fritter metadata in described current journal file to be stored is polymerized;
Step S204: according to predetermined time delay, is stored on described daily record memory device by described current journal file to be stored;
Step S205: when system generation power down is that described buffer memory is powered by standby power supply.
Under normal circumstances, when system generation power down, because the data in the buffer memory that file system carries do not have enough time to write with a brush dipped in Chinese ink on physical disk, the loss of a part of data will be caused.When system is restarted again, due to volatibility during buffer memory, this part data is obviously permanent be lost.
Given this, the present embodiment increases by a block RAM as buffer memory on daily record memory device, for synchronization file system buffer memory, simultaneously also for this RAM provides standby power supply.When system generation power down, the standby power supply at this moment on daily record memory device enters duty immediately, and file system cache is not easily lost, and ensure that the consistance of data.
Standby power supply in the present embodiment has two states: (1) accumulate state, and power supply is also in open mode in a non-operative state, for charging to electric power incoming line; (2) duty, owing to being in open mode before power supply always, when system generation power down, RAM also has electricity supply always.So just can ensure that the buffer memory mirror image of file system can not be lost.There are metadata and data buffer storage, just can have done follow-up recovery operation.
Data file can cause the decline of performance when a large amount of fritter IO writes with a brush dipped in Chinese ink, this may be limited to IOPS bottleneck and the other factors of system.At this moment generally can by data-block cache, the IO being then about to issue to these merges and is polymerized, and technology, ZFSCOW mechanism are write with a brush dipped in Chinese ink in the delay as XFS.And the application is after the problem of power down cache miss is resolved, also just means and do not need to be eager log information to write with a brush dipped in Chinese ink at once on memory device.Therefore, said method is also applicable to the IO of log information.Because originally succession is write with a brush dipped in Chinese ink in daily record, lack the step of continuity check compared to the IO of data file, only needed simple IO to be polymerized and just many little IO can be merged into multiple large IO.Postpone writing with a brush dipped in Chinese ink of daily record, greatly can improve the performance of file system itself like this.
Visible, the storage means of the file system journal that the present embodiment provides, solves the problem of log information and data file contention file system bandwidth, the storage of daily record is given to the scheme of optimization from hardware; Also avoid the impact that tracking is write with a brush dipped in Chinese ink back and forth in daily record simultaneously; Also providing the foundation for postponing writing with a brush dipped in Chinese ink of daily record, eliminating the problem that log information little in a large number frequently writes disk.
The storage means of file system journal provided by the present invention, first needs data file, journal file to be connected in system; The memory device that wherein data file uses can be monolithic physical disk, also can be RAID, this does not affect realization of the present invention; Journal file memory device is independently daily record memory device; Then carry data file, journal file is distinguished to corresponding memory device; The IO of system is mainly divided into two classes: data file IO arrives on memory device through application, file system; Log information arrives daily record information storing device through file system; Logging device RAM synchronization file system buffer memory, when system generation power down, provides the mirror image needed for recovery; Simultaneously during system worked well, file realizes the converging operation to log information IO.
As described in Figure 3, this device comprises the structured flowchart of a kind of embodiment of the memory storage of file system journal provided by the present invention:
Link block 100, for being connected in file system by journal file;
Carry module 200, for described journal file is mounted to daily record memory device, the data storage device in described daily record memory device and file system is separate;
Aggregation module 300, for current journal file to be stored being put into the buffer memory of described daily record memory device, and is polymerized the fritter metadata in described current journal file to be stored;
Memory module 400, for according to predetermined time delay, is stored into described current journal file to be stored on described daily record memory device.
Particularly, the memory storage of file system journal provided by the present invention, daily record memory device can adopt independently netting twine to connect.Daily record memory device can be specially the memory device adopting SerialAttachedSCSI hard disk.Buffer memory can be specially random access memory.
As a kind of preferred implementation, the memory storage of file system journal provided by the present invention can also be provided with further:
Standby power supply, for when system generation power down, for described buffer memory provides standby power supply.
The memory storage of file system journal provided by the present invention is corresponding with said method, does not repeat them here.
To sum up, the present invention is mainly for the log storing method Problems existing of existing Journal File System, and propose a kind of storage means and device of file system journal, it has following advantage:
(1) log information IO no longer takies system bandwidth, thus improves file system performance;
(2) log information is stored on the memory device of separately design, the implementation of file system before comparing, and decrease the time of magnetic head tracking waste back and forth, the more time is used for reading and writing data, and improves the efficiency of disk work;
(3) daily record memory device is provided with RAM and standby power supply, for Journaling File System power down count off, data consistency, fritter metadata are frequently write with a brush dipped in Chinese ink daily record and provided solution.
In this instructions, each embodiment adopts the mode of going forward one by one to describe, and what each embodiment stressed is the difference with other embodiment, between each embodiment same or similar part mutually see.
To the above-mentioned explanation of the disclosed embodiments, professional and technical personnel in the field are realized or uses the present invention.To be apparent for those skilled in the art to the multiple amendment of these embodiments, General Principle as defined herein can without departing from the spirit or scope of the present invention, realize in other embodiments.Therefore, the present invention can not be restricted to these embodiments shown in this article, but will meet the widest scope consistent with principle disclosed herein and features of novelty.
Claims (7)
1. a storage means for file system journal, is characterized in that, comprising:
Journal file is connected in file system;
Described journal file is mounted to daily record memory device, and the data storage device in described daily record memory device and file system is separate;
Current journal file to be stored is put into the buffer memory of described daily record memory device, and the fritter metadata in described current journal file to be stored is polymerized;
According to predetermined time delay, described current journal file to be stored is stored on described daily record memory device.
2. the storage means of file system journal as claimed in claim 1, it is characterized in that, the described buffer memory current journal file to be stored being put into described daily record memory device, and polymerization is carried out to the fritter metadata in described current journal file to be stored comprise:
When system generation power down, be that described buffer memory is powered by standby power supply.
3. a memory storage for file system journal, is characterized in that, comprising:
Link block, for being connected in file system by journal file;
Carry module, for described journal file is mounted to daily record memory device, the data storage device in described daily record memory device and file system is separate;
Aggregation module, for current journal file to be stored being put into the buffer memory of described daily record memory device, and is polymerized the fritter metadata in described current journal file to be stored;
Memory module, for according to predetermined time delay, is stored into described current journal file to be stored on described daily record memory device.
4. the memory storage of file system journal as claimed in claim 3, is characterized in that, described daily record memory device adopts independently netting twine to connect.
5. the memory storage of file system journal as claimed in claim 4, is characterized in that, described daily record memory device is the memory device adopting SerialAttachedSCSI hard disk.
6. the memory storage of file system journal as claimed in claim 3, it is characterized in that, described buffer memory is random access memory.
7. the memory storage of the file system journal as described in any one of claim 3 to 6, is characterized in that, described aggregation module comprises further:
Standby power supply, for when system generation power down, for described buffer memory provides standby power supply.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510587595.XA CN105183868A (en) | 2015-09-15 | 2015-09-15 | Storage method and device for logs of file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510587595.XA CN105183868A (en) | 2015-09-15 | 2015-09-15 | Storage method and device for logs of file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105183868A true CN105183868A (en) | 2015-12-23 |
Family
ID=54905949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510587595.XA Pending CN105183868A (en) | 2015-09-15 | 2015-09-15 | Storage method and device for logs of file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105183868A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677258A (en) * | 2016-02-23 | 2016-06-15 | 浪潮(北京)电子信息产业有限公司 | Method and system for managing log data |
| CN107256273A (en) * | 2017-06-28 | 2017-10-17 | 联想(北京)有限公司 | The caching method and system of a kind of file system journal |
| CN107733667A (en) * | 2016-08-10 | 2018-02-23 | 北京京东尚科信息技术有限公司 | A kind of blog management method and system |
| CN108920303A (en) * | 2018-06-08 | 2018-11-30 | 山东超越数控电子股份有限公司 | A kind of security-enhanced log file system |
| CN109522273A (en) * | 2018-11-15 | 2019-03-26 | 郑州云海信息技术有限公司 | A kind of method and device for realizing data write-in |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101751307A (en) * | 2008-12-12 | 2010-06-23 | 比兹肯解决方法有限公司 | Data storage system, method as well as data storage and backup system |
| CN102024021A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Method for logging metadata in logical file system |
| CN102981934A (en) * | 2012-12-21 | 2013-03-20 | 曙光信息产业(北京)有限公司 | Log transition method and log transition device |
| CN103377069A (en) * | 2012-04-26 | 2013-10-30 | 东方有线网络有限公司 | Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) |
| CN103559119A (en) * | 2013-10-25 | 2014-02-05 | 华为技术有限公司 | File operation request processing method and device |
| CN104750598A (en) * | 2013-12-26 | 2015-07-01 | 南京南瑞继保电气有限公司 | A storage method for IEC61850 log service |
-
2015
- 2015-09-15 CN CN201510587595.XA patent/CN105183868A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101751307A (en) * | 2008-12-12 | 2010-06-23 | 比兹肯解决方法有限公司 | Data storage system, method as well as data storage and backup system |
| CN102024021A (en) * | 2010-11-04 | 2011-04-20 | 曙光信息产业(北京)有限公司 | Method for logging metadata in logical file system |
| CN103377069A (en) * | 2012-04-26 | 2013-10-30 | 东方有线网络有限公司 | Novel method for achieving virtual calculation deployment based on writing of log files into plug-in network attached storage (NAS) |
| CN102981934A (en) * | 2012-12-21 | 2013-03-20 | 曙光信息产业(北京)有限公司 | Log transition method and log transition device |
| CN103559119A (en) * | 2013-10-25 | 2014-02-05 | 华为技术有限公司 | File operation request processing method and device |
| CN104750598A (en) * | 2013-12-26 | 2015-07-01 | 南京南瑞继保电气有限公司 | A storage method for IEC61850 log service |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105677258A (en) * | 2016-02-23 | 2016-06-15 | 浪潮(北京)电子信息产业有限公司 | Method and system for managing log data |
| CN107733667A (en) * | 2016-08-10 | 2018-02-23 | 北京京东尚科信息技术有限公司 | A kind of blog management method and system |
| CN107733667B (en) * | 2016-08-10 | 2021-05-25 | 北京京东尚科信息技术有限公司 | Log management method and system |
| CN107256273A (en) * | 2017-06-28 | 2017-10-17 | 联想(北京)有限公司 | The caching method and system of a kind of file system journal |
| CN108920303A (en) * | 2018-06-08 | 2018-11-30 | 山东超越数控电子股份有限公司 | A kind of security-enhanced log file system |
| CN109522273A (en) * | 2018-11-15 | 2019-03-26 | 郑州云海信息技术有限公司 | A kind of method and device for realizing data write-in |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105183868A (en) | Storage method and device for logs of file system | |
| CN103150258B (en) | Writing, reading and garbage collection method of solid-state memory system | |
| CN103500130B (en) | A kind of method that two-node cluster hot backup data are backed up in realtime | |
| CN102053802B (en) | Network RAID (redundant array of independent disk) system | |
| CN103631536B (en) | A kind of method utilizing the invalid data of SSD to optimize RAID5/6 write performance | |
| CN109791520A (en) | The log recording of the Space Coupling of physical medium perception and playback | |
| CN105138276B (en) | Data storage method and data storage system | |
| CN103049070A (en) | Cache data power failure protection method and computer equipment | |
| CN103455577A (en) | Multi-backup nearby storage and reading method and system of cloud host mirror image file | |
| CN102222085A (en) | Data de-duplication method based on combination of similarity and locality | |
| CN103152395A (en) | Storage method and device of distributed file system | |
| CN104407933A (en) | Data backup method and device | |
| CN102024044A (en) | Distributed file system | |
| CN101567211A (en) | Method for improving usability of disk and disk array controller | |
| CN107357680A (en) | Data processing equipment and the method for data processing | |
| CN102411639A (en) | Multi-copy storage management method and system of metadata | |
| CN102158349A (en) | Log management device and method thereof | |
| CN103221925A (en) | Data processing method and storage equipment | |
| CN105512056A (en) | Method and device for data storage, and terminal | |
| CN101625586A (en) | Method, equipment and computer for managing energy conservation of storage device | |
| CN103916459A (en) | Big data filing and storing system | |
| WO2015058542A1 (en) | Reconstruction method and device for redundant array of independent disks | |
| CN102508727A (en) | Method using software for power fail safeguard of caches in disk array | |
| CN103092765B (en) | Solid-state memory system, device and method for writing data | |
| CN110196818A (en) | Data cached method, buffer memory device and storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151223 |