CN104636452A - Unified file system managing method oriented to SCM - Google Patents
Unified file system managing method oriented to SCM Download PDFInfo
- Publication number
- CN104636452A CN104636452A CN201510044083.9A CN201510044083A CN104636452A CN 104636452 A CN104636452 A CN 104636452A CN 201510044083 A CN201510044083 A CN 201510044083A CN 104636452 A CN104636452 A CN 104636452A
- Authority
- CN
- China
- Prior art keywords
- file
- memory device
- data
- nvm memory
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses a unified file system managing method oriented to SCM. The method includes the steps of firstly, reasonably decomposing metadata according to the characteristics of NVM storage devices of different types through the metadata decomposing and managing method oriented to the characteristics of the NVM storage devices of different types, distributing the metadata in the NVM storage devices of different types to be stored and managed, and establishing a collaborative metadata managing sub-system; secondly, designing a novel file data organizing mode through an integrated data unified managing method oriented to SCM, and decomposing and classifying files so that file data can be reasonably distributed in the NVM storage devices of different types. By means of the method, the NVM storage devices of different types in an SCM system can be managed in a unified mode, and the metadata and data management efficiency and access performance are effectively improved while the storage space of the storage devices of different types is fully utilized.
Description
Technical field
The invention belongs to technical field of memory, be specifically related to the art file management technology of a kind of SCM.Be mainly used in solving in the SCM comprising multiple DIMM and PCI-E interface NVM memory device, how unify the problem of storage space, efficiently management and the access file utilizing dissimilar memory device.
Background technology
Along with the use of the processor technology such as multinuclear, multithreading, the gap between the computing power improved fast in computer system and the performance of storage system increasesd slowly is more and more significant, defines serious storage wall problem.Novel non-volatile memory device (NVM) has higher read or write speed and integrated level and lower power consumption and non-volatile, haves laid a good foundation for improving performance of storage system and reducing energy consumption.Novel NVM device such as MRAM etc. is writing in life-span, read or write speed etc. close to DRAM, the index of the aspects such as integrated level, capacity, reliability and energy consumption is also in improving constantly, and therefore the limitation of NVM device aspect has not been that it is applied to the major obstacle in computer system.
Storage level memory techniques (SCM) is for after employing NVM, the characteristic that memory device performance gap at different levels is less, by the internal memory in unified management computer system and external memory, the high speed unified storage system built in computer systems, which, for solving the storage wall problem of computer system.But there is huge difference between internal memory and external memory; In access mode, internal memory supports the access of byte mode, and external memory can only be accessed in the mode of block; In read or write speed, external memory is far below internal memory; In data retention over time, the data in EMS memory based on DRAM is lost after a power failure, then can preserve for a long time when not powering up based on the data in the external memory of Flash and disk; These differences make existing interior external memory management algorithm completely different, bring huge challenge to the unified storage management system of structure.
Current research person is mainly from how using the performance of NVM raising storage system and SCM storage administration mechanism two aspects to expand research.Have researcher to construct the storage array Onyx of a Based PC M, experiment shows that its performance hyper-base far away is in the SSD of Flash.There is researcher to devise FSMAC, utilize NVM storing metadata, improve the performance of file system.There is researcher to use NVM to construct buffer memory and the buffering of file system, achieve the journal function of file system simultaneously.Researcher is had to pass through to think that PCM is preferably used for building the buffer memory of layer-stepping storage system or storage system.There is researcher to use NVM to build and write buffer memory, improve the SSD performance based on Flash.Have researcher to propose the mixing view FRASH of SCM, it is made up of the data structure being positioned at internal memory and being positioned at disk, for the critical data structure being positioned at internal memory in log structured file system provides non-volatile, significantly can reduce delay, and improve robustness.When having researcher to replace hard disk for use SCM, the block interface of existence can not provide the problem of enough information to memory device, the object-based SCMs of Design and implementation.There is researcher to propose " atomic write " primitive, multiple I/O is operated composition logical groups, in memory device, realizes the atomicity of write operation, reduce the expense of operating system and upper layer application guarantee data consistency and integrality.Do not consider the addressing of SCM byte and non-volatile feature for existing file system, have researcher to utilize memory management module in operating system, achieve the file system SCMFS based on virtual address space.There is researcher for SCM, propose a kind of file system BPFS, use short circuit shadow paging (short-circuit shadow paging) technology to realize atom and fine-grained renewal, improve reliability and the performance of file system.There is researcher to adopt super page to preserve the internal management information of DIMM interface NVM memory device, ensure the reliability of storage administration, and adopt the page of different size to manage small documents and large files respectively, improve the performance of file system.There is researcher to devise Shortcut-JFS Journaling File System for PCM, by only recording difference-byte and establishment checkpoint, original place, reducing write operation.
Currently NVM is applied in the research building SCM, main or for the performance that improves existing storage system and these two aspects of Novel document system building byte-oriented access characteristics.But due to integrated level and the quantitative limitation of DIMM number, DIMM and PCI-E interface NVM memory device must be there is and the situation of depositing during the Large Copacity SCM using NVM memory device to build, all greatest differences is there is in DIMM interface and PCI-E interface in access mode and speed, existing internal memory and external memory operating strategy, cannot the memory devices of unified management different interface type and access mode all just for byte or block access mode.The access mode of different file is not identical yet, and one comprises the access modes such as Stochastic sum order; The access characteristics of different file also there are differences, and very little part only accessed by some files, and what have then needs travel all over file; Also there is very big-difference in the accessed frequency of metadata and data simultaneously; Therefore single file management strategy is difficult to the requirement adapting to the management of SCM file.
Summary of the invention
The object of the invention is to design a kind of unified file system towards SCM, solve in the SCM comprising multiple DIMM and PCI-E interface NVM memory device, how to unify the problem of storage space, efficiently management and the access file utilizing dissimilar memory device.
Technical scheme of the present invention is:
Towards a unified file system management method of SCM, comprise the following steps:
Use and decompose and management method towards NVM memory device characteristic metadata, after metadata rationally being decomposed according to the characteristic of dissimilar NVM memory device, be distributed in multiple dissimilar NVM memory device and carry out storage and management, build collaboration type metadata management subsystem; By towards SCM fusion type data Explore of Unified Management Ideas, design novel file data organizational form, file is decomposed and classifies, thus by file data reasonable layout and multiple dissimilar NVM memory device.
Further, the described decomposition of the metadata towards NVM memory device characteristic comprises the following steps: with management method
Step 1.1, for the difference of DIMM and PCI-E interface NVM memory device characteristic, the metadata of file, to the metadata decomposition method of NVM memory device characteristic, is decomposed into base attribute and extended attribute two class by design surface;
Step 1.2, for the characteristic of base attribute, is kept in DIMM interface NVM memory device by base attribute, design centralized configuration base attribute management method;
Step 1.3, for the characteristic of extended attribute, is distributed in multiple PCI-E interface NVM memory device by extended attribute, design collaboration type delamination attribute management method.
Further, comprise the following steps: towards NVM memory device characteristic metadata decomposition method in described step 1.1
Step 2.1, extraction document internal indicator, file access path and access rights from file metadata, and increase station location marker, together as the base attribute of file, the value of station location marker be 0 expression file data all in DIMM interface NVM memory device, the value of station location marker be 1 expression in PCI-E interface NVM memory device also storage unit divide file data;
Step 2.2, identifies the extended attribute as file using remaining attribute in file metadata and file internal.
Further, in described step 1.2, centralized configuration base attribute management method comprises the following steps:
Step 3.1, the base attribute for each file sets up the identical structure of a structure, builds the base attribute block of file;
Step 3.2, adopts chained list to couple together the base attribute block of file, builds base attribute block chained list;
Step 3.3, to base attribute block chained list, is designated key word with file internal, sets up base attribute concordance list.
Step 3.4, is kept in DIMM interface NVM memory device by the base attribute of All Files;
Step 3.5, on the basis of base attribute block chained list, sets up and safeguards the directory tree of SCM file system, provides the hierarchical directory tree file access and way to manage that meet user's use habit.
Further, in described step 1.3, collaboration type delamination attribute management method comprises the following steps:
Step 4.1 is the structure that each file set up structure is identical, preserves file internal mark, extended attribute quantity and extended attribute address, builds the extended attribute block of file;
Step 4.2, couples together the extended attribute block in PCI-E interface NVM memory device, in each PCI-E interface NVM memory device, build an extended attribute one-level chained list;
Step 4.3, then be that in file, each extended attribute builds the identical structure of a structure, preserve extended attribute title, the value of extended attribute and the address of the next extended attribute of this file, build the extended attribute value block of file;
Step 4.4, all expansion property value block chained lists of file are coupled together, build the extended attribute secondary chained list belonging to each file oneself, and by the extended attribute address of corresponding document extended attribute block in the first address of chained list write extended attribute one-level chained list.
Step 4.5, by file extent property distribution and multiple PCI-E interface NVM memory device, the extended attribute of stored file is in charge of by each PCI-E interface NVM memory device;
Step 4.6, when searching with administration extensions attribute, central processing unit as requested, builds the NVM memory device that instruction sends to PCI-E interface;
Step 4.7, PCI-E interface NVM memory device utilizes flush bonding processor, first according to file internal mark, searches extended attribute one-level chained list, obtains the first address of extended attribute secondary chained list;
Step 4.8, then, PCI-E interface NVM memory device utilizes flush bonding processor, according to metadata attributes name, searches extended attribute secondary chained list, finds respective extension property value block, complete corresponding operating, and by result feedback to central processing unit;
Step 4.9, central processing unit, after receiving PCI-E interface NVM memory device feedack, arranges, and feeds back to user.
Further, describedly specifically to comprise the following steps: towards SCM fusion type data Explore of Unified Management Ideas
Step 5.1, designs novel compound files data organization method, by file data reasonable layout and multiple DIMM and PCI-E interface NVM memory device;
Step 5.2, for the distribution of file data in multiple DIMM and PCI-E interface NVM memory device, design document data unify wiring method;
Step 5.3, for the distribution of file data in multiple DIMM and PCI-E interface NVM memory device, design document collaboration data read method;
Step 5.4, for the distribution of file data in multiple DIMM and PCI-E interface NVM memory device, design fusion type file data amending method.
Further, in described step 5.1, compound files data organization method specifically comprises the following steps:
Step 6.1, each file comprises byte and block subregion two logical data subregions;
Step 6.2, the data block that block subregion is Sizep by a series of size forms, and is kept in PCI-E interface NVM memory device;
Step 6.3, byte partitions size is Sizep+1, is kept in DIMM interface NVM memory device.
Further, described step 5.2 file data are unified wiring method and are specifically comprised the following steps:
Step 7.1, when storing new file, first checking the size (Size represents file data size) of new file, if file size is less than Sizep, then forwarding step 7.2 to, otherwise forward step 7.3 to;
Step 7.2, by the byte partitions in the data of new file write DIMM interface NVM memory device, and is set to 0 by the value of station location marker in metadata base attribute, last ends file data writing process;
Step 7.3, the value of station location marker in metadata base attribute is set to 1, and first character joint in byte partitions is set to 0, then from DIMM interface NVM memory device byte starting position, second, byte partitions, preserving top of file size is the data of Size%Sizep;
Remaining data are write in the block subregion in PCI-E interface NVM memory device, last ends file data writing process by step 7.4 in units of Sizep.
Further, described step 5.3 file collaboration data read method specifically comprises the following steps:
Step 8.1, uses file access path, the base attribute of locating file metadata, obtains the value of station location marker, as forwarded step 8.2 to for 0, otherwise forwards step 8.3 to;
Step 8.2, from the byte partitions file reading data being arranged in DIMM interface NVM memory device, by data feedback to user and ends file data read process;
Step 8.3, file reading data slot A from the block subregion of PCI-E interface NVM memory device, then from DIMM interface NVM memory device file reading data slot B the byte of second, byte partitions;
Step 8.4, from DIMM interface NVM memory device, byte partitions first character joint, reads the address information of file data fragments B, appropriate address data in file data fragments B and file data fragments A is exchanged;
Step 8.5, couples together file data fragments B and file data fragments A, obtains the data that file is complete, feeds back to user, and ends file data read process.
Further, in described step 5.4, fusion type file data amending method specifically comprises the following steps:
Step 9.1, uses file access path, the base attribute of locating file metadata, obtains the value of station location marker, as forwarded step 9.2 to for 0, otherwise forwards step 9.3 to;
Step 9.2, revises file data in the byte partitions being arranged in DIMM interface NVM memory device, by result feedback to user and ends file data modification process;
Step 9.3, byte partitions first character joint from DIMM interface NVM memory device, the address information of data slot in sense byte subregion, compares with needing modified address, if need modified address in byte partitions, perform step 9.4, otherwise perform step 9.5;
Step 9.4, according to needing modified address, the data of byte partitions in amendment DIMM interface NVM memory device, and ends file data modification process;
Step 9.5, according to needing modified address, the data of block subregion in amendment PCI-E interface NVM memory device, and ends file data modification process.
The present invention has beneficial effect.The present invention is directed to the SCM systems of multiple DIMM and the mixing of PCI-E interface NVM memory device, the difference in functionality of foundation DIMM and PCI-E interface NVM memory device and operating characteristic, build novel metadata and Data distribution8 Organization And Management technology, devise towards NVM memory device characteristic metadata decomposition method, centralized configuration base attribute management method, collaboration type delamination attribute management method, compound files data organization method, file data unifies wiring method, file data cooperation read method and fusion type file data amending method, the NVM memory device that Appropriate application is dissimilar, cooperate with each other, complement one another, make full use of the storage space of dissimilar memory device, improve the access performance of metadata and data management efficiency.
Accompanying drawing explanation
Fig. 1 is the structural drawing towards SCM unified file system;
Fig. 2 is centralized configuration base attribute organizational form figure;
Fig. 3 is collaboration type delamination attribute organizational form figure;
Tu4Shi compound files data organizational structure schemes;
Fig. 5 is the test result figure reading performance;
Fig. 6 is the test result figure of write performance;
Fig. 7 reads the test result figure of performance when being and changing byte partitions size;
Fig. 8 is the test result figure of write performance when changing byte partitions size;
Fig. 9 reads the test result figure of performance when being and changing read-write block size;
Figure 10 is the test result figure of write performance when changing read-write block size.
Embodiment
The specific embodiment of the present invention is further illustrated below in conjunction with accompanying drawing.
A kind of unified file system towards SCM of method application of the present invention, comprise the NVM memory device of multiple DIMM and PCI-E interface, the multiple dissimilar NVM memory device of file system energy unified management, according to the characteristic rationalization of NVM memory device and the data of management document and metadata, make full use of the storage space of dissimilar memory device, improve the performance of file system.
The design uses by decomposing with management method towards NVM memory device characteristic metadata and carrying out work towards SCM fusion type data Explore of Unified Management Ideas.
Decompose towards NVM memory device characteristic metadata and comprise towards NVM memory device characteristic metadata decomposition method, centralized configuration base attribute management method and collaboration type delamination attribute management method with management method.
Described towards NVM memory device characteristic metadata decomposition method, comprise the following steps:
Step 1.1, extraction document internal indicator, file access path and access rights from file metadata, and increase station location marker, together as the base attribute of file, the value of station location marker be 0 expression file data all in DIMM interface NVM memory device, the value of station location marker be 1 expression in PCI-E interface NVM memory device also storage unit divide file data;
Step 1.2, identifies the extended attribute as file using remaining attribute in file metadata and file internal.
Described centralized configuration base attribute management method, comprises the following steps:
Step 2.1, the base attribute for each file sets up the identical structure of a structure, builds the base attribute block of file;
Step 2.2, adopts chained list to couple together the base attribute block of file, builds base attribute block chained list;
Step 2.3, to base attribute block chained list, is designated key word with file internal, sets up base attribute concordance list.
Step 2.4, is kept in DIMM interface NVM memory device by the base attribute of All Files;
Step 2.5, on the basis of base attribute block chained list, sets up and safeguards the directory tree of SCM file system, provides the hierarchical directory tree file access and way to manage that meet user's use habit.
Described collaboration type delamination attribute management method, comprises the following steps:
Step 3.1 is the structure that each file set up structure is identical, preserves file internal mark, extended attribute quantity and extended attribute address, builds the extended attribute block of file;
Step 3.2, couples together the extended attribute block in PCI-E interface NVM memory device, in each PCI-E interface NVM memory device, build an extended attribute one-level chained list;
Step 3.3, then be that in file, each extended attribute builds the identical structure of a structure, preserve extended attribute title, the value of extended attribute and the address of the next extended attribute of this file, build the extended attribute value block of file;
Step 3.4, all expansion property value block chained lists of file are coupled together, build the extended attribute secondary chained list belonging to each file oneself, and by the extended attribute address of corresponding document extended attribute block in the first address of chained list write extended attribute one-level chained list.
Step 3.5, by file extent property distribution and multiple PCI-E interface NVM memory device, the extended attribute of stored file is in charge of by each PCI-E interface NVM memory device;
Step 3.6, when searching with administration extensions attribute, central processing unit as requested, builds the NVM memory device that instruction sends to PCI-E interface;
Step 3.7, PCI-E interface NVM memory device utilizes flush bonding processor, first according to file internal mark, searches extended attribute one-level chained list, obtains the first address of extended attribute secondary chained list;
Step 3.8, then, PCI-E interface NVM memory device utilizes flush bonding processor, according to metadata attributes name, searches extended attribute secondary chained list, finds respective extension property value block, complete corresponding operating, and by result feedback to central processing unit;
Step 3.9, central processing unit, after receiving PCI-E interface NVM memory device feedack, arranges, and feeds back to user.
Comprise compound files data organization method towards SCM fusion type data Explore of Unified Management Ideas, file data unifies wiring method, file data cooperates read method and fusion type file data amending method.
Described compound files data organization method, specifically comprises the following steps:
Step 4.1, each file comprises byte and block subregion two logical data subregions;
Step 4.2, the data block that block subregion is Sizep by a series of size forms, and is kept in PCI-E interface NVM memory device;
Step 4.3, byte partitions size is Sizep+1, is kept in DIMM interface NVM memory device.
Described file data unifies wiring method, specifically comprises the following steps:
Step 5.1, when storing new file, first checking the size (Size represents file data size) of new file, if file size is less than Sizep, then forwarding step 5.2 to, otherwise forward step 5.3 to; ;
Step 5.2, by the byte partitions in the data of new file write DIMM interface NVM memory device, and is set to 0 by the value of station location marker in metadata base attribute, last ends file data writing process;
Step 5.3, the value of station location marker in metadata base attribute is set to 1, and first character joint in byte partitions is set to 0, then from DIMM interface NVM memory device byte starting position, second, byte partitions, preserving top of file size is the data of Size%Sizep;
Remaining data are write in the block subregion in PCI-E interface NVM memory device, last ends file data writing process by step 5.4 in units of Sizep.
Described file data cooperation read method, specifically comprises the following steps:
Step 6.1, uses file access path, the base attribute of locating file metadata, obtains the value of station location marker, as forwarded step 6.2 to for 0, otherwise forwards step 6.3 to;
Step 6.2, from the byte partitions file reading data being arranged in DIMM interface NVM memory device, by data feedback to user and ends file data read process;
Step 6.3, file reading data slot A from the block subregion of PCI-E interface NVM memory device, then from second, byte partitions byte from DIMM interface NVM memory device file reading data slot B;
Step 6.4, from DIMM interface NVM memory device, byte partitions first character joint, reads the address information of file data fragments B, appropriate address data in file data fragments B and file data fragments A is exchanged;
Step 6.5, couples together file data fragments B and file data fragments A, obtains the data that file is complete, feeds back to user, and ends file data read process.
Described fusion type file data amending method, specifically comprises the following steps:
Step 7.1, uses file access path, the base attribute of locating file metadata, obtains the value of station location marker, as forwarded step 7.2 to for 0, otherwise forwards step 7.3 to;
Step 7.2, revises file data in the byte partitions being arranged in DIMM interface NVM memory device, by result feedback to user and ends file data modification process;
Step 7.3, byte partitions first character joint from DIMM interface NVM memory device, the address information of data slot in sense byte subregion, compares with needing modified address, if need modified address in byte partitions, perform step 7.4, otherwise perform step 7.5;
Step 7.4, according to needing modified address, the data of byte partitions in amendment DIMM interface NVM memory device, and ends file data modification process;
Step 7.5, according to needing modified address, the data of block subregion in amendment PCI-E interface NVM memory device, and ends file data modification process.
Embodiment 1
As shown in Figure 1, a kind of unified file system towards SCM comprises DIMM interface NVM memory device, PCI-E interface NVM memory device, decomposes with administration module with towards SCM fusion type data uniform management module towards NVM memory device characteristic metadata.The explanation of each device is as shown in table 1.
One kind, table 1 is towards each device in the unified file of SCM
Embodiment 2
The current ripe NVM memory device also not having directly to use, we use internal memory to simulate NVM memory device, and by FUSE User space file system, realize the prototype towards SCM unified file system, concrete steps are as follows:
1, first use FUSE to set up User space file system, receive user to the request of access of file.
2, the mode provided in Fig. 2 and Fig. 3 is used, the base attribute of constituent act metadata and extended attribute.When store and management file metadata, use ramfs analog D IMM interface NVM memory device, by the basic meta-data preservation of file in ramfs, for locating file and managing file system directory tree, realize centralized configuration base attribute management algorithm; Re-use ramdisk and simulate PCI-E interface NVM memory device, ext3 is used to format ramdisk, be as the filename in ramdisk after character string using file access path integration, build the bibliographic structure of individual layer, simulation collaboration type delamination attribute organization and management.
3, the mode in Fig. 4 is used, the data of constituent act.When store and management file data, in order to testing and analyzing more flexibly, byte partitions size Sizeb (Sizeb≤Sizep+1) is also set to customized parameter; And top of file is no more than Sizeb size or according to the residue file data after Sizep size piecemeal stored in ramfs, remainder data is stored in ramdisk.
Run in a computing machine towards SCM unified file system prototype, be furnished with Intel (R) Xeon (R) E5606@2.13Ghz processor and 14G internal memory, use red hat enterprise linux 6.1 as operating system, linux kernel version is the version of 2.6.32, FUSE is 2.8.3.
Use the standard testing instrument of file system I/O performance and linux kernel source code to carry out testing and analyzing, comprise following several aspect:
1, use postmark as testing tool, respectively test when file stored in DIMM interface NVM memory device, stored in PCI-E interface NVM memory device and employing towards I/O performance during SCM unified file prototype, and compare and analyze; Carry out identical testing and analyzing using linux kernel source code as data set more simultaneously
2, change the size of byte partitions, use postmark, test surfaces, to the I/O performance of SCM unified file prototype, analyzes byte partitions size to the impact towards SCM unified file prototype performance.
3, change the size of file system read-write block, use postmark, test surfaces is to the I/O performance of SCM unified file prototype, and Study document system read-write block size is on the impact towards SCM unified file prototype performance.
Fig. 5,6,7,8,9 and 10 carries out test and the result figure compared.
Foregoing is only specific embodiments of the invention, the protection domain be not intended to limit the present invention; Should be understood that under the prerequisite without prejudice to spirit of the present invention and flesh and blood, any improvement of doing, equivalent to replace and modification etc. all falls within protection scope of the present invention.
Claims (10)
1., towards a unified file system management method of SCM, it is characterized in that, comprise the following steps:
Use and decompose and management method towards NVM memory device characteristic metadata, after metadata rationally being decomposed according to the characteristic of dissimilar NVM memory device, be distributed in multiple dissimilar NVM memory device and carry out storage and management, build collaboration type metadata management subsystem;
By towards SCM fusion type data Explore of Unified Management Ideas, design novel file data organizational form, file is decomposed and classifies, thus by file data reasonable layout and multiple dissimilar NVM memory device.
2. the unified file system management method towards SCM according to claim 1, is characterized in that, the described metadata towards NVM memory device characteristic is decomposed and management method comprises the following steps:
Step 1.1, for the difference of DIMM and PCI-E interface NVM memory device characteristic, the metadata of file, to the metadata decomposition method of NVM memory device characteristic, is decomposed into base attribute and extended attribute two class by design surface;
Step 1.2, for the characteristic of base attribute, is kept in DIMM interface NVM memory device by base attribute, design centralized configuration base attribute management method;
Step 1.3, for the characteristic of extended attribute, is distributed in multiple PCI-E interface NVM memory device by extended attribute, design collaboration type delamination attribute management method.
3. the unified file system management method towards SCM according to claim 2, is characterized in that, comprises the following steps: in described step 1.1 towards NVM memory device characteristic metadata decomposition method
Step 2.1, extraction document internal indicator, file access path and access rights from file metadata, and increase station location marker, together as the base attribute of file, the value of station location marker be 0 expression file data all in DIMM interface NVM memory device, the value of station location marker be 1 expression in PCI-E interface NVM memory device also storage unit divide file data;
Step 2.2, identifies the extended attribute as file using remaining attribute in file metadata and file internal.
4. the unified file system management method towards SCM according to claim 2, is characterized in that, in described step 1.2, centralized configuration base attribute management method comprises the following steps:
Step 3.1, the base attribute for each file sets up the identical structure of a structure, builds the base attribute block of file;
Step 3.2, adopts chained list to couple together the base attribute block of file, builds base attribute block chained list;
Step 3.3, to base attribute block chained list, is designated key word with file internal, sets up base attribute concordance list.
Step 3.4, is kept in DIMM interface NVM memory device by the base attribute of All Files;
Step 3.5, on the basis of base attribute block chained list, sets up and safeguards the directory tree of SCM file system, provides the hierarchical directory tree file access and way to manage that meet user's use habit.
5. the unified file system management method towards SCM according to claim 2, is characterized in that, in described step 1.3, collaboration type delamination attribute management method comprises the following steps:
Step 4.1 is the structure that each file set up structure is identical, preserves file internal mark, extended attribute quantity and extended attribute address, builds the extended attribute block of file;
Step 4.2, couples together the extended attribute block in PCI-E interface NVM memory device, in each PCI-E interface NVM memory device, build an extended attribute one-level chained list;
Step 4.3, then be that in file, each extended attribute builds the identical structure of a structure, preserve extended attribute title, the value of extended attribute and the address of the next extended attribute of this file, build the extended attribute value block of file;
Step 4.4, all expansion property value block chained lists of file are coupled together, build the extended attribute secondary chained list belonging to each file oneself, and by the extended attribute address of corresponding document extended attribute block in the first address of chained list write extended attribute one-level chained list.
Step 4.5, by file extent property distribution and multiple PCI-E interface NVM memory device, the extended attribute of stored file is in charge of by each PCI-E interface NVM memory device;
Step 4.6, when searching with administration extensions attribute, central processing unit as requested, builds the NVM memory device that instruction sends to PCI-E interface;
Step 4.7, PCI-E interface NVM memory device utilizes flush bonding processor, first according to file internal mark, searches extended attribute one-level chained list, obtains the first address of extended attribute secondary chained list;
Step 4.8, then, PCI-E interface NVM memory device utilizes flush bonding processor, according to metadata attributes name, searches extended attribute secondary chained list, finds respective extension property value block, complete corresponding operating, and by result feedback to central processing unit;
Step 4.9, central processing unit, after receiving PCI-E interface NVM memory device feedack, arranges, and feeds back to user.
6. the unified file system management method towards SCM according to claim 1, is characterized in that, describedly specifically comprises the following steps: towards SCM fusion type data Explore of Unified Management Ideas
Step 5.1, designs novel compound files data organization method, by file data reasonable layout and multiple DIMM and PCI-E interface NVM memory device;
Step 5.2, for the distribution of file data in multiple DIMM and PCI-E interface NVM memory device, design document data unify wiring method;
Step 5.3, for the distribution of file data in multiple DIMM and PCI-E interface NVM memory device, design document collaboration data read method;
Step 5.4, for the distribution of file data in multiple DIMM and PCI-E interface NVM memory device, design fusion type file data amending method.
7. the unified file system management method towards SCM according to claim 6, it is characterized in that, in described step 5.1, compound files data organization method specifically comprises the following steps:
Step 6.1, each file comprises byte and block subregion two logical data subregions;
Step 6.2, the data block that block subregion is Sizep by a series of size forms, and is kept in PCI-E interface NVM memory device;
Step 6.3, byte partitions size is Sizep+1, is kept in DIMM interface NVM memory device.
8. the unified file system management method towards SCM according to claim 6, is characterized in that, described step 5.2 file data are unified wiring method and specifically comprised the following steps:
Step 7.1, when storing new file, first checking the size (Size represents file data size) of new file, if file size is less than Sizep, then forwarding step 7.2 to, otherwise forward step 7.3 to;
Step 7.2, by the byte partitions in the data of new file write DIMM interface NVM memory device, and is set to 0 by the value of station location marker in metadata base attribute, last ends file data writing process;
Step 7.3, the value of station location marker in metadata base attribute is set to 1, and first character joint in byte partitions is set to 0, then from DIMM interface NVM memory device byte starting position, second, byte partitions, preserving top of file size is the data of Size%Sizep;
Remaining data are write in the block subregion in PCI-E interface NVM memory device, last ends file data writing process by step 7.4 in units of Sizep.
9. the unified file system management method towards SCM according to claim 6, is characterized in that, described step 5.3 file collaboration data read method specifically comprises the following steps:
Step 8.1, uses file access path, the base attribute of locating file metadata, obtains the value of station location marker, as forwarded step 8.2 to for 0, otherwise forwards step 8.3 to;
Step 8.2, from the byte partitions file reading data being arranged in DIMM interface NVM memory device, by data feedback to user and ends file data read process;
Step 8.3, file reading data slot A from the block subregion of PCI-E interface NVM memory device, then from DIMM interface NVM memory device file reading data slot B the byte of second, byte partitions;
Step 8.4, from DIMM interface NVM memory device, byte partitions first character joint, reads the address information of file data fragments B, appropriate address data in file data fragments B and file data fragments A is exchanged;
Step 8.5, couples together file data fragments B and file data fragments A, obtains the data that file is complete, feeds back to user, and ends file data read process.
10. the unified file system management method towards SCM according to claim 6, is characterized in that, in described step 5.4, fusion type file data amending method specifically comprises the following steps:
Step 9.1, uses file access path, the base attribute of locating file metadata, obtains the value of station location marker, as forwarded step 9.2 to for 0, otherwise forwards step 9.3 to;
Step 9.2, revises file data in the byte partitions being arranged in DIMM interface NVM memory device, by result feedback to user and ends file data modification process;
Step 9.3, byte partitions first character joint from DIMM interface NVM memory device, the address information of data slot in sense byte subregion, compares with needing modified address, if need modified address in byte partitions, perform step 9.4, otherwise perform step 9.5;
Step 9.4, according to needing modified address, the data of byte partitions in amendment DIMM interface NVM memory device, and ends file data modification process;
Step 9.5, according to needing modified address, the data of block subregion in amendment PCI-E interface NVM memory device, and ends file data modification process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510044083.9A CN104636452B (en) | 2015-01-28 | 2015-01-28 | A kind of unified file system management method towards SCM |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510044083.9A CN104636452B (en) | 2015-01-28 | 2015-01-28 | A kind of unified file system management method towards SCM |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104636452A true CN104636452A (en) | 2015-05-20 |
| CN104636452B CN104636452B (en) | 2018-02-27 |
Family
ID=53215198
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510044083.9A Active CN104636452B (en) | 2015-01-28 | 2015-01-28 | A kind of unified file system management method towards SCM |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104636452B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598496A (en) * | 2016-12-08 | 2017-04-26 | 蓝信工场（北京）科技有限公司 | Method and device for constructing virtual magnetic disk and processing data |
| WO2020253401A1 (en) * | 2019-06-18 | 2020-12-24 | 深圳前海微众银行股份有限公司 | File management method, apparatus, system and device, and computer-readable storage medium |
| CN112506425A (en) * | 2020-11-19 | 2021-03-16 | 江苏大学 | Multi-core-oriented NVM storage device simulator and design method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130339570A1 (en) * | 2012-06-18 | 2013-12-19 | International Business Machines Corporation | Variability aware wear leveling |
| CN103562897A (en) * | 2011-06-10 | 2014-02-05 | 国际商业机器公司 | Store storage class memory information command |
| US20140101370A1 (en) * | 2012-10-08 | 2014-04-10 | HGST Netherlands B.V. | Apparatus and method for low power low latency high capacity storage class memory |
| CN103810126A (en) * | 2014-01-27 | 2014-05-21 | 上海新储集成电路有限公司 | Mixed DRAM storage and method of reducing refresh power consumption of DRAM storage |
-
2015
- 2015-01-28 CN CN201510044083.9A patent/CN104636452B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103562897A (en) * | 2011-06-10 | 2014-02-05 | 国际商业机器公司 | Store storage class memory information command |
| US20130339570A1 (en) * | 2012-06-18 | 2013-12-19 | International Business Machines Corporation | Variability aware wear leveling |
| US20140101370A1 (en) * | 2012-10-08 | 2014-04-10 | HGST Netherlands B.V. | Apparatus and method for low power low latency high capacity storage class memory |
| CN103810126A (en) * | 2014-01-27 | 2014-05-21 | 上海新储集成电路有限公司 | Mixed DRAM storage and method of reducing refresh power consumption of DRAM storage |
Non-Patent Citations (3)
| Title |
|---|
| XIAOJIAN WU等: "SCMFS: A file system for storage class memory", 《ACM》 * |
| 嵇智源等: "面向大数据的内存数据管理研究现状与展望", 《计算机工程与设计》 * |
| 蔡涛等: "NVMMDS--一种面向非易失存储器的元数据管理方法", 《计算机研究与发展》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598496A (en) * | 2016-12-08 | 2017-04-26 | 蓝信工场（北京）科技有限公司 | Method and device for constructing virtual magnetic disk and processing data |
| WO2020253401A1 (en) * | 2019-06-18 | 2020-12-24 | 深圳前海微众银行股份有限公司 | File management method, apparatus, system and device, and computer-readable storage medium |
| CN112506425A (en) * | 2020-11-19 | 2021-03-16 | 江苏大学 | Multi-core-oriented NVM storage device simulator and design method |
| CN112506425B (en) * | 2020-11-19 | 2024-04-09 | 江苏大学 | Multi-core-oriented NVM (non-volatile memory) storage device simulator and design method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104636452B (en) | 2018-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10579683B2 (en) | Memory system including key-value store | |
| US9594524B2 (en) | System and method for distributed computing in non-volatile memory | |
| US10296250B2 (en) | Method and apparatus for improving performance of sequential logging in a storage device | |
| CN108959119A (en) | The method and system of garbage collection in storage system | |
| US20150058554A1 (en) | Systems, Methods, and Computer Program Products Implementing Hybrid File Structures for Data Storage | |
| Le-Tuan et al. | RDF4Led: An RDF engine for lightweight edge devices | |
| CN104636452A (en) | Unified file system managing method oriented to SCM | |
| Li et al. | Enabling efficient updates in KV storage via hashing: Design and performance evaluation | |
| Khan et al. | MOSIQS: Persistent memory object storage with metadata indexing and querying for scientific computing | |
| Xu et al. | Enhancing HDFS with a full-text search system for massive small files | |
| Xiang et al. | A reliable B-tree implementation over flash memory | |
| Na et al. | Iplb+-tree for flash memory database systems. | |
| CN103530067A (en) | Data operation method and device | |
| Chen et al. | Co-optimizing storage space utilization and performance for key-value solid state drives | |
| Yin et al. | PBFilter: A flash-based indexing scheme for embedded systems | |
| Wang et al. | Meta-Block: Exploiting Cross-Layer and Direct Storage Access for Decentralized Blockchain Storage Systems | |
| CN109597793B (en) | Method for improving IO performance of file system | |
| Mirajkar et al. | Perform wordcount map-reduce job in single node apache hadoop cluster and compress data using lempel-ziv-oberhumer (lzo) algorithm | |
| Chang et al. | Emt: Elegantly measured tanner for key-value store on ssd | |
| Ku et al. | File deduplication with cloud storage file system | |
| US11914587B2 (en) | Systems and methods for key-based indexing in storage devices | |
| CN103559260B (en) | The dynamic adjusting method of inode table based on RAMDISK file system | |
| Wang | Tags: A Unifying Primitive for the Storage Data Path | |
| Lim et al. | Hashing directory scheme for NAND flash file system | |
| Hauser | Parallel I/O for the CGNS system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210114 Address after: Room 3-53, room B2, No.15, Jinyang Road, Huaqiao Town, Kunshan City, Suzhou City, Jiangsu Province Patentee after: Jiangsu abid Information Technology Co.,Ltd. Address before: Zhenjiang City, Jiangsu Province, 212013 Jingkou District Road No. 301 Patentee before: JIANGSU University |
|
| TR01 | Transfer of patent right |