CN105446899B - The method of internal storage data quick and durableization based on storage level memory - Google Patents
The method of internal storage data quick and durableization based on storage level memory Download PDFInfo
- Publication number
- CN105446899B CN105446899B CN201510759531.3A CN201510759531A CN105446899B CN 105446899 B CN105446899 B CN 105446899B CN 201510759531 A CN201510759531 A CN 201510759531A CN 105446899 B CN105446899 B CN 105446899B
- Authority
- CN
- China
- Prior art keywords
- persistent region
- lasting
- scm
- persistent
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000015654 memory Effects 0.000 title claims abstract description 100
- 230000002085 persistent Effects 0.000 claims abstract description 139
- 230000002045 lasting Effects 0.000 claims abstract description 62
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000004044 response Effects 0.000 claims abstract description 5
- 230000000875 corresponding Effects 0.000 claims description 12
- 238000004064 recycling Methods 0.000 claims description 6
- 229910002056 binary alloy Inorganic materials 0.000 claims description 4
- 230000015572 biosynthetic process Effects 0.000 claims description 4
- 238000005755 formation reaction Methods 0.000 claims description 4
- 238000003780 insertion Methods 0.000 claims description 4
- 230000001960 triggered Effects 0.000 claims description 2
- 230000002688 persistence Effects 0.000 abstract description 20
- 238000011030 bottleneck Methods 0.000 abstract description 4
- 241000552094 Mnemosyne Species 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006011 modification reaction Methods 0.000 description 1
- 230000000630 rising Effects 0.000 description 1
Abstract
The present invention provides a kind of methods of internal storage data quick and durableization based on storage level memory.This method is realized based on flat mixing memory architecture by the collaborative design of client layer and inner nuclear layer.Including：It is one piece of persistent region by the SCM of certain capacity and the data abstraction being stored on this SCM；In client layer design application programming interface (API), memory access of the response application program to persistent region；The Buddy system of kernel are extended, realize isomery mixing memory management；In design persistent region manager, realization includes the functions such as lasting mapping, the persistent region metadata management of persistent region.The present invention can be to avoid under conventional store framework, Data Linearization process needed for data persistence, while can realize direct access and the original place update of persistant data, the level for simplifying software stack under conventional architectures is called, I/O bottlenecks can be effectively relieved, improve the memory access performance of persistant data.
Description
Technical field
The invention belongs to the systems software technologies based on mixing memory architecture in persistence memory field, and in particular, to
It is a kind of using the memory management of nonvolatile memory, process address space management, address mapping technique based on storage level memory
The method of internal storage data quick and durableization.
Background technology
With the arrival of big data, being continuously increased with the increasingly sophisticated of data type for data volume is internal storage data persistence
More stringent requirements are proposed.In data processing and calculating process, metadata, application data and the high property of system software
Various types of data such as the calculating state that can be calculated may all be needed by efficient persistence.Currently, internal storage data is lasting
Change some lasting primitive (file either block I/O) that operating system offer is provided or Database Systems both skills
Art.When realizing persistence with system primitives, application program needs voluntarily to pay close attention to the atomicity of program code to ensure persistently
Newer integrality, while also relating to serializing/unserializing operation of internal storage data.And Database Systems can not be very
The persistence problem of system software metadata is coped with well, and faces the processing of data impedance mismatch problem.In conclusion working as
The method of preceding internal storage data persistence can all introduce the additional treatments of application program, this undoubtedly increases application programming
Complexity, the software stack of level and frequent I/O operation have been greatly reduced the efficiency of data persistence.
The development of nonvolatile memory technology provides more efficiently solution for internal storage data persistence.It is non-volatile
Memory due to its is non-volatile, can the good characteristics such as byte addressing, lower memory access latency and high storage density, become academic
The focus of attention on boundary and business circles.M.K.Qureshi et al. is in M.K.Qureshi, V.Srinivasan, and J.A.Rivers.
The expansible main storage system .ACM SIGARCH Computer Architecture of high-performance based on phase transition storage
It is proposed builds the design of main storage system with nonvolatile memory in News, vol.37, no.3, pp.24-33, and gives certain
Performance testing index.J.Y.Jung et al. is in J.Y.Jung and S.Cho.Memorage：Based on emerging non-volatile storage
Adaptability main memory and storage architecture .Proceedings of the 27th international ACM conference
On International conference on supercomputing.ACM have inquired into isomery in 2013, pp.115-126
Mix memory structure and the influence etc. to Design of System Software.It by rambus research shows that access department of computer science
System, the structure mixing memory system together with DRAM can utmostly obtain the good characteristic using non-volatile memory device, simultaneously
Broader design space can be provided for the research and development of system software and application software.
Through retrieval, H.Volos etc. is in H.Veles, A.J.Tack, and M.M.Swift.Mnemosyne：Light-duty is lasting
Memory system .ACM SIGARCH Computer Architecture News, vol.39, no.1.ACM, 2011, pp.91-
A kind of lasting memory system based on nonvolatile memory technology is proposed in 104, there are following defects for the lasting memory system：
1) mapping mechanism of the lasting memory system of H.Veles places one's entire reliance upon the mapping mechanism of traditional file system；
2) its persistence internal storage data needs to be mapped to the persistence that fixed virtual address just can guarantee its address of cache.
Currently without the explanation or report for finding technology similar to the present invention, it is also not yet collected into money similar both at home and abroad
Material.
Invention content
In view of the above defects of the prior art, the purpose of the present invention is utilize natural non-volatile of storage level memory
The good characteristics such as property and high density, low latency, it is quick and durable to design and Implement a kind of internal storage data based on storage level memory
The method of change.By providing API, (Application Programming Interface, application programming connect the present invention
Mouthful), for application program provide can direct memory access SCM (Storage Class Memory, storage level memory) interface；Inside
Stratum nucleare designs persistent region manager, realizes the persistent region logically and physically mapping of attribute and persistent region metadata
Management etc. so that persistent region persistently can re-run process in application program or system.
To achieve the above object, the present invention is achieved by the following technical solutions.
A method of internal storage data quick and durableization based on storage level memory includes the following steps：
Step 1：The storage level memory (SCM) of certain capacity is abstracted as one piece with the persistant data for being stored in this SCM to hold
Long region；
Step 2：It is designed for the api interface of application call, the memory access that response application program accesses persistent region is asked
It asks；
Step 3：The user program process address space of persistent region mapping is dynamically distributed and managed；
Step 4：Kernel buddy system (Buddy system) is extended, realizes the management and distribution of interior verification SCM；
Step 5：Design persistent region manager, including lasting mapping table and persistently to the processing letter of lasting mapping table
Number realizes the management of persistent region metadata and the mapping of the logically and physically attribute to persistent region；
Step 6：Skip leaf processing function of the design for persistent region memory access.
Preferably, in the step 1, each piece of persistent region defines a unique ID to mark by application program
Know, is denoted as P_ID.
Preferably, in the step 2, client layer design api interface, access of the application program to persistent region,
Pass through api interface trigger request；The api interface can provide the visit for dynamically applying for and recycling persistent region for user program
Ask function, meanwhile, when application persistent region accesses, using unique ID of persistent region as API Calls parameter.
Preferably, in the step 3, when dynamically distributing the process address space, first in user program process address
Then the memory headroom of the address of cache segment identification 4k integral multiples in space calls Memory Allocation to calculate in the memory headroom of the mark
Method carries out dynamic process address space distribution.
Preferably, in the step 4, include the following steps：
Linux kernel is extended in inner nuclear layer, increases a SCM_ZONE memories domain, for describing SCM, and passes through
Unified managements of the Buddy system to DRAM and SCM is realized in this extension；
In order to improve the durability of stored memory, while extending Buddy system, Buddy system couple are improved
Distribution in SCM and recycling algorithm；The doubly linked list for efficiently using Buddy system will be released when the page recycles
Page chain enter the table tail to corresponding chained list, to homogenize the memory access frequency of the page, reduce the formation of hot pages, extend
SCM's writes the service life；
Increase by one and write the critical chained list of frequency, writes the critical chained list of frequency by being added to more than the page for writing frequency threshold, keep away
Exempt from due to physical data memory access mistake caused by the physical fault of SCM.
Preferably, in the step 5, the persistent region manager to all persistent regions in SCM for carrying out pipe
Reason realizes the mapping of the logically and physically attribute of persistent region and the pipe of persistent region metadata by persistent region manager
Reason, specially：
Logically and physically attribute is mapping through a triple<P_ID, P_addr, P_size>Form persistently mapping note
Record, wherein P_ID are the unique mark of persistent region, and P_addr is the physical start address for storing persistent region, and P_size is to hold
The size that long region is taken up space；Lasting mapping table is the hash table that a plurality of lasting map record is formed, and the hash table is not only
Store the mapping relations of the logically and physically attribute of each persistent region, at the same have recorded each persistent region size and
The physical layout informations such as initial address, therefore when one establishment, deletion or update persistent region, will all trigger to persistently mapping
Insertion, deletion or the operation for updating a lasting map record of table.In addition in system initialization, Buddy systmem
It needs to be scanned lasting mapping table to build the idle chained list of persistent region；
Lasting mapping table is retrieved by P_ID to obtain the P_addr of persistent region, which is calculated by Hash
Method improves retrieval rate, specially：
1) expression of the character string of P_ID is converted into binary representation；
2) by mixing hash algorithm, the binary system P_ID processing to 128 finally obtains 24 storage sequences
Number；
3) storage address that the lasting map record corresponding to P_ID is calculated in sequence number again is thus stored；
Whether effectively indicate the highest order of P_ID binary representations as the lasting map record：Wherein, 0 table
Show that lasting map record is invalid；1 indicates that lasting map record is accurate, complete and effective.
Preferably, in the step 6, for the processing function that skips leaf of persistent region memory access, specifically, being to search with P_ID
Rope keyword retrieves lasting mapping table, obtains the physical address of corresponding persistent region, then updates the page of user program process again
Table realizes the mapping of virtual address and physical address.
Compared with traditional data persistence technology, the present invention has following advantage：
1, the present invention efficiently uses the excellent spies such as the natural non-volatile and low latency of non-volatile memory device, high density
Property, support the persistence of application data and system data to avoid the sequence of data while keeping original programming idea
Change/unserializing operation, while can realize the original place update of persistant data, simplify persistant data memory access process under conventional architectures
In level call, I/O bottlenecks can be effectively relieved, improve the memory access performance of persistant data.
2, the present invention uses lasting map manager, is cleverly realized using persistent region ID by lasting mapping table
The mapping of persistent region logical AND physical attribute, this method this allow calling program by persistent region as at main memory
Reason, not only relieves the dependence to traditional file systems mapping mechanism, while overcoming virtual address must fixed defect.
3, the present invention is it is possible to prevente effectively from Data Linearization process under conventional store framework needed for internal storage data persistence,
Improve the performance of data persistence；
4, the directly access and original place update of the achievable persistant data of the present invention, simplifies the layer of software stack under conventional architectures
Grade is called, and I/O bottlenecks can be effectively relieved, improve the memory access performance of persistant data；
5, the present invention can break away from dependence of the data address mapping to traditional file systems, and can be according to practical need
Ask, persistent region be mapped to the process address space appropriate so that the persistence of internal storage data have opposite independence and
Flexibility.
Description of the drawings
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is that the present invention is based on the implementing procedure figures of the method for internal storage data quick and durableization of SCM.
Fig. 2 is program example of the application program to persistent region memory access.
Fig. 3 is the process chart that skips leaf for accessing persistent region triggering.
Fig. 4 is to delete persistent region flow chart.
Specific implementation mode
It elaborates below to the embodiment of the present invention：The present embodiment is carried out lower based on the technical solution of the present invention
Implement, gives detailed embodiment and specific operating process.It should be pointed out that those skilled in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention
Protect range.
Embodiment
A kind of method of internal storage data quick and durableization based on storage level memory is present embodiments provided, including is walked as follows
Suddenly：
The storage level memory (SCM) of certain capacity is abstracted as one piece with the persistant data for being stored in this SCM and holds by step 1
Long region；
Further, each persistent region is denoted as P_ID by an ID unique mark.
Step 2 designs the api interface for application call in client layer, and response application program visits persistent region
The access request asked；
Further, the access request or including any one of following or appoint multinomial：Application persistent region memory headroom,
Realize persistent region to the mapping of the process address space, release persistent region memory headroom, releasing persistent region to process address
Persistent region is deleted in the mapping in space；
It further, must be using persistent region ID as API Calls parameter when the access request that application persistent region accesses.
Step 3 is dynamically distributed and is managed to the user program process address space of persistent region mapping；
Further, the memory headroom of persistent region cross-page cannot store, in view of this, empty dynamically distributing process address
Between when, the memory headroom in the address of cache segment identification 4k integral multiples of the process address space first designs allocation algorithm, in the mark
The process address space of knowledge carries out dynamic memory distribution.
Step 4 extends kernel Buddy system in inner nuclear layer, realizes the management and distribution of interior verification SCM；
Traditional memory management is this to DRAM design, it is therefore desirable to memory management, especially to buddy system
(Buddy system) is extended and improves, and realizes the unified management to DRAM and SCM；
Further, specifically comprise the following steps：First, linux kernel is extended, is increased in a SCM_ZONE
Domain is deposited, for describing SCM, and by this extension, realizes unified managements of the Buddy system to isomery mixing memory；
Secondly, it is contemplated that SCM writes the limited physical characteristic of number and it is right to improve its while extending Buddy system
Distribution in SCM and recycling algorithm；Main thought is to efficiently use the doubly linked list of Buddy system, is recycled in the page
When, the page chain being released is entered into the table tail to corresponding chained list；This improved purpose is to homogenize the memory access frequency of the page, is subtracted
The formation of few hot pages writes the service life to extend SCM；
In addition, increasing by one writes the critical chained list of frequency, frequency is write by this is added to more than the page for centainly writing frequency threshold
Critical chained list is avoided due to physical data memory access mistake caused by the physical fault of SCM.
Step 5, design persistent region manager；Including lasting mapping table and persistently to the processing letter of lasting mapping table
Number；
Further, persistent region manager is the management to all persistent regions in SCM, realizes the logic of persistent region
The management etc. of mapping and persistent region metadata with physical attribute；
Further, specifically comprise the following steps：
First, logically and physically attribute is mapping through a triple<P_ID, P_addr, P_size>It realizes, wherein
P_ID is the unique mark of persistent region, and P_addr is to store the physical start address of persistent region, and P_size then holds for this
The size that long region is taken up space；Lasting mapping table is that a plurality of triple records the hash table to be formed, which not only deposits
The mapping relations of the logically and physically attribute of each persistent region have been stored up, while having had recorded the size of each persistent region and rising
The physical layout informations such as beginning address, therefore when one establishment, deletion, update persistent region, will all trigger to lasting mapping table
Insertion, deletion, update a lasting map record operation.In addition in system initialization, Buddy systmem are also required to
Lasting mapping table is scanned to build the idle chained list of persistent region；
Secondly, physical address retrieval is carried out by P_ID, which is a kind of high-frequency operation, is calculated by Hash in this step
Method improves retrieval rate, and concrete scheme is：
1) expression of the character string of P_ID is converted into its binary representation；
2) by mixing hash algorithm, the binary system ID processing to 128 finally obtains 24 storage sequences
Number；
3) thus the storage address of the lasting map record is calculated in sequence number again；
Finally, the highest order of P_ID binary representations is used as whether the record effectively indicates：Wherein, it 0 indicates
Lasting map record is invalid；1 indicates that lasting map record is accurate, complete and effective.
Step 6, design handle function for skipping leaf for persistent region memory access；
Further, different in the processing mode of disk from traditional persistence, persistent region is once establish, and data are just
It is to be persisted on storage level memory, physical page need not be distributed again for it, is only established not in page table empty
The mapping of quasi- address and physical address；For such case, this step devises the processing function that skips leaf based on persistent region,
Function is, using P_ID as search key, retrieves lasting mapping table, obtains the physical address of corresponding persistent region, then again more
The page table of new process, realizes the mapping of virtual address and physical address.
The present embodiment is further described below in conjunction with the accompanying drawings.
The method of internal storage data quick and durableization provided in this embodiment based on storage level memory, first by certain capacity
SCM and be stored in the persistant data of this SCM and be abstracted as one piece of persistent region, each piece of persistent region is required for application program fixed
Justice one unique ID is identified.Application program must provide a persistent region ID, which is lasting in memory access persistent region
The embodiment of the unique mark in region and its logical attribute.The ID supports maximum 16 character strings.
The present invention is realized by the collaborative design of client layer and inner nuclear layer.
The API that client layer provides is entrance of the application program to persistent region memory access, and table 1 is to be exposed to application program
Api interface lists the API that the present invention designs and Implements.Application program passes through P_alloc (P_ID, size) and P_mmap (P_
ID) memory access of the application to persistent region；P_unmmap (addr) and P_free (addr) is release persistent region in process address
Region of memory in space；P_delete (P_ID) is to delete a persistent region completely.Parameter P_ID be persistent region only
One mark.
Table 1
| API | Function describes |
| P_alloc (P_ID, size) | For persistent region dynamic assigning memory space |
| P_mmap(P_ID) | Persistent region is mapped to the process address space |
| P_unmmap(addr) | Release mapping of the persistent region to the process address space |
| P_free(addr) | The memory headroom of release dynamics distribution |
| P_delere(P_ID) | A persistent region is deleted completely |
Since the memory headroom of persistent region cross-page cannot store, the present invention when dynamically distributing address space,
The memory headroom of 4k integral multiples is first identified in the address of cache section of the process address space first, allocation algorithm is designed, in the mark
Space carries out dynamic memory distribution, and bottom calls improved mmap () system function to realize.
Inner nuclear layer mainly realizes extension and the persistent region manager of Buddy system.
The extension of Buddy system increases a SCM_ZONE memories domain first, for describing SCM, and passes through this expansion
Unified managements of the Buddy system to isomery mixing memory is realized in exhibition.In view of the physical characteristic of SCM and DRAM is in the presence of poor
It is different, while extending Buddy system, it is improved for the distribution of SCM and recycling algorithm.Main thought is effective
Using the doubly linked list of Buddy system, and disable the cold and hot caching functions of its cpu.When per sub-distribution, Buddy
System chooses the page from the head node of corresponding chained list on demand and completes distribution, and when the page recycles, the page that will be released
Chain enters the table tail to corresponding chained list, in the case of capable of being merged with partner, is first merged with partner, then will merge again
Page block chain afterwards enters the table tail to corresponding chained list.This improved purpose is to homogenize the memory access frequency of the page, reduces hot spot page
The formation in face writes the service life to extend SCM.In addition increase by one and write the critical chained list of frequency, will be more than centainly to write frequency threshold
The page be added to the chained list, to maintain the correctness of data memory access.
Persistent region manager is the management to all persistent regions in SCM, realizes the logically and physically category of persistent region
The mapping of property and the management etc. of persistent region metadata.
First, logically and physically the triple that is mapping through of attribute is realized<P_ID, P_addr, P_size>, wherein
P_ID is the unique mark of persistent region as described above, and P_addr is the physical start address of storage persistent region, and P_
The size that size is then taken up space for the persistent region；Lasting mapping table is that a plurality of triple records the hash table to be formed, should
Hash table not only stores the mapping relations of the logically and physically attribute of each persistent region, while having recorded each persistent region
Size and the physical layout informations such as initial address all will triggering therefore when one establishment, deletion, update persistent region
To the insertion of lasting mapping table, deletion, the operation for updating a lasting map record.In addition in system initialization, Buddy
Systmem is also required to be scanned lasting mapping table to build the idle chained list of persistent region；
Secondly as it is a high-frequency operation to carry out the retrieval of P_addr by P_ID, and its efficiency requirements is also compared
It is higher, therefore the present invention improves retrieval rate by hash algorithm, the expression of the character string of P_ID is converted into first secondly
System representation, then by mono- 256 algorithms of sha and division Hash, the binary system ID processing to 128, final
To 24 storage sequence numbers, thus the storage address of the lasting map record is calculated in sequence number again；Clash handle uses
Linear probing method, and it is whether idle come identification record storage unit by bitmap.
The highest order of P_ID binary representations is used as whether the record effectively indicates by the last present invention：0：Persistently
Map record is invalid, and possible state is data are imperfect, data are inaccurate or map information fails etc.；1：Persistently mapping note
It records accurate, complete and effective.
Fig. 1 is the implementation process of internal storage data persistence method.
Fig. 2 is the example code that application program carries out persistent region memory access, and the completely compatible existing programming of programming is thought
Think and code normalized written.
When application program is by API Access persistent region, persistent region manager responds its access request.
Call transfer function that the character string expression of P_ID is converted into its binary representation first.Then it calls and breathes out
Uncommon function, calculates storage address of the lasting map record in lasting mapping table, then judges whether the lasting mapping is effective.
If invalid, explanation is to access the persistent region for the first time, then just needing to call memory allocation algorithm, is held for this
Long region is then the physical page needed for its distribution by Buddy system in process address space storage allocation region,
Process page table is updated later, establishes the mapping of virtual address and physical address, updates lasting mapping table later, establishes the lasting area
The lasting map record in domain.
If effectively, illustrate persistent region oneself through creating successfully, need to check whether that oneself passes through at this time and established for the region
The mapping of virtual address and physical address.If oneself is set up, data memory access can be directly carried out.If do not established,
Processing of skipping leaf is triggered, process flow is as shown in Figure 3.
Persistent region memory access skip leaf place it is different with the processing mode of disk from traditional persistence, persistent region is once built
Vertical, data are exactly to be persisted on storage level memory, physical page need not be distributed again for it, only not in page
The mapping of virtual address and physical address is established in table.For such case, the present invention devises skipping leaf based on persistent region
Function is handled, function is to retrieve lasting mapping table using P_ID as search key, obtains corresponding persistent region physically
Then location updates the page table of process again, realize the mapping of virtual address and physical address.
Once establishing the mapping of virtual address and physical address, so that it may to carry out memory access to persistent region.
When application program deletes a persistent region by API, persistent region manager equally using P_ID as keyword, is pressed
According to the above method, lasting mapping table is retrieved, obtains the lasting map record of the persistent region, judges whether the lasting mapping has
Effect, if in vain, throw exception, if it is valid, the mark of the map record is set as invalid, flow chart such as Fig. 4 institutes
Show.
The method of internal storage data quick and durableization provided in this embodiment based on storage level memory (SCM), based on flat
Mixing memory architecture, realized by the collaborative design of client layer and inner nuclear layer.By the SCM of certain capacity and it is stored in this
Data abstraction on SCM is one piece of persistent region；In client layer design application programming interface (API), response application program
Memory access to persistent region；The Buddy system of kernel are extended, realize isomery mixing memory management；In design persistent region pipe
Device is managed, realization includes the functions such as lasting mapping, the persistent region metadata management of persistent region.The present embodiment can be to avoid passing
It unites under storage architecture, the Data Linearization process needed for data persistence, while can realize direct access and the original of persistant data
Ground updates, and the level for simplifying software stack under conventional architectures is called, and I/O bottlenecks can be effectively relieved, improve the memory access of persistant data
Performance.
Specific embodiments of the present invention are described above.It is to be appreciated that the invention is not limited in above-mentioned
Particular implementation, those skilled in the art can make various deformations or amendments within the scope of the claims, this not shadow
Ring the substantive content of the present invention.
Claims (6)
1. a kind of method of internal storage data quick and durableization based on storage level memory, which is characterized in that include the following steps：
Step 1：The SCM of certain capacity and the persistant data for being stored in this SCM are abstracted as one piece of persistent region；Wherein SCM is
Storage level memory；
Step 2：It is designed for the api interface of application call, the access request that response application program accesses persistent region；
Step 3：The user program process address space of persistent region mapping is dynamically distributed and managed；
Step 4：Kernel Buddy system are extended, realize the management and distribution of interior verification SCM；
Step 5：Design persistent region manager, including lasting mapping table and persistently to the processing function of lasting mapping table, it is real
The management of existing persistent region metadata and the mapping of the logically and physically attribute to persistent region；
Step 6：Skip leaf processing function of the design for persistent region memory access；
In the step 1, each piece of persistent region defines a unique ID to identify by application program, is denoted as P_
ID；
In the step 5, the persistent region manager is for being managed all persistent regions in SCM, by lasting
Zone manager realizes the mapping of the logically and physically attribute of persistent region and the management of persistent region metadata, specially：
Logically and physically attribute is mapping through a triple<P_ID, P_addr, P_size>Lasting map record is formed,
Middle P_ID is the unique mark of persistent region, and P_addr is the physical start address for storing persistent region, and P_size is lasting area
The size that domain is taken up space；Lasting mapping table is the hash table that a plurality of lasting map record is formed, which not only stores
The mapping relations of the logically and physically attribute of each persistent region, while having recorded the physical layout letter of each persistent region
Breath, therefore when one establishment, deletion or update persistent region, all insertions, deletion or update by triggering to lasting mapping table
The operation of one lasting map record；In addition in system initialization, Buddy systmem are also required to carry out lasting mapping table
It scans to build the idle chained list of persistent region；
Lasting mapping table is retrieved by P_ID to obtain the P_addr of persistent region, which is carried by hash algorithm
High retrieval rate, specially：
1) expression of the character string of P_ID is converted into binary representation；
2) by mixing hash algorithm, the binary system P_ID processing to 128 finally obtains 24 storage sequence numbers；
3) storage address that the lasting map record corresponding to P_ID is calculated in sequence number again is thus stored；
Whether effectively indicate the highest order of P_ID binary representations as the lasting map record：Wherein, it 0 indicates to hold
Long map record is invalid；1 indicates that lasting map record is accurate, complete and effective.
2. the method for internal storage data quick and durableization according to claim 1 based on storage level memory, which is characterized in that
In the step 2, api interface is designed in client layer, access of the application program to persistent region is triggered by api interface
Request；The api interface can provide the access function for dynamically applying for and recycling persistent region for user program, meanwhile, Shen
Please persistent region access when, using unique ID of persistent region as API Calls parameter.
3. the method for internal storage data quick and durableization according to claim 1 based on storage level memory, which is characterized in that
In the step 3, when dynamically distributing the process address space, first in the address of cache section of the user program process address space
The memory headroom of 4k integral multiples is identified, then calls memory allocation algorithm in the memory headroom of the mark, with carrying out dynamic process
Location space is distributed.
4. the method for internal storage data quick and durableization according to claim 1 based on storage level memory, which is characterized in that
In the step 4, include the following steps：
Linux kernel is extended in inner nuclear layer, increases a SCM_ZONE memories domain, for describing SCM, and passes through this
Unified managements of the Buddy system to DRAM and SCM is realized in extension；
While extending Buddy system, Buddy system are improved for the distribution of SCM and recycling algorithm；It is i.e. effectively sharp
The page chain being released is entered into the table tail to corresponding chained list when the page recycles with the doubly linked list of Buddy system, to
The memory access frequency for homogenizing the page, reduces the formation of hot pages, extend SCM writes the service life；
Increase by one write the critical chained list of frequency, write the critical chained list of frequency by being added to more than the page for writing frequency threshold, avoid by
Physical data memory access mistake caused by physical fault in SCM.
5. the method for internal storage data quick and durableization according to claim 1 based on storage level memory, which is characterized in that
The physical layout information of the persistent region includes any one of following or appoints multinomial：
The size of persistent region；
The initial address of persistent region.
6. the method for internal storage data quick and durableization according to claim 1 based on storage level memory, which is characterized in that
In the step 6, design is examined for the processing function that skips leaf of persistent region memory access specifically, using P_ID as search key
Suo Chijiu mapping tables obtain the physical address of corresponding persistent region, then update the page table of user program process again, realize virtual
The mapping of address and physical address.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510759531.3A CN105446899B (en) | 2015-11-09 | The method of internal storage data quick and durableization based on storage level memory |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510759531.3A CN105446899B (en) | 2015-11-09 | The method of internal storage data quick and durableization based on storage level memory |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105446899A CN105446899A (en) | 2016-03-30 |
| CN105446899B true CN105446899B (en) | 2018-08-31 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750202A (en) * | 2012-06-06 | 2012-10-24 | 宇龙计算机通信科技(深圳)有限公司 | Data protection method and device |
| CN104239225A (en) * | 2014-09-04 | 2014-12-24 | 浪潮(北京)电子信息产业有限公司 | Method and device for managing heterogeneous hybrid memory |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750202A (en) * | 2012-06-06 | 2012-10-24 | 宇龙计算机通信科技(深圳)有限公司 | Data protection method and device |
| CN104239225A (en) * | 2014-09-04 | 2014-12-24 | 浪潮(北京)电子信息产业有限公司 | Method and device for managing heterogeneous hybrid memory |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109213772B (en) | Data storage method and NVMe storage system | |
| US10552336B2 (en) | Memory system and method for controlling nonvolatile memory | |
| US8843454B2 (en) | Elimination of duplicate objects in storage clusters | |
| CN110347852B (en) | File system embedded with transverse expansion key value storage system and file management method | |
| US11392544B2 (en) | System and method for leveraging key-value storage to efficiently store data and metadata in a distributed file system | |
| US8326839B2 (en) | Efficient file access in a large repository using a two-level cache | |
| CN107180092B (en) | File system control method and device and terminal | |
| CN110018998B (en) | File management method and system, electronic equipment and storage medium | |
| CN107463447A (en) | B + tree management method based on remote direct nonvolatile memory access | |
| US7870122B2 (en) | Self-tuning index for flash-based databases | |
| CN106844584B (en) | Metadata structure, operation method, positioning method and segmentation method based on metadata structure | |
| CN110968269A (en) | SCM and SSD-based key value storage system and read-write request processing method | |
| CN109407985B (en) | Data management method and related device | |
| WO2020215580A1 (en) | Distributed global data deduplication method and device | |
| KR101806394B1 (en) | A data processing method having a structure of the cache index specified to the transaction in a mobile environment dbms | |
| KR101936364B1 (en) | Memory management system using flash memory and method thereof | |
| TW201040717A (en) | Flash memory managing methods and computing systems utilizing the same | |
| Xu et al. | Building a fast and efficient LSM-tree store by integrating local storage with cloud storage | |
| CN105446899B (en) | The method of internal storage data quick and durableization based on storage level memory | |
| US20200019539A1 (en) | Efficient and light-weight indexing for massive blob/objects | |
| US11586353B2 (en) | Optimized access to high-speed storage device | |
| CN111796767A (en) | Distributed file system and data management method | |
| US7979638B2 (en) | Method and system for accessing data using an asymmetric cache device | |
| CN112417058A (en) | Data processing method, storage system and storage medium | |
| CN110262755A (en) | A kind of file memory method of embedded system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |