CN108628753B - Memory space management method and device - Google Patents

Memory space management method and device Download PDF

Info

Publication number
CN108628753B
CN108628753B CN201710184680.0A CN201710184680A CN108628753B CN 108628753 B CN108628753 B CN 108628753B CN 201710184680 A CN201710184680 A CN 201710184680A CN 108628753 B CN108628753 B CN 108628753B
Authority
CN
China
Prior art keywords
node
file
target file
storage
directory entry
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710184680.0A
Other languages
Chinese (zh)
Other versions
CN108628753A (en
Inventor
舒继武
陆游游
曾开胜
王元钢
戴芬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Huawei Technologies Co Ltd
Original Assignee
Tsinghua University
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University, Huawei Technologies Co Ltd filed Critical Tsinghua University
Priority to CN201710184680.0A priority Critical patent/CN108628753B/en
Publication of CN108628753A publication Critical patent/CN108628753A/en
Application granted granted Critical
Publication of CN108628753B publication Critical patent/CN108628753B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023Free address space management
    • G06F12/0238Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory

Abstract

The application provides a memory space management method and a memory space management device, wherein the method comprises the following steps: receiving a file access request of a first thread, wherein the file access request comprises data of a first target file to be stored; acquiring a first bitmap line segment tree corresponding to a first storage space to which the first thread belongs, wherein the first bitmap line segment tree comprises N/S leaf nodes, N is used for indicating the number of storage blocks in the first storage space, and S is used for indicating the number of storage blocks in the first storage space managed by each leaf node; determining a first leaf node in the first bitmap segment tree; selecting at least one free memory block from the S memory blocks managed by the first leaf node; storing data of the first target file into the at least one free memory block. The memory space management method and the memory space management device can randomly allocate the idle memory blocks, so that the wear balance of the memory is realized, and the service life of the memory is prolonged.

Description

Memory space management method and device
Technical Field
The present application relates to the field of computers, and more particularly, to a memory space management method and apparatus.
Background
A Non-Volatile Memory (NVM) is a new type of Memory, and has many advantages, such as being addressable by bytes, not losing data after power failure, high storage density, no need of frequent refreshing, and low power consumption, and therefore, it is very suitable for being used as a Memory (or main Memory). However, NVM also has some disadvantages, such as relatively long write latency, limited write times, etc. The NVM can be combined with an existing Dynamic Random Access Memory (DRAM) to form a hybrid Memory, where temporary data generated by an application can be stored in the DRAM and persistent data generated by the application can be stored in the NVM.
Conventional disk-based file systems are not well suited for use on NVM because their I/O requests for data must go through a deep software stack to complete the entire process. To solve this problem, some Persistent File systems with Byte-addressable Persistent File systems (BPFS), Storage Class File Memory File systems (SCMFS), Persistent Memory File Systems (PMFS), Hierarchical File systems (HiNFS), and the like for reconstructing Byte granularity based on NVM have appeared. However, the aging problem of these file systems is rarely considered, for example, PMFS, and after a long period of operation, as the creation and deletion operations of the file are accumulated, the efficiency of system operation decreases with the increase of the operation time, and therefore, a method for effectively managing the memory space needs to be provided.
Disclosure of Invention
The application provides a memory space management method and device, which can randomly allocate idle memory blocks, realize wear balance of a memory and prolong the service life of the memory.
In a first aspect, a method for managing a memory space is provided, where the method is applied to a storage device in which a file system is built in a memory, where the memory is a non-volatile memory NVM, and the method includes:
receiving a file access request of a first thread, wherein the file access request comprises data of a first target file to be stored;
acquiring a first bitmap line segment tree corresponding to a first storage space to which a first thread belongs, wherein the memory comprises a plurality of storage spaces, each storage space is managed by adopting different bitmap line segment trees, the first bitmap line segment tree comprises N/S leaf nodes, N is used for indicating the number of storage blocks in the first storage space, S is used for indicating the number of storage blocks in the first storage space managed by each leaf node, N is not less than 2, S is not less than 1, and N is not less than S;
determining a first leaf node in the first bitmap line segment tree, wherein the first leaf node records information of a free storage block, and the first leaf node randomly searches for a Log from a root node of the first bitmap line segment tree to the direction of the leaf nodemThe node is obtained by searching after N/S times, wherein m is used for indicating the number of next-level child nodes of any node in the first bitmap line segment tree, and the value of m is not less than 2;
selecting at least one free storage block from S storage blocks managed by the first leaf node, wherein the use state information of the S storage blocks is recorded in the first leaf node;
storing data of the first target file into the at least one free storage block.
The memory space management method of the embodiment of the application receives a file access request of a first thread, determines a first leaf node in a first bitmap segment tree by acquiring the first bitmap segment tree corresponding to a first storage space to which the first thread belongs, wherein the first leaf node records information of an idle storage block, and randomly searches Log from a root node of the first bitmap segment tree to the leaf nodemAnd (N/S) searching is carried out, at least one free storage block is selected from S storage blocks managed by the first leaf node, the data of the first target file is stored in the at least one free storage block, and the free storage blocks can be randomly allocated, so that the wear balance of the memory is realized, and the service life of the memory is prolonged. In addition, in the application embodiment, if data in the storage block needs to be recovered, the bitmap corresponding to the corresponding block can be quickly located by using the bitmap line segment tree, and the elastic bitmap line segment tree is modified, so that the searching efficiency is improved, and the searching performance is better.
Optionally, the file access request may further include a file identifier. And the file identifier is a file handle distributed for the target file when the target file to be accessed by the file access request is opened by the process. In other words, the file identification is used to indicate the target file to be accessed. And the process can find the description information of the target file according to the file identification. Specifically, the file identifier may include a storage path of the target file, a file type (directory file or data file) of the target file, and the like.
In an embodiment of the present application, the first thread runs in the first storage space. The first memory space corresponds to a first bitmap segment tree.
Alternatively, the use state information of each of the S memory blocks may be represented using a bitmap. For example, 0 or 1 may be used to indicate whether a memory block is free. Specifically, for example, if the bit position is 0, it indicates that the memory block is free, and if the bit position is 1, it indicates that the memory block is occupied or does not satisfy the condition; alternatively, bit position 1 indicates that the memory block is free, and bit position 0 indicates that the memory block is occupied.
In some possible implementations, the method further includes:
acquiring a file creating request of the first thread, wherein the file creating request comprises path information of the first target file, the path information of the first target file comprises a first directory entry to be created, and the first directory entry is a directory entry to which the first target file belongs;
acquiring a directory entry of a parent directory where the first target file is located according to the path information;
determining a first node from a first space linked list of a directory entry of the parent directory according to the number of bytes needed by the first directory entry to be created, wherein the space linked list comprises P nodes, each node of the P nodes is used for indicating an idle interval in a first storage block corresponding to the directory entry of the parent directory, and the first node comprises address information of the first interval, wherein P is an integer not less than 1;
storing the first directory entry in the first extent to which the first node points.
In this embodiment of the present application, the space chain table includes P nodes, where each node in the P nodes may be used to indicate a free interval in the first storage block corresponding to the directory entry of the parent directory, where P is an integer not less than 1. That is, in the first storage block corresponding to the directory entry of the parent directory where the first target file is located, if there is a free interval (for example, the first interval), the first storage block may be used to store the directory entry of the first target file.
Optionally, the first node includes address information of the first interval. For example, the address information includes a start address and an end address, or the address information includes a start address and an offset (or length). In this way, the memory space management device may determine a specific position of the first interval in the space linked list according to the address information, so as to store the first directory entry in the first interval.
In some possible implementations, the method further includes:
acquiring a second file creating request from the first thread, wherein the second file creating request comprises second path information of a second target file, the second path information comprises a second directory entry to be created, and the second directory entry is a directory entry to which the second target file belongs;
acquiring a directory entry of a parent directory where the second target file is located according to the second path information;
when the plurality of storage blocks corresponding to the directory entry of the parent directory where the second target file is located do not have allocatable free intervals, acquiring the first bitmap segment tree corresponding to the first storage space to which the first thread belongs;
determining a second leaf node according to the first bitmap line segment tree, wherein the second leaf node records the information of allocable free storage blocks, and the second leaf node randomly searches for a Log from the root node of the first bitmap line segment tree to the leaf nodem(N/S) post-lookupObtaining;
selecting at least one free storage block from the first storage space according to the bitmap managed by the second leaf node;
storing the second directory entry to be created in the at least one free memory block.
Here, since the second create file request and the first create file request belong to the same thread (i.e. the first thread), when determining a free memory block for the second create file request, the search is still performed in the first bitmap segment tree corresponding to the first memory space to which the first thread belongs, that is, other leaf nodes, such as the second leaf node, of the first bitmap segment tree are searched.
In this embodiment of the present application, further, if a directory entry needs to be added to a target file in a certain thread, if there is no allocatable free block in a storage block corresponding to the directory entry of a parent directory where the target file is located, other leaf nodes may be searched in a bitmap line segment tree corresponding to a storage space to which the thread belongs to obtain a free storage block, so as to store the directory entry to be added.
In some possible implementations, the method further includes:
receiving a file deletion request, wherein the file deletion request comprises path information of a second target file to be deleted;
determining a directory entry of a parent directory to which the second target file belongs according to the path information of the second target file;
searching the identifier of a second storage block to which the second target file belongs and the address information of a second interval to which the directory entry of the second target file belongs according to the directory entry of the parent directory to which the second target file belongs;
calculating a hash value of the identity of the second storage block;
searching a second space linked list of the directory entry of the parent directory to which the second target file belongs according to the hash value, wherein the second space linked list is used for recording available idle intervals in a second storage block corresponding to the directory entry of the parent directory to which the second target file belongs;
and inserting a second node into the second space linked list, wherein the second node records the address information of the second interval.
It should be noted that "inserting a second node in the second spatial linked list" means that the second node is available, or the second node has been recycled, or it may be understood that a directory entry stored in the second node has been deleted. The second spatial linked list is used to record the intervals representing the availability.
Further, when the second interval is recovered (for example, the directory entry file1.txt is deleted in the second target file), the inode of the second target file and the storage block occupied by the second target file need to be recovered (which may also be understood as a delete storage block).
In some possible implementations, determining a first leaf node in the first bitmap segment tree includes:
randomly searching an ith intermediate node from a root node of the first bitmap line segment tree to a leaf node, wherein the ith intermediate node is obtained by randomly searching the root node for i times, and i is not less than 1 and not more than Logm(N/S),Logm(N/S) is a natural number greater than 1, and the number of available free storage blocks in the subtree of the ith intermediate node is recorded in the ith intermediate node;
when the number of available idle storage blocks in the subtree of the ith intermediate node does not meet the allocation condition, selecting the jth intermediate node from the sibling nodes of the ith intermediate node, wherein the subtree of the jth intermediate node contains the idle storage blocks meeting the allocation condition;
determining a first leaf node in the subtree of the jth intermediate node.
Here, the sibling node of the ith intermediate node may be understood as an intermediate node at the same level as the ith intermediate node.
That is, if the number of available free storage blocks in the subtree of an intermediate node does not satisfy the allocation condition, an intermediate node containing a free storage block satisfying the allocation condition may be selected from siblings of the intermediate node to store the directory entry of the target file.
In some possible implementations, selecting at least one free storage block from the first space according to a bitmap managed by the first leaf node includes:
and randomly selecting at least one idle storage block from the S storage blocks managed by the first leaf node.
In a second aspect, a memory space management apparatus is provided, configured to perform the method in the first aspect or any possible implementation manner of the first aspect. In particular, the apparatus comprises means or elements for performing the method of the first aspect described above or any possible implementation manner of the first aspect.
The third method provides a memory space management device. The apparatus includes a processor, a memory, and a communication interface. The processor is coupled to the memory and the communication interface. The memory is for storing instructions, the processor is for executing the instructions, and the communication interface is for communicating with other network elements under control of the processor. The processor, when executing the instructions stored by the memory, causes the processor to perform the method of the first aspect or any possible implementation of the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, which stores a program that causes a memory space management apparatus to execute the first aspect, and any one of its various implementations.
Drawings
Fig. 1 is an application scenario diagram of a next-generation nonvolatile memory provided in an embodiment of the present application.
Fig. 2 is a schematic layout diagram of a data structure in a file system according to an embodiment of the present application.
Fig. 3 is a schematic flow chart of a memory space management method according to an embodiment of the present application.
Fig. 4 is a schematic flow chart of a memory space management method according to another embodiment of the present application.
Fig. 5 is a schematic flow chart of a memory space management method according to still another embodiment of the present application.
FIG. 6 is a diagram illustrating an example of a spatial linked list according to an embodiment of the present application.
Fig. 7 is a schematic diagram of an example of a bitmap line segment tree according to an embodiment of the present application.
Fig. 8 is a schematic block diagram of a memory space management apparatus according to an embodiment of the present application.
Fig. 9 is a schematic block diagram of a memory space management device according to another embodiment of the present application.
Detailed Description
The technical solution in the present application will be described below with reference to the accompanying drawings.
The technical scheme of the embodiment of the application can be applied to a computer system based on a Non-Volatile Memory (NVM). The NVM runs a File System, such as a Byte-addressable Persistent Memory File System (BPFS), a Storage Class File Memory File System (SCMFS), a Persistent Memory File System (PMFS), and a Hierarchical File System (HiNFS), for example. The NVM is used to store data and metadata for a file system. Alternatively, the NVM can be combined with a Dynamic Random Access Memory (DRAM) to form a hybrid Memory, where temporary data generated by an application can be stored in the DRAM and persistent data generated by the application can be stored in the NVM.
Fig. 1 is a diagram of an application scenario of a next-generation non-volatile memory (NVM) according to an embodiment of the present disclosure. In one application scenario shown in fig. 1, NVM 102 is used as memory. As shown in fig. 1, the dynamic random access memory DRAM 103 is connected to a Central Processing Unit (CPU)101 through a memory bus. An operating system and other software programs are installed in the Central Processing Unit (CPU)101, so that the Central Processing Unit (CPU)101 can access the DRAM 103, the disk 104, and the NVM 102. It is understood that, in the embodiment of the present application, the Central Processing Unit (CPU)101 is only one example of a processor. In addition to the Central Processing Unit (CPU)101, the processor may be Another Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement the embodiments of the present Application.
DRAM 103 has an advantage of high access speed, and CPU 101 can access DRAM 103 at high speed to perform a read or write operation on any memory cell in DRAM 103, and thus DRAM 103 is generally used as a main memory. DRAM 103 is typically used to store various running software in the operating system, input and output data, and information exchanged with external memory, etc. However, DRAM 103 is volatile, and when the computer is powered down, the information in DRAM 103 is no longer preserved.
The disk 104 may serve as an external storage for storing data. Of course, it is understood that the external memory may be a nonvolatile memory such as a Solid State Disk (SSD) in addition to the magnetic disk 104. Generally, a storage medium as an external memory needs to have a nonvolatile characteristic, and data stored in the external memory is still stored after a computer is powered off. In addition, the storage capacity of the external memory is large.
The new non-volatile memory (NVM)102 may be directly attached to the memory bus as DRAM 103. A Central Processing Unit (CPU)101 can access the NVM 102 through a Memory Management Unit (MMU), and quickly store data in the NVM 102 or read data from the NVM 102. It is understood that when NVM 102 is used directly on a memory bus, the latency of data access (latency) can be reduced as with DRAM 103. Also, since the NVM 102 has non-volatility, data can be better saved compared to the DRAM 103.
It should be noted that the nonvolatile memory capable of being suspended on the memory bus described in the embodiment of the present application may include: phase Change Memories (PCMs), Resistive Random Access Memories (RRAMs), Magnetic Random Access Memories (MRAMs), Ferroelectric Random Access Memories (FRAMs), and other novel nonvolatile memories. These new NVM types have the characteristics of fast access and non-volatile, and can address by Byte (Byte) and write data in bits (bit) into non-volatile memory.
It is understood that the application scenario shown in fig. 1 may be applied to a storage device that establishes a file system based on a memory. In the embodiment of the present invention, the storage device may include a computer, a server, a storage array, and the like, which establish a file system in a memory. The application scenario of the embodiment of the present invention is not limited herein. It should be noted that, in the embodiments of the present invention, the memories are all NVM unless otherwise specified.
In the embodiment of the present application, the NVM 102 stores therein a file system and a file. As one skilled in the art will appreciate, a file system is a software structure in an operating system that is responsible for managing and storing file information. From a system perspective, a file system is a system that organizes and allocates space of a file storage device, is responsible for file storage, and protects and retrieves stored files. The central concept of the file system is a Super Block (SB), an inode (inode), a data Block (data Block), a directory Block (directory Block), and an indirection Block. The superblock includes general information of the file system, such as the size of the file system. All information of one file except for the file name is included in the inode. The file name is stored in a directory block along with the inode number, and the directory entry includes the file name and the inode number of the file. There is only a small amount of data block space in the inode, and if more is needed, pointer space to data blocks is dynamically allocated, and these dynamically allocated blocks are indirect blocks. In the embodiment of the present application, the NVM 102 may also store data and metadata of files.
The following is referred to for a detailed description of the file system:
the file system is used for storing, organizing data, and defining access policies for files. Data in a file system is divided into metadata and data, where data refers to actual data in a general file, and metadata refers to system data used to describe characteristics of a file. The metadata may include, for example, access rights, file owner, and distribution information (inode) of file data blocks, and the like. A user needs to manipulate a file and first obtain its metadata to locate the file and obtain the content or related attributes of the file. Metadata is data of data, and each file corresponds to a metadata structure in the field of file systems. This structure includes attributes of the file, such as time of file creation, modification and access, file owner, permissions, mapping information, etc.
The file system consists of three parts: the interface to the file system, the software set that manipulates and manages the files, the file data, and the attributes. When a process reads or writes a file, an operating system opens a target file to be accessed by the process according to a file name, and then performs read operation or write operation on the opened target file according to a received read request or write request. The file name may be a full path name of the file, and is a logical description of location information of the target file in the disk. For example, the file name of the target file may be: d \ FILE1. In the process of opening the file through the full path name of the file, the disk needs to be searched layer by layer according to the full path of the file, the disk is read continuously, and a corresponding data structure is created in the memory to represent the directory structure of the target file. After a file is opened, in the process of accessing a target file according to a read request or a write request, if a read operation or a write operation is also performed on the file according to the full path name of the target file, frequent disk reading or disk writing is involved, the process is complex, and the processing time is long.
Therefore, in practical applications, during the process of opening the target file by the operating system, the operating system may allocate a file handle (also referred to as a file descriptor) to the target file accessed by the process, and maintain a file handle array inside the process. The file handle may be represented by a number, for example, the file handle may be: fd 0, fd1 or fd 2. The file handle array stores pointers to the file description information. The file description information includes pointers to information such as a file directory structure, metadata (metadata), and access positions. The file directory structure is used for describing the logic position of the file, the file directory structure is stored in the memory, and the process can locate the position of the target file through the file directory structure. Metadata is data used to describe file data, and specifically, the metadata includes information about the organization, data fields, and relationships of the file data. The access position is used to indicate the starting position that the process is currently accessing. The access location may be a logical location. In general, the access location information may be 0 for representing that the access is started from the start address of the file. The file access position can also be set to other positions except 0 through system call in the process of opening the file. During the process of accessing the file (reading/writing the file), the process can also set the file access position through system call according to the file handle. As known to those skilled in the art, in the case of random reading and writing, the access position may be any one set by a system call, and in the case of sequential reading and writing, the access position currently accessed is the end position of the last access. In the process of performing read operation or write operation on the target file, the process may find the target file description information in the file handle array maintained by the process according to the file handle of the target file. And searching the metadata, the access position and other information of the file in the file description information, thereby realizing the read operation or write operation of the target file. It is understood that the file handle is a file identifier for identifying the target file in the process of reading/writing the target file by the current process. In this embodiment of the application, the file identifier may also be a file descriptor other than the file handle, which is not limited herein as long as the process can identify the target file through the file identifier and find the description information of the target file.
In order to facilitate understanding of the technical solutions provided by the embodiments of the present application, the following embodiments of the present application are described with reference to fig. 2. Fig. 2 shows a schematic layout of a data structure in a file system according to an embodiment of the present application. As shown in FIG. 2, the embodiment of the present application provides a plurality of memory spaces (e.g. memory space 0 to memory space N shown in FIG. 2) for an internal memory (e.g. NVM)Each storage space in (a) is managed individually using a Flexible Bitmap Segment Tree (FBST). The corresponding physical address space of the NVM can include: super Block (SB), Log Area, and Data Area (Data Area). Wherein SB can hold multiple copies; the Log area is used for writing logs. In the Data Area space, some free space is available for storing directory entries, e.g. by
Figure BDA0001254507460000071
Representing the storage space in which the directory entry is stored. Further, in the embodiment of the present application, a storage space for storing directory entries is to be provided with a hash index table and a space linked list
Figure BDA0001254507460000075
And (6) managing. In FIG. 2, using
Figure BDA0001254507460000072
Indicating the section in which the directory entry data is stored
Figure BDA0001254507460000073
Indicating free space, a free interval in the storage space where the directory entry needs to be reclaimed
Figure BDA0001254507460000074
In time, a hash index table may be used for lookup. It should be understood that the block diagram in fig. 2 is only for the convenience of understanding of those skilled in the art, and does not limit the embodiments of the present application.
The memory space management method in the embodiment of the present application may be executed by a Processor (Processor) in a computer System, where an Operating System (OS) is run in the Processor, the computer System further includes a memory, and a file System is established in the memory. For convenience of description, the following description will be given taking a processor as an example of a CPU.
Fig. 3 shows a schematic flow chart of a memory space management method 300 according to an embodiment of the present application. The method 300 may be performed by a CPU. The method 300 is applied to a storage device with a file system built in a memory, wherein the memory is a non-volatile memory NVM, and the method 300 includes:
s310, receiving a file access request of a first thread, wherein the file access request comprises data of a first target file to be stored;
optionally, the file access request may include a write request, where the write request is used to write data of the first target file to be stored. It should be understood that the file access request may also optionally include an access request such as a file read request.
Optionally, the file access request may further include a file identifier. And the file identifier is a file handle distributed for the target file when the target file to be accessed by the file access request is opened by the process. In other words, the file identification is used to indicate the target file to be accessed. And the process can find the description information of the target file according to the file identification. Specifically, the file identifier may include a storage path of the target file, a file type (directory file or data file) of the target file, and the like. Wherein a process may include multiple threads, each thread being an execution entity in the process, each thread having a unique identifier.
S320, obtaining a first bitmap line segment tree corresponding to a first storage space to which the first thread belongs, where the memory includes multiple storage spaces, each storage space is managed by using a different bitmap line segment tree, and the first bitmap line segment tree includes N/S leaf nodes, where N is used to indicate the number of storage blocks included in the first storage space, S is used to indicate the number of storage blocks in the first storage space managed by each leaf node, N is not less than 2, S is not less than 1, and N is not less than S;
because the file access request includes the file identifier, the CPU can determine the thread to which the file access request belongs according to the file identifier. And each thread has a unique identifier corresponding to the storage space, and the CPU can acquire the storage space corresponding to the thread according to the identifier of the thread and then obtain the bitmap line segment tree corresponding to the storage space. In other words, the CPU knows the memory space in which a thread runs after obtaining a file access request from a thread.
In an embodiment of the present application, the first thread runs in the first storage space. The first memory space corresponds to a first bitmap segment tree. The first storage space comprises N (N is not less than 2) storage blocks (Block), the first bitmap segment tree comprises N/S leaf nodes, and each leaf node manages S storage blocks. Specifically, each leaf node may represent, by a bitmap (bitmap), the use state information of each of S (S is not less than 1) memory blocks it manages. For example, each of the S storage blocks uses a bit (bit) to represent on the bitmap, and the bit number of the bitmap managed by the leaf node is S bits.
It should be noted that the intermediate node in the tree of the first bitmap segment tree at least includes information about the number of free memory blocks indicated by the subtree of the intermediate node, so as to facilitate subsequent searching for free memory blocks. Generally, the intermediate node does not include location information of free memory blocks.
Alternatively, the first bitmap line segment tree may be the elastic bitmap line segment tree FBST in fig. 2 or fig. 7 below. The elastic bitmap line segment tree can realize the height change of the tree by adjusting the bitmap digit number contained in the leaf node, namely the value size of S. The height change of the elastic bitmap line segment tree can affect the time overhead of memory block allocation and/or memory block recovery in the memory. The height change of the elastic bitmap line segment tree can be flexibly configured according to the size of the memory space of the memory.
It should be understood that if there are multiple threads in memory, each thread running in a separate memory space, each memory space may be managed using a bitmap line segment tree structure.
S330, determining a first leaf node in the first bitmap line segment tree, wherein the first leaf node records information of a free storage block, and the first leaf node randomly searches for a Log from a root node of the first bitmap line segment tree to the direction of the leaf nodemObtained by (N/S) post-searches, where m is used to indicateThe number of the next-level child nodes of any one node in the first bitmap segment tree is not less than 2;
here, the first leaf node is a random walk Log from the root node of the first bitmap line segment tree in the direction of the leaf nodemAnd (N/S) searching, namely searching by adopting a random walk algorithm. Wherein m is used for indicating the number of next-level nodes of any one node of the first bitmap line segment tree. For example, when m is 2, the first bitmap segment tree is a binary tree, and the number of nodes at the next level of any node in the first bitmap segment tree is 2.
It should be noted that, the "randomly walking Log from the root node to the leaf node of the first bitmap line segment tree" is describedmThe step (N/S) of searching for the first leaf node necessarily refers to the step-by-step searching from the root node to the child node of the next stage, but not the searching of the node of the same stage.
Optionally, the first leaf node may record information of a free storage block in the S storage blocks it manages through a bitmap.
S340, selecting at least one free storage block from the S storage blocks managed by the first leaf node, where the first leaf node records the use state information of the S storage blocks;
alternatively, the use state information of each of the S memory blocks may be represented using a bitmap. For example, 0 or 1 may be used to indicate whether a memory block is free. Specifically, for example, if the bit position is 0, it indicates that the memory block is free, and if the bit position is 1, it indicates that the memory block is occupied or does not satisfy the condition; alternatively, bit position 1 indicates that the memory block is free, and bit position 0 indicates that the memory block is occupied.
Optionally, S340 may include:
and randomly selecting at least one idle storage block from the S storage blocks managed by the first leaf node.
Specifically, the first leaf node records the use status (i.e., whether it is free) information of S storage blocks in a bitmap manner. The CPU may randomly select at least one free memory block among the S memory blocks according to the use state information of each memory block.
S350, storing the data of the first target file into the at least one free storage block.
Optionally, when the data of the first target file is stored in the at least one free storage block, the CPU may record information thereof in an index node (inode) corresponding to the first target file.
In the memory space management method of the embodiment of the application, a CPU receives a file access request of a first thread, acquires a first bitmap line segment tree corresponding to a first storage space to which the first thread belongs, determines a first leaf node in the first bitmap line segment tree, where the first leaf node records information of an idle storage block, and randomly searches for a Log from a root node of the first bitmap line segment tree to the leaf nodemAnd (N/S) searching is carried out, at least one free storage block is selected from S storage blocks managed by the first leaf node, the data of the first target file is stored in the at least one free storage block, and the free storage blocks can be randomly allocated, so that the wear balance of the memory is realized, and the service life of the memory is prolonged. In addition, in the application embodiment, if data in the storage block needs to be recovered, the bitmap corresponding to the corresponding block can be quickly located by using the bitmap line segment tree, and the elastic bitmap line segment tree is modified, so that the searching efficiency is improved, and the searching performance is better.
The memory space management method of the embodiment of the application can be executed by a CPU, wherein an operating system runs in the CPU. In the process of accessing a file system, a CPU (central processing unit) allocates a storage space of a memory (NVM) by using the memory space management method of the embodiment of the application, so that wear balance of the NVM is realized, and the performance of the file system is improved.
It should be understood that each file in the file system corresponds to an inode, and therefore, the inode corresponding to the target file can be obtained by determining that the target file is stored in the storage block in which the file system is located.
The above describes storing data of the first target file, and further, if a free interval exists in the storage block storing the first target file, a directory file (directory entry) of the first target file may also be created, and the directory file (directory entry) of the first target file may be stored in the free interval. A memory space management method 400 according to another embodiment of the present application will be described with reference to fig. 4. The method 400 may be performed by a CPU. As shown in fig. 4, the method 400 includes:
s410, acquiring a file creating request of the first thread, wherein the file creating request comprises path information of the first target file, the path information of the first target file comprises a first directory entry to be created, and the first directory entry is a directory entry to which the first target file belongs;
s420, acquiring a directory entry of a parent directory where the first target file is located according to the path information;
s430, determining a first node from a first space chain table of a directory entry of the parent directory according to the number of bytes needed by the first directory entry to be created, wherein the first space chain table comprises P nodes, each node of the P nodes is used for indicating an idle interval in a first storage block corresponding to the directory entry of the parent directory, and the first node comprises address information of the first interval, wherein P is an integer not less than 1;
s440, storing the first directory entry in the first interval pointed to by the first node.
Specifically, the CPU may further obtain a file creation request of the first thread, where the file creation request includes storage path information of the first target file, where the path information of the first target file includes a first directory entry to be created, and for example, a full path of the first target file is: txt, the first directory entry to be created is: txt. The CPU may obtain, according to the path information of the first target file, a directory entry of a parent directory in which the first target file is located, where, for example, the directory entry of the parent directory in which the first target file is located is: and/dir 1/123. In addition, specifically, the CPU may calculate a required number of bytes according to a length of a name string of a first directory entry to be created, and then determine a first node from a space linked list (such as a first space linked list) of the directory entry of the parent directory, so as to store the first directory entry in a first interval pointed to by the first node. Alternatively, the CPU may point to the first section through a pointer.
In this embodiment of the present application, the space chain table includes P nodes, where each node in the P nodes may be used to indicate a free interval in the first storage block corresponding to the directory entry of the parent directory, where P is an integer not less than 1. That is, in the first storage block corresponding to the directory entry of the parent directory where the first target file is located, if there is a free interval (for example, the first interval), the first storage block may be used to store the directory entry of the first target file.
Alternatively, the first interval may be described by a length or a range (extend), and may be specifically represented by (first, last). Where extend is a concept in a file system.
Optionally, the first node includes address information of the first interval. For example, the address information includes a start address and an end address, or the address information includes a start address and an offset (or length). In this way, the CPU can determine the specific position of the first interval in the space linked list according to the address information, thereby storing the first directory entry in the first interval.
The above describes an embodiment of storing a directory file (directory entry) of a target file into a free interval (i.e., a first interval) in a storage block corresponding to a directory entry of a parent directory in which the target file is located. Optionally, if there is no allocable free space in the plurality of storage blocks corresponding to the directory entry of the parent directory where the target file is located, it is necessary to determine a free storage block for the directory entry to be created in the first bitmap segment tree. The second create file request of the first thread is explained as an example.
Optionally, as an embodiment, the method 300 or the method 400 may further include:
acquiring a second file creating request from the first thread, wherein the second file creating request comprises second path information of a second target file, the second path information comprises a second directory entry to be created, and the second directory entry is a directory entry to which the second target file belongs;
acquiring a directory entry of a parent directory where the second target file is located according to the second path information;
when the plurality of storage blocks corresponding to the directory entry of the parent directory where the second target file is located do not have allocatable free intervals, acquiring the first bitmap segment tree corresponding to the first storage space to which the first thread belongs;
determining a second leaf node according to the first bitmap line segment tree, wherein the second leaf node records the information of allocable free storage blocks, and the second leaf node randomly searches for a Log from the root node of the first bitmap line segment tree to the leaf nodemObtained by searching after (N/S) times;
selecting at least one free storage block from the first storage space according to the bitmap managed by the second leaf node;
storing the second directory entry to be created in the at least one free memory block.
Specifically, the CPU may further obtain a second file creation request of the first thread, where the second file creation request includes a second target file, the second path information includes a second directory entry to be created, and the second directory entry is a directory entry to which the second target file belongs. And the CPU can acquire the directory entry of the parent directory where the second target file is located according to the second path information. If it is determined that there is no allocatable free space in the plurality of storage blocks corresponding to the directory entry of the parent directory where the second target file is located, the first bitmap segment tree corresponding to the first storage space to which the first thread belongs needs to be acquired. Similarly, the CPU searches the allocatable free memory blocks in the first bitmap line segment tree to determine a second leaf node, where the determination of the second leaf node is similar to the process of the determination of the first leaf node. And finally, the CPU selects at least one free storage block in the first storage space according to the bitmap managed by the second leaf node, and stores the second directory entry to be created.
Here, since the second create file request and the first create file request belong to the same thread (i.e. the first thread), when determining a free memory block for the second create file request, the search is still performed in the first bitmap segment tree corresponding to the first memory space to which the first thread belongs, that is, other leaf nodes, such as the second leaf node, of the first bitmap segment tree are searched.
It should be understood that, in the embodiment of the present application, the numbers "first" or "second" … are introduced only for distinguishing different objects, for example, for distinguishing different "object files", or for distinguishing different "leaf nodes", or for distinguishing different "directory entries", or for distinguishing different "create file requests", or for distinguishing different "space linked lists", or for distinguishing different "storage blocks", and the like, and are not limited to the embodiment of the present application.
Therefore, in the embodiment of the present application, further, if a directory entry needs to be added to a target file in a certain thread, if there is no allocable free block in a storage block corresponding to the directory entry of a parent directory where the target file is located, other leaf nodes may be searched in a bitmap line segment tree corresponding to a storage space to which the thread belongs to obtain a free storage block, so as to store the directory entry to be added.
Embodiments of storing data or directory entries for a target file are described above. Further, an embodiment of the present application further provides a memory space management method, which may manage a storage space to be recovered through a space linked list and a hash index, and more specifically, insert the storage space to be recovered into the space linked list in an interval or node manner. A memory space management method 500 according to still another embodiment of the present application will be described with reference to fig. 5. The method 500 may be performed by a CPU. As shown in fig. 5, the method 500 includes:
s510, receiving a file deleting request, wherein the file deleting request comprises path information of a second target file to be deleted;
s520, determining a directory entry of a parent directory to which the second target file belongs according to the path information of the second target file;
s530, searching the identifier of a second storage block to which the second target file belongs and the address information of a second interval to which the directory entry of the second target file belongs according to the directory entry of the parent directory to which the second target file belongs;
s540, calculating a hash value of the identifier of the second storage block;
s550, searching a second space linked list of the directory entry of the parent directory to which the second target file belongs according to the hash value, wherein the second space linked list is used for recording available idle intervals in a second storage block corresponding to the directory entry of the parent directory to which the second target file belongs;
and S560, inserting a second node into the second space linked list, wherein the second node records the address information of the second interval.
Specifically, the CPU may further receive a file deletion request, where the file deletion request may include path information of a second target file to be deleted, and then determine, according to the path information of the second target file, a directory entry of a parent directory to which the second target file belongs. For example, the full path of the second target file is: and/dir 2/123/file1.txt, the directory entry of the parent directory to which the second target file belongs is: and/dir 2/123. The CPU may search, according to the directory entry of the parent directory to which the second target file belongs, an identifier of a second storage block to which the second target file belongs, and address information of a second interval to which the directory entry of the second target file belongs. For example, the identifier of the second storage block to which the second target file belongs is represented by (block _ index _ nr). Optionally, the address information includes a start address of the second section and an offset address of the second section, such as (in _ block _ offset _ start), (in _ block _ offset _ len); or the address information includes a start address of the second section and an end address of the second section, for example, denoted as (in _ block _ offset _ start) and (in _ block _ offset _ end). And then, the CPU obtains a hash value of the identifier of the second storage block through hash calculation, wherein the hash value can be understood as address information of the second interval, and then searches a second space linked list according to the hash value, wherein the second space linked list is used for recording available idle intervals in the second storage block corresponding to directory entries of a parent directory to which the second target file belongs. Finally, the CPU may insert a second node into the second space linked list, where the second node records address information of a second interval, that is, the address information indicates that the second interval has been released and is recycled into the second space linked list. Therefore, under the condition that the number of certain directory files in the file system is large, the space linked list is searched through the hash value, the system bottleneck caused by the fact that the related operations of the directory items are not efficient enough can be solved, and the efficient distribution and release of the storage space can be achieved.
Specifically, for example, the key of the hash value is the block number of the second storage block, and the value is the starting address of the second storage block in the second space linked list.
It should be noted that "inserting a second node in the second spatial linked list" means that the second node is available, or the second node has been recycled, or it may be understood that a directory entry stored in the second node has been deleted. Wherein the second spatial linked list is used for recording the available intervals.
Further, when the second interval is recovered (for example, the directory entry file1.txt is deleted in the second target file), the inode of the second target file and the storage block occupied by the second target file need to be recovered (which may also be understood as a delete storage block).
Optionally, if the recovered second interval and its adjacent interval in the second spatial chain table are completely adjacent, the second interval and its adjacent interval may be merged.
Optionally, when the directory entry is deleted, the bit position of the corresponding leaf node on the bitmap line segment tree may be set to 1 (or may be other identifiers), which indicates that the storage block is in an idle state.
FIG. 6 is a diagram illustrating an example of a spatial linked list according to an embodiment of the present application. As shown in fig. 6, the space chain table includes a plurality of nodes (node 0, node 1, node 2, and node 3, respectively), and each node is configured to indicate an idle interval in a corresponding Block. The 4 storage blocks corresponding to the 4 nodes are respectively: block0, Block1, Block2, and Block 3. When a file deletion request is received, the file deletion request comprises path information of a target file in Block2 to be deleted, a CPU searches a directory entry of a parent directory to which the target file belongs according to the path information of the target file, and searches an Identifier (ID) of a Block2 to which the target file belongs and an address (such as a starting address and an offset) of an interval to which the directory entry of the target file belongs according to the directory entry of the parent directory to which the target file belongs. In FIG. 6, using
Figure BDA0001254507460000121
Indicates the interval to be recovered in Block 2. The CPU searches the space linked list according to the hash value of the identification of Block2 by calculating the hash value. Then, after the section to be recovered is found, the CPU can recover the section to be recovered
Figure BDA0001254507460000122
The directory entry stored in the space list is deleted to release the interval, and then the interval to be recovered is inserted into the space list
Figure BDA0001254507460000123
Further, the interval to be recovered can be divided into
Figure BDA0001254507460000124
And merging with its neighbors for subsequent allocation.
Therefore, in the embodiment of the present application, further, by calculating a hash value of the identifier of the second storage block, searching the second space linked list of the directory entry of the parent directory to which the second target file belongs according to the hash value, and inserting the second node into the second space linked list, the idle interval can be recovered, thereby implementing efficient allocation and release of the storage space.
Optionally, as an embodiment, determining a first leaf node in the first bitmap segment tree includes:
searching the ith intermediate node from the root node of the first bitmap line segment tree to the leaf node at random, wherein the ith intermediate node is obtained after searching the root node at random for i times, and i is not less than 1 and not more than Logm(N/S),Logm(N/S) is a natural number greater than 1, and the number of available free storage blocks in the subtree of the ith intermediate node is recorded in the ith intermediate node;
when the number of available idle storage blocks in the subtree of the ith intermediate node does not meet the allocation condition, selecting the jth intermediate node from the sibling nodes of the ith intermediate node, wherein the subtree of the jth intermediate node contains the idle storage blocks meeting the allocation condition;
determining a first leaf node in the subtree of the jth intermediate node.
Specifically, the CPU may search from a root node of the first bitmap line segment tree step by step in the direction of the leaf node, and randomly search for the ith intermediate node. That is, the ith intermediate node is obtained by randomly searching the root node i times. The ith intermediate node records the number of available free storage blocks in the subtree but does not include the position information of the available free storage blocks. If the capacity of the number of available free storage blocks in the subtree of the ith intermediate node is insufficient, the data of a new file to be created or a new target file cannot be accommodated, or the requirement is not met, the CPU may select the jth intermediate node from the sibling nodes of the ith intermediate node, wherein the subtree of the jth intermediate node contains the free storage blocks meeting the distribution condition, and then continue searching (Log) in the subtree of the jth intermediate node in the direction of the leaf nodem(N/S) -i) times to find the firstAnd the leaf node selects at least one free storage block from the storage blocks managed by the first leaf node, and stores the data of the file to be created or the target file into the at least one free storage block.
Here, the sibling node of the ith intermediate node may be understood as an intermediate node located at the same level as the ith intermediate node.
That is, if the number of available free storage blocks in the subtree of an intermediate node does not satisfy the allocation condition, an intermediate node containing a free storage block satisfying the allocation condition may be selected from the siblings of the intermediate node to store the directory entry of the target file or the data of the target file.
The structure of the bitmap line segment tree will be described below with reference to fig. 7. FIG. 7 shows a schematic diagram of an example of a bitmap line segment tree according to an embodiment of the present application. It should be understood that fig. 7 is only introduced to facilitate understanding of the bitmap line segment tree by those skilled in the art, and does not limit the embodiments of the present application, and in practice, the bitmap line segment tree may have other forms of transformation or adjustment.
As shown in FIG. 7, the bitmap line segment tree is a 2-way tree, LogmM in (N/S) takes the value of 2. The bitmap line segment tree management method includes that the number N of storage blocks managed by the bitmap line segment tree is 64 (including the storage block numbers 0-63), the bitmap line segment tree management method specifically includes that N/S of the storage blocks managed by each leaf node is 8 leaf nodes, each leaf node records the use state information of 8 storage blocks by adopting a bitmap method, and the height of the bitmap line segment tree is Log2(N/S) ═ 3. Optionally, the height of the bitmap line segment tree may be flexibly adjusted or changed according to the value of S. Specifically, the root node of the bitmap line segment tree is the top level (number 0-number 63); the leaf nodes of the bitmap line segment tree are at the lowest level, that is, 8 storage blocks managed by each leaf node in 8 leaf nodes are: number 0-number 7, number 8-number 15, number 16-number 23, number 24-number 31, number 32-number 39, number 40-number 47, number 48-number 55, number 56-number 63. Each leaf node adopts
Figure BDA0001254507460000131
Representing the use state of each of the 8 memory blocks; two levels between the root node and the leaf node of the bitmap line segment tree are intermediate nodes. Alternatively, the intermediate nodes (numbered 0-31) may be referred to as left child nodes; the intermediate nodes (number 32-number 63) may be referred to as right child nodes. The left child node and the right child node are brother nodes. When the number of the available storage blocks in the subtree of the left child node does not meet the allocation condition, the CPU can select the leaf node from the right child node and then select the free storage blocks from the leaf node on the assumption that the right child node contains the free storage blocks meeting the allocation condition.
The method for managing the memory space can randomly distribute the idle memory blocks, thereby realizing wear balance of the memory and prolonging the service life of the memory. In addition, in the application embodiment, if data in the storage block needs to be recovered, the bitmap corresponding to the corresponding block can be quickly located by using the bitmap line segment tree, and the elastic bitmap line segment tree is modified, so that the searching efficiency is improved, and the searching performance is better. Further, the free interval can be recovered, thereby realizing efficient allocation and release of the storage space.
It should be understood that the technical solution provided in the embodiment of the present application may be applied to a Linux operating system, and may also be applied to other operating systems in which each process has its own dedicated space.
It should also be understood that the technical scheme provided by the embodiment of the application does not need to modify hardware, and has strong universality.
The method for memory space management according to the embodiment of the present application is described in detail above with reference to fig. 1 to 7, and the memory space management device according to the embodiment of the present application is described in detail below with reference to fig. 8 to 9. It should be understood that the memory space management apparatus in the embodiment of the present application may be integrated in a storage device in a module form, or may exist independently, which is not limited thereto.
Fig. 8 shows a schematic block diagram of a memory space management apparatus 800 according to an embodiment of the present application, which is applied to a storage device with a file system built in a memory, where the memory is a non-volatile memory NVM. Alternatively, the device may be self-contained. As shown in fig. 8, the apparatus 800 includes:
a receiving module 810, configured to receive a file access request of a first thread, where the file access request includes data of a first target file to be stored;
an obtaining module 820, configured to obtain a first bitmap line segment tree corresponding to a first storage space to which the first thread belongs, where the memory includes multiple storage spaces, each storage space is managed by using a different bitmap line segment tree, and the first bitmap line segment tree includes N/S leaf nodes, where N is used to indicate the number of storage blocks included in the first storage space, S is used to indicate the number of storage blocks in the first storage space managed by each leaf node, N is not less than 2, S is not less than 1, and N is not less than S;
a determining module 830, configured to determine a first leaf node in the first bitmap line segment tree, where the first leaf node records information of a free storage block, and the first leaf node is a Log randomly searched from a root node of the first bitmap line segment tree to a leaf nodem(N/S) times of searching, wherein m is used for indicating the number of next-level child nodes of any node in the first bitmap line segment tree, and the value of m is not less than 2;
a selecting module 840, configured to select at least one free storage block from S storage blocks managed by the first leaf node, where the first leaf node records the use status information of the S storage blocks;
a storage module 850, configured to store the data of the first target file into the at least one free storage block.
Optionally, as an embodiment, the obtaining module 820 is further configured to:
acquiring a file creating request of the first thread, wherein the file creating request comprises path information of the first target file, the path information of the first target file comprises a first directory entry to be created, and the first directory entry is a directory entry to which the first target file belongs; acquiring a directory entry of a parent directory where the first target file is located according to the path information;
the determining module 830 is further configured to determine a first node from a first space chain table of a directory entry of the parent directory according to the number of bytes needed by the first directory entry to be created, where the space chain table includes P nodes, each node of the P nodes is used to indicate a free interval in a first storage block corresponding to the directory entry of the parent directory, and the first node includes address information of the first interval, where P is an integer not less than 1;
the storing module 850 is further configured to store the first directory entry in the first span pointed to by the first node.
Optionally, as an embodiment, the obtaining module 820 is further configured to:
acquiring a second file creating request from the first thread, wherein the second file creating request comprises second path information of a second target file, the second path information comprises a second directory entry to be created, and the second directory entry is a directory entry to which the second target file belongs; acquiring a directory entry of a parent directory where the second target file is located according to the second path information; when the plurality of storage blocks corresponding to the directory entry of the parent directory where the second target file is located do not have allocatable free intervals, acquiring the first bitmap segment tree corresponding to the first storage space to which the first thread belongs;
the determining module 830 is further configured to determine a second leaf node according to the first bitmap line segment tree, where the second leaf node records information of allocatable free storage blocks, and the second leaf node randomly searches for a Log from a root node of the first bitmap line segment tree to a leaf nodemObtained by searching after (N/S) times;
the selecting module 840 is further configured to select at least one free storage block from the first storage space according to the bitmap managed by the second leaf node;
the storage module 850 is further configured to store the second directory entry to be created in the at least one free storage block.
As shown in fig. 9, optionally, as an embodiment, the apparatus 800 includes:
a receiving module 810, configured to receive a file deletion request, where the file deletion request includes path information of a second target file to be deleted;
a determining module 830, configured to determine, according to the path information of the second target file, a directory entry of a parent directory to which the second target file belongs;
a searching module 860, configured to search, according to a directory entry of a parent directory to which the second target file belongs, an identifier of a second storage block to which the second target file belongs, and address information of a second interval to which the directory entry of the second target file belongs;
a calculating module 870 for calculating a hash value of the identity of the second memory block;
the searching module 860 is further configured to search, according to the hash value, a second space linked list of directory entries of a parent directory to which the second target file belongs, where the second space linked list is used to record available idle intervals in a second storage block corresponding to directory entries of the parent directory to which the second target file belongs;
an inserting module 880, configured to insert a second node into the second space linked list, where address information of the second interval is recorded in the second node.
Optionally, as an embodiment, the determining module 830 is specifically configured to:
randomly searching an ith intermediate node from a root node of the first bitmap line segment tree to a leaf node, wherein the ith intermediate node is obtained by randomly searching the root node for i times, and i is not less than 1 and not more than Logm(N/S),Logm(N/S) is a natural number greater than 1, and the number of available free storage blocks in the subtree of the ith intermediate node is recorded in the ith intermediate node;
when the number of available idle storage blocks in the subtree of the ith intermediate node does not meet the allocation condition, selecting the jth intermediate node from the sibling nodes of the ith intermediate node, wherein the subtree of the jth intermediate node contains the idle storage blocks meeting the allocation condition;
determining the first leaf node in the subtree of the jth intermediate node.
Optionally, as an embodiment, the selecting module 840 is specifically configured to:
and randomly selecting at least one idle storage block from the S storage blocks managed by the first leaf node.
It should be understood that the above-mentioned and other operations and/or functions of the modules in the memory space management apparatus 800 according to the embodiment of the present application are respectively for implementing the corresponding flows of the methods (including the methods in fig. 3, fig. 4, and fig. 5) in the foregoing embodiments, and are not described herein again for brevity.
The memory space management device 800 according to the embodiment of the present application receives a file access request of a first thread, determines a first leaf node in a first bitmap segment tree by obtaining the first bitmap segment tree corresponding to a first storage space to which the first thread belongs, where the first leaf node records information of an idle storage block, and randomly searches for a Log from a root node of the first bitmap segment tree to the leaf nodemAnd (N/S) searching is carried out, at least one free storage block is selected from S storage blocks managed by the first leaf node, the data of the first target file is stored in the at least one free storage block, and the free storage blocks can be randomly allocated, so that the wear balance of the memory is realized, and the service life of the memory is prolonged.
Further, the idle interval can be recovered by calculating the hash value of the identifier of the second storage block, searching the second space chain table of the directory entry of the parent directory to which the second target file belongs according to the hash value, and inserting the second node into the second space chain table, thereby realizing efficient allocation and release of the storage space.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the application may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application, which essentially or partly contribute to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Claims (12)

1. A memory space management method is applied to a storage device with a file system built in a memory, wherein the memory is a nonvolatile memory (NVM), and the method is characterized by comprising the following steps:
receiving a file access request of a first thread, wherein the file access request comprises data of a first target file to be stored;
acquiring a first bitmap line segment tree corresponding to a first storage space to which a first thread belongs, wherein the memory comprises a plurality of storage spaces, each storage space is managed by adopting different bitmap line segment trees, the first bitmap line segment tree comprises N/S leaf nodes, N is used for indicating the number of storage blocks in the first storage space, S is used for indicating the number of storage blocks in the first storage space managed by each leaf node, N is not less than 2, S is not less than 1, and N is not less than S;
determining a first leaf node in the first bitmap line segment tree, wherein the first leaf node records information of a free storage block, and the first leaf node randomly searches for a Log from a root node of the first bitmap line segment tree to the direction of the leaf nodem(N/S) times of searching, wherein m is used for indicating the number of next-level child nodes of any node in the first bitmap line segment tree, and the value of m is not less than 2;
selecting at least one free storage block from S storage blocks managed by the first leaf node, wherein the use state information of the S storage blocks is recorded in the first leaf node;
storing data of the first target file into the at least one free storage block.
2. The method of claim 1, further comprising:
acquiring a file creating request of the first thread, wherein the file creating request comprises path information of the first target file, the path information of the first target file comprises a first directory entry to be created, and the first directory entry is a directory entry to which the first target file belongs;
acquiring a directory entry of a parent directory where the first target file is located according to the path information;
determining a first node from a first space linked list of a directory entry of the parent directory according to the number of bytes needed by the first directory entry to be created, wherein the space linked list comprises P nodes, each node of the P nodes is used for indicating an idle interval in a first storage block corresponding to the directory entry of the parent directory, and the first node comprises address information of the first interval, wherein P is an integer not less than 1;
storing the first directory entry in the first extent to which the first node points.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
acquiring a second file creating request from the first thread, wherein the second file creating request comprises second path information of a second target file, the second path information comprises a second directory entry to be created, and the second directory entry is a directory entry to which the second target file belongs;
acquiring a directory entry of a parent directory where the second target file is located according to the second path information;
when the plurality of storage blocks corresponding to the directory entry of the parent directory where the second target file is located do not have allocatable free intervals, acquiring the first bitmap segment tree corresponding to the first storage space to which the first thread belongs;
determining a second leaf node according to the first bitmap line segment tree, wherein the second leaf node records the information of allocable free storage blocks, and the second leaf node randomly searches for a Log from the root node of the first bitmap line segment tree to the leaf nodemObtained by searching after (N/S) times;
selecting at least one free storage block from the first storage space according to the bitmap managed by the second leaf node;
storing the second directory entry to be created in the at least one free memory block.
4. The method according to any one of claims 1 to 2, further comprising:
receiving a file deletion request, wherein the file deletion request comprises path information of a second target file to be deleted;
determining a directory entry of a parent directory to which the second target file belongs according to the path information of the second target file;
searching the identifier of a second storage block to which the second target file belongs and the address information of a second interval to which the directory entry of the second target file belongs according to the directory entry of the parent directory to which the second target file belongs;
calculating a hash value of the identity of the second storage block;
searching a second space linked list of the directory entry of the parent directory to which the second target file belongs according to the hash value, wherein the second space linked list is used for recording available idle intervals in a second storage block corresponding to the directory entry of the parent directory to which the second target file belongs;
and inserting a second node into the second space linked list, wherein the second node records the address information of the second interval.
5. The method of claim 1, wherein determining a first leaf node in the first bitmap segment tree comprises:
searching the ith intermediate node from the root node of the first bitmap line segment tree to the leaf node at random, wherein the ith intermediate node is obtained after searching the root node at random for i times, and i is not less than 1 and not more than Logm(N/S),Logm(N/S) is a natural number greater than 1, and the number of available free storage blocks in the subtree of the ith intermediate node is recorded in the ith intermediate node;
when the number of available idle storage blocks in the subtree of the ith intermediate node does not meet the allocation condition, selecting the jth intermediate node from the sibling nodes of the ith intermediate node, wherein the subtree of the jth intermediate node contains the idle storage blocks meeting the allocation condition;
determining the first leaf node in the subtree of the jth intermediate node.
6. The method of claim 1, wherein selecting at least one free memory block from the S memory blocks managed by the first leaf node comprises:
and randomly selecting at least one idle storage block from the S storage blocks managed by the first leaf node.
7. A memory space management device is applied to a storage device with a file system built in a memory, wherein the memory is a nonvolatile memory (NVM), and the memory space management device is characterized by comprising:
the device comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a file access request of a first thread, and the file access request comprises data of a first target file to be stored;
an obtaining module, configured to obtain a first bitmap line segment tree corresponding to a first storage space to which a first thread belongs, where the memory includes multiple storage spaces, each storage space is managed by using a different bitmap line segment tree, and the first bitmap line segment tree includes N/S leaf nodes, where N is used to indicate the number of storage blocks included in the first storage space, S is used to indicate the number of storage blocks in the first storage space managed by each leaf node, N is not less than 2, S is not less than 1, and N is not less than S;
a determining module, configured to determine a first leaf node in the first bitmap line segment tree, where the first leaf node records information of a free storage block, and the first leaf node randomly searches for a Log from a root node of the first bitmap line segment tree to a leaf nodemThe node is obtained by searching after N/S times, wherein m is used for indicating the number of next-level child nodes of any node in the first bitmap line segment tree, and the value of m is not less than 2;
a selecting module, configured to select at least one free storage block from S storage blocks managed by the first leaf node, where the first leaf node records use state information of the S storage blocks;
and the storage module is used for storing the data of the first target file into the at least one free storage block.
8. The apparatus of claim 7, wherein the obtaining module is further configured to:
acquiring a file creating request of the first thread, wherein the file creating request comprises path information of the first target file, the path information of the first target file comprises a first directory entry to be created, and the first directory entry is a directory entry to which the first target file belongs; acquiring a directory entry of a parent directory where the first target file is located according to the path information;
the determining module is further configured to determine a first node from a first space linked list of directory entries of the parent directory according to the number of bytes needed by the first directory entry to be created, where the space linked list includes P nodes, each node of the P nodes is used to indicate a free interval in a first storage block corresponding to a directory entry of the parent directory, and the first node includes address information of the first interval, where P is an integer not less than 1;
the storage module is further configured to store the first directory entry in the first interval pointed to by the first node.
9. The apparatus of claim 7 or 8, wherein the obtaining module is further configured to:
acquiring a second file creating request from the first thread, wherein the second file creating request comprises second path information of a second target file, the second path information comprises a second directory entry to be created, and the second directory entry is a directory entry to which the second target file belongs; acquiring a directory entry of a parent directory where the second target file is located according to the second path information; when the plurality of storage blocks corresponding to the directory entry of the parent directory where the second target file is located do not have allocatable free intervals, acquiring the first bitmap segment tree corresponding to the first storage space to which the first thread belongs;
the determining module is further configured to determine a second leaf node according to the first bitmap line segment tree, where the second leaf node records information of an allocable free storage block, and the second leaf node randomly searches for a Log from a root node of the first bitmap line segment tree to a leaf nodemObtained by searching after (N/S) times;
the selection module is further used for selecting at least one free storage block from the first storage space according to the bitmap managed by the second leaf node;
the storage module is further configured to store the second directory entry to be created in the at least one free storage block.
10. The apparatus of claim 7, wherein the receiving module is further configured to:
receiving a file deletion request, wherein the file deletion request comprises path information of a second target file to be deleted;
the determining module is further configured to determine, according to the path information of the second target file, a directory entry of a parent directory to which the second target file belongs;
the device further comprises:
the searching module is used for searching the identifier of a second storage block to which the second target file belongs and the address information of a second interval to which the directory entry of the second target file belongs according to the directory entry of the parent directory to which the second target file belongs;
a calculation module for calculating a hash value of the identity of the second storage block;
the searching module is further configured to search a second space linked list of directory entries of a parent directory to which the second target file belongs according to the hash value, where the second space linked list is used to record available idle intervals in a second storage block corresponding to the directory entries of the parent directory to which the second target file belongs;
and the inserting module is used for inserting a second node into the second space linked list, wherein the second node records the address information of the second interval.
11. The apparatus according to any one of claims 7 to 8, wherein the determining module is specifically configured to:
randomly searching an ith intermediate node from a root node of the first bitmap line segment tree to a leaf node, wherein the ith intermediate node is obtained by randomly searching the root node for i times, and i is not less than 1 and not more than Logm(N/S),Logm(N/S) is a natural number greater than 1, and the number of available free storage blocks in the subtree of the ith intermediate node is recorded in the ith intermediate node;
when the number of available idle storage blocks in the subtree of the ith intermediate node does not meet the allocation condition, selecting the jth intermediate node from the sibling nodes of the ith intermediate node, wherein the subtree of the jth intermediate node contains the idle storage blocks meeting the allocation condition;
determining the first leaf node in the subtree of the jth intermediate node.
12. The apparatus of claim 7, wherein the selection module is specifically configured to:
and randomly selecting at least one idle storage block from the S storage blocks managed by the first leaf node.
CN201710184680.0A 2017-03-24 2017-03-24 Memory space management method and device Active CN108628753B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710184680.0A CN108628753B (en) 2017-03-24 2017-03-24 Memory space management method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710184680.0A CN108628753B (en) 2017-03-24 2017-03-24 Memory space management method and device

Publications (2)

Publication Number Publication Date
CN108628753A CN108628753A (en) 2018-10-09
CN108628753B true CN108628753B (en) 2021-02-23

Family

ID=63707879

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710184680.0A Active CN108628753B (en) 2017-03-24 2017-03-24 Memory space management method and device

Country Status (1)

Country Link
CN (1) CN108628753B (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109522308B (en) * 2018-11-06 2023-06-20 北京太一星晨信息技术有限公司 Connection handle management method and device
CN109508326B (en) * 2018-11-22 2020-03-17 北京百度网讯科技有限公司 Method, device and system for processing data
CN109710542B (en) * 2018-12-28 2021-03-16 北京像素软件科技股份有限公司 Full N-way tree construction method and device
CN110221782A (en) * 2019-06-06 2019-09-10 重庆紫光华山智安科技有限公司 Video file processing method and processing device
CN111078920B (en) * 2019-11-22 2021-09-03 北京云宽志业网络技术有限公司 Data processing method, device, equipment and computer readable storage medium
US11507431B2 (en) * 2020-04-01 2022-11-22 Microsoft Technology Licensing, Llc Resource allocation for virtual machines
CN111722962A (en) * 2020-06-19 2020-09-29 艾索信息股份有限公司 Storage system and method for CDP data
CN111857575A (en) * 2020-06-24 2020-10-30 国汽(北京)智能网联汽车研究院有限公司 Method, device and equipment for determining memory space of computing platform and storage medium
CN112596908B (en) * 2020-12-28 2022-09-16 中孚安全技术有限公司 Memory management method and system based on complete binary tree
CN112784264B (en) * 2021-01-18 2024-04-02 北京洛塔信息技术有限公司 Jail-break state detection method, device, equipment and storage medium
CN112947860B (en) * 2021-03-03 2022-11-04 成都信息工程大学 Hierarchical storage and scheduling method for distributed data copies
CN115190136B (en) * 2021-04-21 2024-03-01 统信软件技术有限公司 Data storage method, data transmission method and computing equipment
CN114489508B (en) * 2022-01-26 2023-09-01 重庆紫光华山智安科技有限公司 Data management method, device, equipment and medium
CN115168304B (en) * 2022-09-06 2023-01-20 北京奥星贝斯科技有限公司 Data processing method, device, storage medium and equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101329906A (en) * 2008-07-24 2008-12-24 中国科学院上海微系统与信息技术研究所 Non-volatile sequent modular memory, data storage and reading method
CN102713826A (en) * 2009-12-22 2012-10-03 苹果公司 Methods and apparatuses to allocate file storage via tree representations of a bitmap
US8375012B1 (en) * 2011-08-10 2013-02-12 Hewlett-Packard Development Company, L.P. Computer indexes with multiple representations
CN106354890A (en) * 2016-11-22 2017-01-25 中国科学院上海微系统与信息技术研究所 Implementation method of random access file system based on N-ary tree structure
CN106469123A (en) * 2015-08-10 2017-03-01 北京忆恒创源科技有限公司 A kind of write buffer distribution based on NVDIMM, method for releasing and its device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7272698B2 (en) * 2003-03-19 2007-09-18 Autodesk, Inc. Heap memory management
US7356655B2 (en) * 2003-05-15 2008-04-08 International Business Machines Corporation Methods, systems, and media for managing dynamic storage
CN100541453C (en) * 2007-12-03 2009-09-16 腾讯数码(天津)有限公司 Large capacity cache implementation method and storage system
CN101330529B (en) * 2008-07-29 2010-08-25 杭州电子科技大学 Method for distributing address of self-pruning type tree-shaped structure network
CN101488153A (en) * 2009-02-12 2009-07-22 浙江大学 Method for implementing high-capacity flash memory file system in embedded type Linux
US9658999B2 (en) * 2013-03-01 2017-05-23 Sony Corporation Language processing method and electronic device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101329906A (en) * 2008-07-24 2008-12-24 中国科学院上海微系统与信息技术研究所 Non-volatile sequent modular memory, data storage and reading method
CN102713826A (en) * 2009-12-22 2012-10-03 苹果公司 Methods and apparatuses to allocate file storage via tree representations of a bitmap
US8375012B1 (en) * 2011-08-10 2013-02-12 Hewlett-Packard Development Company, L.P. Computer indexes with multiple representations
CN106469123A (en) * 2015-08-10 2017-03-01 北京忆恒创源科技有限公司 A kind of write buffer distribution based on NVDIMM, method for releasing and its device
CN106354890A (en) * 2016-11-22 2017-01-25 中国科学院上海微系统与信息技术研究所 Implementation method of random access file system based on N-ary tree structure

Also Published As

Publication number Publication date
CN108628753A (en) 2018-10-09

Similar Documents

Publication Publication Date Title
CN108628753B (en) Memory space management method and device
CN108804510B (en) Key value file system
JP6205650B2 (en) Method and apparatus utilizing non-uniform hash function to place records in non-uniform access memory
US9047301B2 (en) Method for optimizing the memory usage and performance of data deduplication storage systems
CN106294190B (en) Storage space management method and device
KR101767710B1 (en) Card-based management of discardable files
CN107180092B (en) File system control method and device and terminal
KR100856245B1 (en) File system device and method for saving and seeking file thereof
CN101488153A (en) Method for implementing high-capacity flash memory file system in embedded type Linux
CN103064639A (en) Method and device for storing data
WO2019001020A1 (en) Storage space arrangement method, apparatus, and system based on distributed system
US20150324281A1 (en) System and method of implementing an object storage device on a computer main memory system
US9430492B1 (en) Efficient scavenging of data and metadata file system blocks
KR20220139784A (en) Object storage method and distributed object storage and system using the same
JP2012512460A (en) Discardable file
KR20150071500A (en) Method and Apparatus for Managing Data
JP5715964B2 (en) Managing downloadable files
US7752206B2 (en) Method and data processing system for managing a mass storage system
CN109407985B (en) Data management method and related device
KR100907477B1 (en) Apparatus and method for managing index of data stored in flash memory
CN107203479B (en) Hierarchical storage system, storage controller and hierarchical control method
CN106294189B (en) Memory defragmentation method and device
CN104537023B (en) A kind of storage method and device of reverse indexing record
US10430106B2 (en) Policy based tiered allocation for hybrid storage devices
US10394717B1 (en) Central processing unit cache friendly multithreaded allocation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant