WO2021129354A1 - 存储系统中管理数据索引的方法和装置 - Google Patents

存储系统中管理数据索引的方法和装置 Download PDF

Info

Publication number
WO2021129354A1
WO2021129354A1 PCT/CN2020/134025 CN2020134025W WO2021129354A1 WO 2021129354 A1 WO2021129354 A1 WO 2021129354A1 CN 2020134025 W CN2020134025 W CN 2020134025W WO 2021129354 A1 WO2021129354 A1 WO 2021129354A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
data index
logical address
address range
length
Prior art date
Application number
PCT/CN2020/134025
Other languages
English (en)
French (fr)
Inventor
德语心
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20907053.1A priority Critical patent/EP4068111A4/en
Publication of WO2021129354A1 publication Critical patent/WO2021129354A1/zh
Priority to US17/846,208 priority patent/US20220327102A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2246—Trees, e.g. B+trees
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/0292—User address space allocation, e.g. contiguous or non contiguous base addressing using tables or multilevel address translation means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/04—Addressing variable-length words or parts of words
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2272—Management thereof

Definitions

  • This application relates to the field of computer technology, and in particular to a method and device for managing data indexes in a storage system.
  • Data index is used to quickly query data. Using data index can shield data semantics and provide consistent and efficient storage services.
  • the storage address of the data on the storage system is generated according to a specific data index operation unit as a unit to generate a data index.
  • the storage system uses a solid-granularity data index operation unit to create a data index. According to the diversity of data access in the storage system, the data access requirement cannot be met. Therefore, how to use a data index operation unit with a suitable granularity to create a data index becomes a problem to be solved by the storage system.
  • the embodiments of the present application provide methods and devices for managing data indexes in a storage system, which are helpful for realizing the creation of data indexes using data index operation units with appropriate granularity, thereby saving storage space occupied by data indexes.
  • a method for managing data indexes in a storage system including: acquiring first data to be written, and a logical address range of the first data to be written is a first logical address range. Then, the data index is generated based on the alignment between the logical address range of the data index to be generated in the first logical address range and the data index operation unit of different length in the storage system.
  • the storage system includes a data index operation unit with a first length and a data index operation unit with a second length; the first length is greater than the second length.
  • the data index is generated based on the larger the length of the data index operation unit.
  • the smaller the amount of data index the more storage space occupied by the data index is saved. . Therefore, the use of this technical solution helps to save the storage space occupied by the data index.
  • the logical address range of the data index to be generated can be understood as the logical address range in the first logical address range that does not participate in the operation of generating the data index.
  • the data index is generated based on the alignment between the logical address range of the data index to be generated in the first logical address range and the data index operation units of different lengths in the storage system, which specifically includes: when the data index is to be generated When the logical address range is aligned with the data index operation unit of the first length, the first data index is generated for the first data based on the data index operation unit of the first length; the first data is the data index to be generated from the first data to be written
  • the logical address range starts with the first logical address and the length is the data of the logical address range of the first length.
  • the first data index includes the location of the first data in the storage system.
  • the data index is generated based on the alignment between the logical address range of the data index to be generated in the first logical address range and the data index operation units of different lengths in the storage system, which specifically includes: when the data index is to be generated When the logical address range is not aligned with the data index operation unit of the first length, a second data index is generated for the second data based on the data index operation unit of the second length.
  • the second data is data in the first data to be written starting from the first logical address of the logical address range of the data index to be generated and the length is the second length of the logical address range; the second data index includes the second data in the storage The location in the system.
  • the data index is preferably generated based on the larger-length data index operation unit, and the smaller the number of data indexes, it helps to save the storage space occupied by the data index.
  • the larger the length of the data index operation unit the higher the priority.
  • the data to be generated when the first logical address of the logical address range of the data index to be generated is divisible by the first length, and the logical address range of the data index to be generated is greater than or equal to the first length, the data to be generated The logical address range of the index is aligned with the data index operation unit of the first length.
  • the The logical address range for generating the data index is not aligned with the first-length data index operation unit.
  • the method further includes: deleting the first target data index when the first target data index already exists in the storage system.
  • a target data index is a data index generated for the third data based on the data index operation unit of the second length.
  • the logical address range of the first data includes the logical address range of the third data.
  • the first target data index contains the position of the third data in the storage system. In this way, it helps to save the storage space occupied by the data index, and helps to improve the efficiency of garbage collection during garbage collection.
  • the method further includes: when the first target data index already exists in the storage system, the storage system Mark the first target data index as invalid.
  • the method further includes: when the second target data index already exists in the storage system, adding the second target data index 2.
  • the status of the target part in the target data index is marked as invalid.
  • the second target data index is a data index generated for the fourth data based on the data index operation unit of the first length.
  • the target part is a data index generated for the data of the second logical address range included in the logical address range of the fourth data.
  • the second logical address range is the same as the logical address range of the second data. In this way, it is helpful to distinguish between new data and old data in the data query process and garbage collection process, thereby improving the efficiency of data query and garbage collection.
  • the storage system may use status bits to mark each section in the second target data index.
  • the method further includes: deleting the second target data index when the states of all parts of the second target data index are marked as invalid. In this way, it helps to distinguish between new data and old data in the garbage collection process, thereby improving the efficiency of garbage collection.
  • the method further includes: when the number of invalid parts in all parts of the second target data index is greater than or equal to the threshold, generating new parts in all the parts whose status is valid Data index, and delete the second target data index. In this way, it helps to further save the storage space occupied by the data index; and, in the garbage collection process, distinguish between new data and old data, thereby improving the efficiency of garbage collection.
  • the method further includes: acquiring the logical address range to be queried; then, encoding the logical address range to be queried based on the second length to obtain the first encoding result; then, querying the first encoding result based on the first encoding result 2.
  • Data index includes the first encoding result.
  • the method further includes: when the first encoding result does not exist in the storage system, encoding the logical address range to be queried based on the first length to obtain the second encoding result; then, based on the second encoding result The first data index is queried; where the first data index includes the second encoding result; then, based on the data indicated by the first data index, the data of the logical address range to be queried is determined.
  • the first logical address in the first logical address range can be divisible by the second length, and the length of the first logical address range is an integer multiple of the second length.
  • acquiring the first data to be written specifically includes: receiving a data write request; the data write request includes the first data to be written.
  • a data write request is received, where the data write request includes the initial data to be written and the initial logical address range; the initial data to be written is supplemented based on the second length to obtain the first data to be written.
  • the first logical address range may be the logical address range carried in the write request, or the logical address range supplemented based on the logical address range.
  • data is complemented based on a data index operation unit with a smaller length, which helps to reduce read amplification.
  • generating the first data index for the first data based on the data index operation unit of the first length includes: encoding the logical address range of the first data based on the data index operation unit of the first length to obtain The first encoding result; the first data index (such as the key of the first data index) includes the first encoding result.
  • generating the second data index for the second data based on the data index operation unit of the second length includes: encoding the logical address range of the second data based on the data index operation unit of the second length to obtain The first encoding result; the second data index (such as the key of the second data index) includes the second encoding result.
  • the embodiment of the present application does not limit the specific encoding method, for example, the prior art can be referred to.
  • the storage system includes a first prefix tree and a second prefix tree; the generation time of the first prefix tree is before the second prefix tree; the first prefix tree includes the first data index; the second prefix tree Including the second data index.
  • the logical address range of the data indicated by the first data index includes the logical address range of the data indicated by the second data index.
  • the method further includes: combining the first prefix tree and the second prefix tree to obtain a combined prefix tree.
  • the combined prefix tree includes a first data index and a second data index.
  • the first data index includes a plurality of parts, the target part of the plurality of parts is marked as invalid, and the target part is the part corresponding to the logical address range of the data indicated by the second data index.
  • the storage system includes a first prefix tree and a second prefix tree; the generation time of the first prefix tree is before the second prefix tree; the first prefix tree includes the second data index; the second prefix tree Including the first data index.
  • the logical address range of the data indicated by the first data index includes the logical address range of the data indicated by the second data index.
  • the method further includes: combining the first prefix tree and the second prefix tree to obtain a combined prefix tree.
  • the combined prefix tree includes the first data index, but does not include the second data index.
  • the method may further include: if the second encoding result does not exist in the cache of the storage system, reading the first data index from the hard disk of the storage system.
  • the logical address range corresponding to the first data index includes the logical address range to be queried; if the status of the part corresponding to the logical address range to be queried in the first data index is marked as valid, then the data is based on the data indicated by the first data index Determine the data of the logical address range to be queried. If the status of the part of the first data index corresponding to the logical address range to be queried is marked as invalid, the second data index of the logical address range corresponding to the part is obtained from the hard disk.
  • the data index operation unit based on a larger length is preferred to read data from the cache. In this way, it helps to reduce the number of times to read data from the hard disk, thereby speeding up the data reading rate.
  • a device for managing data indexes in a storage system which is used to implement the above-mentioned first aspect or any one of the methods provided in the first aspect.
  • the device can be a storage controller or a server or a chip or the like.
  • the device includes various modules for executing the method provided in the first aspect or any one of the possible designs.
  • the device includes a memory and a processor, the memory is used to store computer program instructions, and the processor is used to call the computer program instructions to execute the method provided in the first aspect or any of its possible designs.
  • a computer-readable storage medium on which computer program instructions are stored.
  • the computer program instructions When the computer program instructions are called by a computer, the computer executes the method provided in the first aspect or any one of its possible designs.
  • a computer program product is provided.
  • the computer program instructions contained in the computer program product are executed on a computer, the method provided in the first aspect or any one of its possible designs is executed.
  • any of the above-provided devices for managing data indexes in storage systems, computer-readable storage media, or computer program products are all used to execute the above-provided methods for managing data indexes in the corresponding storage systems. Therefore, for the beneficial effects that can be achieved, please refer to the beneficial effects in the corresponding method, which will not be repeated here.
  • FIG. 1 is a schematic diagram of a prefix tree applicable to an embodiment of the present application
  • FIG. 2 is a schematic diagram of a storage system applicable to an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a storage controller applicable to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of another storage system applicable to the embodiments of the present application.
  • FIG. 5 is a schematic structural diagram of a server applicable to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of a method for managing data indexes in a storage system according to an embodiment of the application
  • FIG. 7 is a schematic diagram of a prefix tree insertion process provided by an embodiment of the application.
  • FIG. 8 is a schematic diagram of a modification process of a prefix tree provided by an embodiment of this application.
  • FIG. 9 is a schematic diagram of another modification process of a prefix tree provided by an embodiment of the application.
  • FIG. 10 is a schematic flowchart of another method for managing data indexes in a storage system according to an embodiment of the application.
  • FIG. 11 is a schematic diagram of a prefix tree merging process provided by an embodiment of this application.
  • FIG. 12 is a schematic diagram of another prefix tree merging process provided by an embodiment of this application.
  • FIG. 13 is a schematic diagram of a process of decomposing a first data index according to an embodiment of this application.
  • FIG. 14 is a schematic structural diagram of an apparatus for managing data indexes in a storage system provided by an embodiment of the application.
  • the data index is used to indicate the location of the data in the storage system.
  • the storage system receives a data write request, and the data write request includes a logical address range of the data.
  • a data index is established for the data.
  • the data index usually contains the location of the data in the storage system.
  • the location of the data in the storage system may specifically be the location of the data on the hard disk in the storage system or the location of the data on the storage system volume.
  • the storage system can generally determine the key of the data based on the encoding result of the logical address range of the data, and is used to construct the data index of the data.
  • the key may include the identifier of the data, the identifier of the volume where the data is located, or the identifier of the file to which the data belongs. In another implementation manner, the key may include the identification of the data, the identification of the volume where the data is located, or the identification encoding result of the file to which the data belongs.
  • Storage systems usually manage data indexes in the form of a prefix tree.
  • the logical address is an access address provided by the storage system to the client.
  • the location of the data in the storage system is the storage address corresponding to the logical address in the storage system, which is used to store the data.
  • the client can be a host, a server, a virtual machine, or an application.
  • the storage system receives a data query or read request, generates a corresponding key according to the data identifier and the logical address range of the data carried in the data query or read request, and queries whether the key is included in the data index to determine whether the data is in the storage system s position.
  • the key may include the identification information of the file where the data is located (or the encoding result of the identification information of the file where the data is located) and the encoding result of the logical address range of the data.
  • the identification information of the file may include the file name and/or the file path.
  • the embodiment of the present application does not limit the encoding method of the identification information of the file. For brevity of description, the following descriptions are given by taking "the key including the file name of the file where the data is located, and the result of encoding the logical address range of the data" as an example.
  • the embodiment of the present application does not limit the encoding method of the logical address range of the data, and the specific example can be referred to the following.
  • the length of the data can be characterized by the range of the logical address of the data.
  • the data length of the logical address range [0 bit, 7 bit] is 8 bits.
  • the length identifier can be any kind of information that uniquely represents the length.
  • the length identifier can be the length itself, or an identifier.
  • the binary number "1" is used to represent the first length
  • the binary number "0" is used to represent the second length.
  • the data index operation unit is also called the data index granularity, that is, the length of the data indicated by the data index.
  • the data index operation unit corresponds to the minimum storage granularity and minimum operation granularity of the data in the storage system, that is, the minimum logical address length.
  • the logical address range of the data needs to be aligned with the data index operation unit when reading and writing data once. In this way, it helps to reduce the storage space occupied by the data index and speed up the read and write rate.
  • the logical address range of the data is aligned with the data index operation unit, which can be the offset of the first logical address of the data when the data is read and written once (for example, the first logical address of the data is relative to its location).
  • the offset of the first logical address of the file can be divisible by the length of the data index operation unit, and the length of the data is equal to the length of the data index operation unit.
  • the offset of the first logical address of the data when the storage system reads and writes data is an integer multiple of 8 bits, such as 0 bits, 8 bits, 16 bits, 24 bits... .
  • the logical address range of the data can include: [0 bit, 7 bit], [8 bit, 15 bit], [16 bit, 23 bit]...
  • the offset of the first logical address of the data when the storage system reads and writes data once is an integer multiple of 32 bits, such as 0 bits, 32 bits, 64 bits.
  • the logical address range of data when the storage system reads and writes data at one time can include: [0 bit, 31 bit], [32 bit, 63 bit], [64 bit, 95 bit]...
  • Prefix tree also known as dictionary tree, is an ordered tree used to store associative arrays, the keys of which are usually strings. Unlike the binary search tree, the key of the prefix tree is not directly stored in the node, but is determined by the position of the node in the tree. All descendants of a node have the same prefix, which is the string corresponding to this node, and the root node corresponds to an empty string. In general, not all nodes have corresponding values, only the keys corresponding to leaf nodes and some internal nodes have relevant values. In some embodiments of the present application, a prefix tree may be used to manage data indexes.
  • FIG. 1 it is a schematic diagram of a prefix tree applicable to the embodiments of the present application.
  • the prefix tree includes a root node, several intermediate nodes connected to the root node, and several leaf nodes connected to each intermediate node.
  • a data index may be composed of a root node and a leaf node in the prefix tree, and an intermediate node connecting the two.
  • Figure 1 shows three leaf nodes, which correspond to data indexes 1-3, respectively.
  • the key in the data index 1 is abc
  • the value corresponding to the key is the address 1 stored in the leaf node c
  • the address is the location of the data indicated by the data index 1 in the storage system.
  • the key in data index 2 is abd
  • the value corresponding to this key is address 2 stored in leaf node d
  • address 2 is the location of the data indicated by data index 2 in the storage system.
  • the key in the data index 3 is efg
  • the corresponding value of this key is the address 3 stored in the leaf node g, which is the location of the data indicated by the data index 3 in the storage system.
  • the logical address range corresponding to the data index refers to the logical address range of the data indicated by the data index.
  • garbage data When data is written in the way of additional writing, when the data is modified, the data before modification will become invalid data in the storage system, that is, garbage data.
  • the data before modification and the data after modification have the same logical address range, but have different locations in the storage system. Therefore, the status of the data index for indicating the data before modification becomes invalid. Therefore, garbage collection is required.
  • Garbage collection can be understood as: sorting out the data in the storage system to delete invalid data, and relocating valid data to shrink the data to save storage space.
  • the storage system can perform garbage collection based on the data index operation unit. If the data index contains multiple sections, the data index is divided into multiple sub-granularity data indexes. For example, the first part of the data index contains the location of the data in the storage system as address A. Because the data in address A is modified, that is, the logical address range of the data in address A is written to the data again. Therefore, the first part of the data index The status becomes invalid.
  • the storage system can allocate a status bit to each part of the data index, which is used to characterize the status of the part. When the number of invalid parts in the data index reaches (that is, greater than or equal to) the threshold, the data represented by the data index is garbage collected. When the number of invalid parts in the data index reaches (that is, greater than or equal to) the threshold, the valid parts in the data index can be formed into a new data index. When all parts of the data index become invalid, the data index can be deleted directly.
  • words such as “exemplary” or “for example” are used as examples, illustrations, or illustrations. Any embodiment or design solution described as “exemplary” or “for example” in the embodiments of the present application should not be construed as being more preferable or advantageous than other embodiments or design solutions. To be precise, words such as “exemplary” or “for example” are used to present related concepts in a specific manner.
  • At least one refers to one or more.
  • Multiple means two or more.
  • "and/or” is merely an association relationship describing associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean that A exists alone, and A and A exist at the same time. B, there are three cases of B alone.
  • the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.
  • the storage system in the embodiment of this application may be a storage array (e.g. of series, series). As shown in FIG. 2, the storage array includes a storage controller 201 and multiple hard disks. Hard disks include solid state disks (SSD) or magnetic disks.
  • SSD solid state disks
  • the storage controller 201 includes a processor 301, a memory 302, and an interface 303.
  • the memory 302 stores computer program instructions.
  • the processor 301 executes the computer program instructions in the memory 302, and manages and performs data on the storage system. Access operation.
  • Central processing unit (CPU), field programmable gate array (FPGA) or other hardware can be used as processor 301, or FPGA or other hardware and CPU can be used as processor 301, processor 301 Communicate with interface 303.
  • the memory 302 in the embodiment of the present application may provide memory for the processor 301.
  • the interface 303 may be a network interface card (NIC) or a host bus adapter (HBA) or the like.
  • the storage controller 201 may be used to execute the method for managing data indexes in the embodiment of the present application.
  • the storage system of the embodiment of the present application may also be a distributed storage system (such as of Series) and so on. To of series.
  • the distributed storage system includes multiple servers, such as server 1, server 2, server 3,..., server 6, and the servers communicate with each other through an InfiniBand or Ethernet network.
  • the number of servers in the distributed storage system can be increased or decreased according to actual needs, which is not limited in the embodiment of the present application.
  • the server in a distributed storage system is also called a storage node.
  • the server of the distributed storage system may include the structure shown in FIG. 5.
  • each server in the distributed storage system may include a processor 501, a memory 502, an interface 503, a hard disk 1, a hard disk 2, and a hard disk 3, and so on.
  • Computer program instructions are stored in the memory 502.
  • the processor 501 executes computer program instructions in the memory 502 to perform corresponding operations.
  • a CPU, FPGA or other hardware can serve as the processor 501, or a combination of FPGA (or other hardware) and a CPU can serve as the processor 501 together.
  • the memory 502 in the embodiment of the present application may provide memory for the processor 501.
  • the interface 503 can be a hardware interface, such as NIC or HBA, or a program interface module.
  • Hard disks include SSDs or disks, etc.
  • the server can be used to execute the method for managing data indexes in the embodiments of the present application.
  • FIG. 6 a schematic flowchart of a method for managing data indexes in a storage system provided by an embodiment of this application.
  • This embodiment specifically describes the method for managing data indexes executed by the storage system "under the triggering condition of the data write request".
  • the method may include the following steps:
  • the storage system receives a data write request.
  • the data write request may be a data write request from the client.
  • the data write request includes the initial data to be written and the logical address range of the initial data to be written (that is, the initial logical address range).
  • the initial logical address range can be of any length, and can be specifically determined based on the needs of the client.
  • the data write request may also include: the file name of the file where the initial data to be written is located, and the path of the initial data to be written in the file where it is located.
  • S102 The storage system judges whether the initial data to be written needs to be filled. If yes, execute S103; if not, execute S104.
  • the length of the initial logical address range is an integer multiple of the second length
  • the second length is 8 bits and the initial logical address range is [0 bits, 7 bits] or [8 bits, 23 bits], etc., there is no need to fill in the initial data to be written.
  • the initial data to be written needs to be filled. For example, if the second length is 8 bits, then:
  • Example 1 When the initial logical address range is [1 bit, 7 bits], the first logical address (ie 1 bit) of the initial logical address range cannot be divisible by the second length (ie 8 bits), so it is necessary to fill in the initial waiting Write data.
  • Example 2 When the initial logical address range is [0 bits, 6 bits], the length of the initial logical address range (ie 7 bits) is not an integer multiple of the second length (ie 8 bits), so it is necessary to fill in the initial data to be written .
  • Example 3 When the initial logical address range is [1 bit, 9 bits], the first logical address (that is, 1 bit) of the initial logical address range cannot be divisible by the second length (that is, 8 bits), and the initial logical address range is The length (ie, 9 bits) is not an integer multiple of the second length (ie, 8 bits), so the initial data to be written needs to be filled.
  • S103 The storage system fills in the initial data to be written based on the data index operation unit of the second length to obtain the first data to be written, and uses the logical address range of the first data to be written as the first logical address range. After S103 is executed, S105 is executed.
  • complementing the initial data to be written based on the data index operation unit of the second length may include:
  • the storage system When the first logical address in the initial logical address range is not divisible by the second length, the storage system reads data from the hard disk in the logical address range of the second length where the first logical address is located; then, based on the initial data to be written And the read data to obtain the first data to be written.
  • the initial logical address range is [1 bit, 7 bits].
  • the storage system can read the data in the logical address range [0 bit, 7 bit] from the hard disk and read The data whose logical address range is [1 bit, 7 bits] in the data is replaced with the initial data to be written to obtain the first data to be written.
  • the first data to be written includes data whose logical address is [0 bit, 1 bit] read from the hard disk, and the initial data to be written whose logical address range in the data write request is [1 bit, 7 bits] ,
  • the first logical address range is [0 bit, 7 bit].
  • the storage system reads the last logical address in the initial logical address range from the hard disk The data of the logical address range of the second length where the address is located; then, based on the initial data to be written and the read data, the first data to be written is obtained.
  • the initial logical address range is [0 bit, 6 bit].
  • the storage system can read data in the logical address range [0 bit, 7 bit] from the hard disk and read The data whose logical address range is [0 bit, 6 bit] in the data is replaced with the initial data to be written to obtain the first data to be written.
  • the first data to be written includes the initial data to be written whose logical address range is [0 bit, 6 bit] in the data write request, and the data whose logical address is [6 bit, 7 bit] read from the hard disk, The first logical address range is [0 bit, 7 bit].
  • the first logical address in the initial logical address range is not divisible by the second length, and the length of the initial logical address range is not an integral multiple of the second length, read the first logical address in the initial logical address range from the hard disk Then, based on the initial data to be written and the read data, the first data to be written is obtained based on the data of the second length of the logical address range where the last logical address is located and the data of the second length of the logical address range where the last logical address is located.
  • the embodiment of the application supports the use of a smaller-granularity data index operation unit to complete the initial data to be written instead of using a larger-granularity data index operation unit to complete the initial data to be written, which helps to reduce the generation of data in the process of writing data.
  • the read amplification can increase the speed of reading data from the hard disk in the process of writing data, thereby improving the efficiency of writing data.
  • read amplification means that the amount of data read from the hard disk is greater than the amount of data to be written.
  • S104 The storage system uses the initial data to be written as the first data to be written, and the initial logical address range as the first logical address range. After S104 is executed, S105 is executed.
  • the storage system stores the first data to be written, and records the position of the first data to be written in the storage system. Among them, when the storage system writes data to the hard disk, an additional writing method can be adopted, but of course it is not limited to this.
  • the storage system storing the first data to be written specifically includes allocating a storage location in the storage system for the first data to be written, writing the first data to be written to the location, and recording the location of the first data to be written in the storage system .
  • the location can be the location of the data on the hard disk in the storage system or the location of the data on the storage system volume.
  • the storage system may execute S106-S113 one or more times, until the operation of generating a data index is executed based on each sub-logical address range in the first logical address range.
  • the sub-logical address range is a logical address range aligned with a data index operation unit of the first length, or a logical address range aligned with a data index operation unit of the second length.
  • S106 The storage system judges whether the logical address range of the data index to be generated is aligned with the data index operation unit of the first length. When S106-S113 is executed for the first time, the logical address range of the data index to be generated is the first logical address range.
  • the logical address range of the data index to be generated is the same as the first length.
  • One-length data index operation unit is aligned.
  • the logical address range of the data index to be generated It is not aligned with the data index operation unit of the first length.
  • the first logical address range of the data index to be generated is 0 bit
  • the length of the logical address range of the data index to be generated is 32 bits
  • the logical address range of the generated data index is aligned with the data index operation unit of the first length. If the first logical address range is [0 bits, 12 bits], when S106 is executed for the first time, the first logical address in the logical address range of the data index to be generated is 0 bits, but the logical address range of the data index to be generated The length of is 13 bits. If the above alignment condition is not met, the logical address range of the data index to be generated is not aligned with the data index operation unit of the first length.
  • S107 The storage system generates a first data index for the first data based on the data index operation unit of the first length.
  • the first data index includes the first key and the location of the first data in the storage system. For example, identification information of the offset of the first data in the hard disk and the length of the first data (ie, the first length).
  • the first key may include an encoding result of the identification information of the file carried in the data write request (hereinafter marked as file A) and an encoding result of the logical address range of the first data.
  • the location of the first data in the storage system may include that the first data is data starting from the first logical address of the logical address range of the to-be-generated data index and having a length of the logical address range of the first length.
  • the prefix tree generated by the storage system is first stored in the cache. As more and more data indexes are generated, more and more nodes will be inserted into the prefix tree; when the nodes in the prefix tree in the cache are When the number is greater than or equal to a certain threshold, the storage system will write the prefix tree to the hard disk. Later, when data is written, the storage system will generate a new prefix tree and continue to insert data indexes in the prefix tree.
  • An implementation, in S107 further includes inserting the first data index into the prefix tree.
  • the prefix tree described in S107 may be a prefix tree stored in a cache in a storage system. For a specific example of the storage system inserting a data index into the prefix tree, refer to Figure 7 below.
  • S107 may include: the storage system encodes the logical address range of the first data based on the data index operation unit of the first length to obtain an encoding result of the logical address range of the first data. For example, use “a” as the encoding result of the logical address range of the first data, where “a” indicates that the logical address range of the first data is the a-th logical address range of the first length in file A, and a is an integer . Alternatively, use “ax" as the encoding result of the logical address range of the first data, where the meaning of "a” is referred to above, and x may be a preset character string.
  • the encoding result of [32 bits, 63 bits] can be "1ff"
  • the storage system queries whether the first target data index is included.
  • the first target data index is a data index generated for the third data based on the data index operation unit of the second length.
  • the logical address range of the first data includes the logical address range of the third data; the first target data index includes the third data in the The location in the storage system.
  • the first target data index may be that the storage system is based on the logical address range [ Data index generated by 0 bit, 7 bit], [8 bit, 15 bit], [16 bit, 23 bit] or [24 bit, 31 bit].
  • step S108 Based on the query in step S108, if not, it means that the storage system writes the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time, and then executes S113.
  • the storage system writes the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time, and then executes S113.
  • the storage system writes the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time, and then executes S113.
  • the storage system writes the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time, and then executes S113.
  • the storage system writes the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time
  • step S108 Based on the query in step S108, if yes, it means that the storage system is not writing the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time, then execute S109.
  • the storage system is not writing the data of the "logical address range corresponding to the first target data index (that is, the logical address range of the third data)" for the first time.
  • S108 specifically is that the storage system queries whether the prefix tree contains the first target data index.
  • S109 The storage system deletes the first target data index, and executes S113.
  • the third data is new data
  • the data indicated by the first target data index is old data. Deleting the data index of the old data helps to save the storage space occupied by the data index and helps to improve the efficiency of garbage collection during garbage collection.
  • S109 specifically is that the storage system deletes the first target data index in the prefix tree.
  • the storage system marks the first target data index as invalid.
  • the storage system generates a second data index for the second data based on the data index operation unit of the second length.
  • the second data index includes the second key and the location of the second data in the storage system.
  • the second key may include the encoding result of the identification information of the file (hereinafter referred to as file A) carried in the data write request and the encoding result of the logical address range of the second data.
  • the second data is the data of the logical address range starting from the first logical address of the logical address range of the data index to be generated and the length is the second length in the first data to be written.
  • S110 further includes inserting the second data index into the prefix tree.
  • the location of the second data in the storage system may include identification information of the offset of the second data in the hard disk and the length of the second data (ie, the second length).
  • the data index operation unit of the first length is preferentially used to generate the data index.
  • the data index of the second length is combined together
  • the operating unit creates a data index.
  • S110 may include: the storage system encodes the logical address range of the second data based on the data index operation unit of the second length to obtain an encoding result of the logical address range of the second data. For example, using “ab” as the encoding result of the logical address range of the second data, "ab” indicates that the logical address range of the second data is the b-th in the a-th first-length logical address range in file A. Two-length logical address range. Both a and b are integers.
  • the encoding result of [8 bits, 15 bits] can be "01" , Used to indicate that [8 bits, 15 bits] is the first logical address range of the second length in the 0th logical address range of the first length in file A.
  • the storage system queries whether the second target data index is included.
  • the second target data index is a data index generated for the fourth data based on the data index operation unit of the first length.
  • the logical address range of the fourth data includes the logical address range of the second data.
  • the second target data index contains the location of the fourth data in the storage system.
  • S111 specifically is that the storage system queries whether the second target data index is included in the prefix tree.
  • the second target data index may be that the storage system is based on "[8 bits, 15 bit] data is located in the first length of the logical address range (that is, the logical address range [0 bit, 31 bit])" generated data index.
  • the storage system queries whether the prefix tree contains the second target data index. If so, it means that the storage system writes data of the "logical address range corresponding to the second data index" for the first time, and then executes S113. For an example in this case, refer to FIG. 7.
  • the storage system queries whether the prefix tree contains the second target data index. If not, it means that the storage system is not writing the data of the "logical address range corresponding to the second data index" for the first time, and S112 is executed. For an example in this case, refer to FIG. 9.
  • the second target data index may include multiple sections, and the status of each section may be marked as valid or invalid.
  • One section corresponds to one of the "logical address range corresponding to the second target data index" and The second-length data index operation unit is aligned to the logical address range.
  • the embodiment of the present application does not limit how to mark the status of each part as invalid (or valid).
  • the status bits included in the second target data index may adopt a bitmap mode. As an example, one status bit can be one bit.
  • the second target data index may include 4 status bits, These 4 status bits correspond to the logical address range [0 bit, 7 bit], [8 bit, 15 bit], [16 bit, 23 bit] and [24 bit, 31 bit] respectively, and are used to indicate the second target. Whether the data in the logical address range [0 bit, 7 bit], [8 bit, 15 bit], [16 bit, 23 bit] and [24 bit, 31 bit] in the data indicated by the data index is valid.
  • the storage system may generate the status of multiple parts after the second target data index is generated, and mark the status of the multiple parts as valid; When the status of one or some parts is marked as invalid, the status of the one or more parts is marked as invalid.
  • the storage system may generate the status of a certain part or parts of the second target data index as invalid, generate the status of the multiple parts, and change the status of the part or parts of the second target data index.
  • the status is marked as invalid. In this way, it helps to save storage resources.
  • the storage system marks the status of the target part of the second target data index as invalid.
  • the target part is a data index generated for the data of the second logical address range included in the logical address range of the fourth data; the second logical address range is the same as the logical address range of the second data.
  • the second data is new data
  • the data in the logical address range B in the logical address range corresponding to the second target data index is the old data.
  • Executing S111 and S112 helps to distinguish new data from old data in the data query process and garbage collection process, thereby improving the efficiency of data query and garbage collection.
  • the second target data index can be deleted to save data The storage space occupied by the index.
  • S113 The storage system judges whether each sub-logical address range in the first logical address range participates in the generation operation.
  • the method for managing data indexes in the storage system supports the management of data indexes based on data index operation units of different lengths, and can select operation units of different lengths to generate data based on the logical address range carried by the data write operation index.
  • the data index is generated based on the operation unit with a larger length, and the smaller the number of data indexes, the more storage space occupied by the data index is saved. Therefore, using the technical solution provided by the embodiment of the present application helps to save the storage space occupied by the data index.
  • the data write request includes the file name "A" of file A, the first data to be written, and the first logical address range [8 bits, 71 bits], and the storage system writes the first logical address range to the first logical address range.
  • the first length is 32 bits
  • the second length is 8 bits.
  • Figure a in FIG. 7 illustrates the multiple sub-logical address ranges divided into the first logical address range.
  • One sub-logical address range represents the logical address range that participates in the generation operation when S106-S113 is executed once.
  • Figure b in Figure 7 illustrates the location of the data in each sub-logical address range in the storage system, where the location includes the offset and the data length. Among them, the offset corresponding to each sub-logical address range is marked as offset 1-5.
  • Figure c in Figure 7 illustrates the prefix tree before and after the execution of the data write request.
  • there is no data index in the prefix tree only one root node exists.
  • the logical address range of the data index to be generated is [8 bits, 71 bits], which cannot be aligned with the data index operation unit of the first length. Therefore, the storage system is based on the data index of the second length
  • the operation unit generates data index 1.
  • the logical address range involved in the generation operation is [8 bits, 15 bits].
  • the key in data index 1 may be "A01", which means that [8 bits, 15 bits] is the first logical address range of the second length in the 0th logical address range of the first length in the file A.
  • the position in data index 1 can be "offset 1, 8 bits".
  • the logical address range of the data index to be generated is [16 bits, 71 bits], which cannot be aligned with the data index operation unit of the first length. Therefore, the storage system is based on the data index of the second length
  • the operation unit generates data index 2.
  • the logical address range involved in the generation operation is [16 bits, 23 bits].
  • the key in data index 2 can be "A02”.
  • the position in data index 2 may be "offset 2, 8 bits”.
  • the logical address range of the data index to be generated is [24 bits, 71 bits], which cannot be aligned with the data index operation unit of the first length. Therefore, the storage system is based on the data index of the second length
  • the operation unit generates data index 3.
  • the logical address range involved in the generation operation is [24 bits, 31 bits].
  • the key in data index 3 can be "A03”.
  • the position in data index 3 can be "offset 3, 8 bits”.
  • the logical address range of the data index to be generated is [32 bits, 71 bits], which can be aligned with the data index operation unit of the first length. Therefore, the storage system is based on the data index of the first length
  • the operation unit generates a data index 4.
  • the logical address range involved in the generation operation is [32 bits, 63 bits].
  • the key in data index 4 can be "A1ff", which means that [32 bits, 63 bits] is the first logical address range of the first length in file A, and "ff" is a preset character.
  • the position in data index 4 can be "offset 4, 32 bits".
  • the logical address range of the data index to be generated is [64 bits, 71 bits], which cannot be aligned with the data index operation unit of the first length. Therefore, the data index operation unit of the second length is based on Generate data index 5.
  • the logical address range involved in the generation operation is [64 bits, 71 bits].
  • the key in data index 3 can be "A20”.
  • the position in data index 5 can be "offset 5, 8 bits”.
  • data indexes are generated based on a larger-granularity data index operation unit, and the smaller the number of data indexes, the more storage space occupied by the data indexes is saved. For example, if the data index is generated based on the data index operation unit of the second length, for the above data index 4, 4 data indexes need to be generated. It can be seen that the technical solution provided by the embodiment of the present application helps to save data. The storage space occupied by the index.
  • This embodiment describes the modification process of the prefix tree.
  • the first length is 32 bits
  • the second length is 8 bits.
  • FIG. 8 it is a schematic diagram of a modification process of a prefix tree provided by an embodiment of this application.
  • the key in the data index generated by the storage system can be "A00", and the position can be "offset x, 8 bits", as shown in a in Figure 8. As shown in the figure.
  • the key in the data index generated by the storage system can be "A0ff", and the position can be "offset y, 32 bits".
  • the storage system can first insert the data index into the prefix tree; then, since the prefix tree contains the first target data index (that is, the data index as shown in figure a in Figure 8), the storage system can delete the first target The data is indexed, and the prefix tree shown in figure b in Figure 8 is obtained.
  • FIG. 9 it is a schematic diagram of another modification process of a prefix tree provided by an embodiment of this application.
  • the key in the data index 1 generated by the storage system can be "A0ff", and the position can be "offset x, 32 bits", as shown in Figure 9. As shown in figure a.
  • the key in the data index 2 generated by the storage system can be "A00", and the position can be "offset y, 8 bits".
  • the storage system can first insert the data index 2 into the prefix tree; then, since the prefix tree contains the second target data index (that is, the data index shown in figure a in Figure 9), the storage system can be The data index shown in the figure sets the status of multiple parts, and marks the status of the part corresponding to the logical address range [0 bit, 7 bit] in the multiple parts as invalid, thereby obtaining the b diagram in Figure 9 The prefix tree shown.
  • FIG. 10 it is a schematic flowchart of a method for managing data indexes in a storage system according to an embodiment of this application.
  • This embodiment specifically describes the method for managing data indexes executed in the data query process.
  • the method can include:
  • the storage system obtains the logical address to be queried.
  • the logical address range to be queried is aligned with the data index operation unit of the first length or aligned with the data index operation unit of the second length.
  • the logical address range to be queried in this embodiment may be the sub-logical address range described above.
  • the alignment of the logical address range to be queried with the data index operation unit of the first length may include: the first logical address of the logical address range to be queried can be divisible by the first length, and the length is the same as the first length.
  • the alignment of the logical address range to be queried with the data index operation unit of the second length may include: the first logical address of the logical address range to be queried can be divisible by the second length, and the length is the same as the second length.
  • S201 can be extended to: the storage system receives a data query request, and determines the logical address range to be queried based on the logical address range carried in the data query request.
  • the embodiment of the present application does not limit it. For example, refer to the related description of obtaining the sub-logical address range above.
  • S202 The storage system encodes the logical address range to be queried based on the data index operation unit of the second length to obtain the first encoding result.
  • the storage system can encode [0 bits, 7 bits] based on the data index operation unit of the second length, and obtain the first encoding result "00 ".
  • S203 The storage system judges whether the first encoding result exists in the cache of the storage system. For example, the storage system queries whether the key of the prefix tree contains the first encoding result. If yes, execute S204. If not, execute S205.
  • Example 1 Based on the example in S202, assuming that the prefix tree is as shown in figure b in Figure 9, the storage system can obtain the data index containing "key contains 00" in the prefix tree (ie data index 1) after querying, that is Hit.
  • Example 2 Based on the example in S202, assuming that the prefix tree is as shown in figure b in Figure 8, the storage system can obtain the data index that does not contain "key contains 00" in the prefix tree after querying, that is, there is no hit.
  • S204 The storage system determines the data of the logical address to be queried based on the data indicated by the second data index. Wherein, the key of the second data index includes the first encoding result. After S204 is executed, this query process ends.
  • the storage system can use the data indicated by the data index 1 as the data of the logical address to be queried.
  • the data indicated by data index 1 is the data of the logical address to be queried.
  • S205 The storage system encodes the logical address range to be queried based on the data index operation unit of the first length to obtain a second encoding result.
  • S206 The storage system judges whether the second encoding result exists in the cache of the storage system. For example, the storage system queries whether the key of the prefix tree contains the second encoding result. If yes, execute S207. If not, it means that the data of the logical address to be queried does not exist in the cache of the storage system, and S208 is executed.
  • the storage system may encode [0 bit, 7 bit] based on the data index operation unit of the first length to obtain the second encoding result "0ff".
  • S207 The storage system determines the data of the logical address to be queried based on the data indicated by the first data index; wherein the key of the first data index includes the second encoding result. After S207 is executed, this query process ends.
  • the storage system may use the data of the logical address to be queried among the data indicated by the data index in the b diagram in Figure 9 including "00" as the data requested by the data query request.
  • the storage system may obtain the data of the logical address to be queried based on the first data index and the second data index. For example, based on the prefix tree shown in Figure b in Figure 9, assuming that the logical address range to be queried is [0 bits, 31 bits], then both the first encoding result and the second encoding result will be hit.
  • the data of the logical address to be queried may include: the data indicated by the second data index, that is, the data whose logical address range is [0 bit, 7 bit]; and the logical address range of the partial data indicated by the first data index is [ 8-bit, 31-bit] data.
  • the data index operation unit based on multiple granularities encodes the logical address to be queried, if the encoding results are all hits, it means the logic indicated by the data index generated by the data index operation unit based on the smaller granularity.
  • the address range is that the data of the logical address range to be queried is new data
  • the logical address range indicated by the data index generated based on the larger-grained data index operation unit is that the data of the logical address to be queried is old data.
  • determining the data of the logical address to be queried based on the encoding result obtained by the smaller-granularity data index operation unit is helpful to make the queried data new data.
  • the smaller the granularity of the data index operation unit the higher the priority.
  • the storage system reads the first data index from the hard disk.
  • the logical address range corresponding to the first data index includes the logical address range to be queried.
  • the storage system can first read from the hard disk the logical address range of the first length data index operation unit where [0 bit, 7 bit] is located (ie [ 0 bit, 31 bit]) corresponding to the first data index. If the state of the part corresponding to [0 bit, 7 bit] in the first data index is marked as invalid, then the second data index corresponding to the logical address range [0 bit, 7 bit] is obtained from the hard disk.
  • the above S208 to S209 are a solution for querying the data index from the hard disk provided by the embodiment of the present application.
  • the data is first read from the cache from the data index operation unit based on a larger granularity. This helps reduce the number of times to read data from the hard disk. Since the rate of reading data from the hard disk is slow, executing S208-S209 will help speed up the data reading rate.
  • the storage system can merge multiple prefix trees stored in the hard disk.
  • the process of merging prefix trees is described.
  • the principle of merging prefix trees is similar to that of the prefix tree insertion process and modification process above. The difference is that the prefix tree insertion process and modification process described above are for the update of a prefix tree, and this implementation
  • the example mainly describes the merging of multiple prefix trees. The following takes the merging of the first prefix tree and the second prefix tree as an example for specific description:
  • the storage system merges the root nodes of the two into the same root node.
  • the storage system For an intermediate node in one of the first prefix tree and the second prefix tree, if the same intermediate node is found in the other prefix tree, the storage system merges the two intermediate nodes It is the same intermediate node. Otherwise, the storage system retains the intermediate node in the merged prefix tree.
  • the first prefix tree includes a first intermediate node
  • the second prefix tree includes a second intermediate node, if the root node of the first prefix tree (marked as the first root node) and the root node of the second prefix tree ( If the path from the first root node to the first intermediate node is the same as the path from the second root node to the second intermediate node, the first intermediate node is considered to be the same as the second intermediate node.
  • Scenario 1 Assuming that the generation time of the first prefix tree is before the second prefix tree, the first prefix tree includes a first data index, and the first data index is generated based on a data index operation unit of a first length.
  • the second prefix tree includes a second data index, and the second data index is generated based on the data index operation unit of the second length.
  • the logical address range corresponding to the first data index includes the logical address range corresponding to the second data index.
  • the merged prefix tree includes the first data index and the second data index, and the first data index includes the state of multiple parts, and the logical address range corresponding to the "second data index" in the multiple parts The status of the corresponding part is marked as invalid.
  • FIG. 11 it is a schematic diagram of a prefix tree merging process in this scenario.
  • Figure a in Figure 11 illustrates the first prefix tree.
  • the first prefix tree includes the first data index, the logical address range corresponding to the first data index is [0 bit, 31 bit], and the key of the first data index is "A0ff".
  • Figure b in Figure 11 illustrates the second prefix tree.
  • the second prefix tree includes a second data index, the logical address range corresponding to the second data index is [0 bit, 7 bit], and the key of the second data index is "A00".
  • the c diagram in Figure 11 illustrates the combined prefix tree.
  • the merged prefix tree includes the first data index and the second data index, and the first data index includes the state of multiple parts, and the state of the part corresponding to [0 bit, 7 bit] is marked as invalid.
  • Scenario 2 Assuming that the generation time of the first prefix tree is before the second prefix tree, the first prefix tree includes the second data index, and the second data index is generated based on the data index operation unit of the second length; the second prefix tree includes The first data index is generated based on the data index operation unit of the first length.
  • the logical address range corresponding to the first data index includes the logical address range corresponding to the second data index.
  • the merged prefix tree includes the first data index, but does not include the second data index.
  • FIG. 12 it is a schematic diagram of a prefix tree merging process in this scenario.
  • Figure a in Figure 12 illustrates the first prefix tree.
  • the first prefix tree includes a second data index, the logical address range corresponding to the second data index is [0 bits, 7 bits], and the key of the second data index is "A00".
  • Figure b in Figure 12 illustrates the second prefix tree.
  • the second prefix tree includes the first data index, the logical address range corresponding to the first data index is [0 bit, 31 bit], and the key of the first data index is "A0ff".
  • the c diagram in Figure 12 illustrates the combined prefix tree.
  • the combined prefix tree includes the first data index, but does not include the second data index.
  • Scenario 3 Based on the foregoing scenario 1 and/or scenario 2, the prefix tree is merged, and if each status bit of a certain data index in the merged prefix tree is marked as invalid, the storage system deletes the data index. This is a technical solution proposed in consideration of "the data indicated by the data index is old data". In this way, it helps to save the storage space of the data index and helps to improve the efficiency of garbage collection.
  • any data index (marked as the first data index) generated by the data index operation unit based on the first length, when the number of parts marked as invalid is greater than or equal to the threshold, it is based on not being marked as invalid.
  • a data index is generated for the logical address range corresponding to the part of, and the first data index is deleted. In this way, it helps to distinguish between new data and old data in the data query process, thereby improving the efficiency of data query; and, in the garbage collection process, distinguishing between new data and old data, thereby improving the efficiency of garbage collection.
  • This solution is equivalent to: when the number of parts marked as invalid in the data index generated by the data index operation unit of the first length is large, the data index generated by the data index operation unit of the first length is decomposed It is the data index generated by the data index operation unit based on the second length, and deletes the invalid data index generated by the data index operation unit based on the second length (that is, the data generated based on the second length and the indicated data is old data index). In this way, more garbage data can be deleted during the garbage collection process, thereby improving the efficiency of garbage collection.
  • This scheme can be applied in the garbage collection process.
  • the embodiments of the present application are not limited to this.
  • the solution can also be applied to the process of merging prefix trees, or the process of writing operations, etc.
  • FIG. 13 it is a schematic diagram of a process of decomposing a first data index provided by an embodiment of this application.
  • Figure a in Figure 13 illustrates the first data index, where the leaf node contains the offset x of the first data indicated by the first data index, the length of the first data 32 bits, and the status bit 0011, where , "0011" indicates that the logical address range [0 bit, 7 bit], [8 bit, 15 bit], [16 bit, 23 bit] and [24 bit, 31 bit] corresponding to the status bits are invalid, invalid, and valid respectively ,effective.
  • two data indexes as shown in figure b in FIG. 13 can be obtained after decomposing.
  • One of the data indexes is the data index corresponding to the logical address [16 bits, 23 bits], the key of the data index is "A02", and the position is "offset y and length 8 bits".
  • Another data index is the data index corresponding to the logical address [24 bits, 31 bits].
  • the key of the data index is "A03" and the position is "offset z and length 8 bits”.
  • the offset x in this example refers to the offset of the data indicated by the first data index in the storage system
  • the offset y refers to the logical address range [16 bits, 23 bits] in the storage system.
  • the offset z is the offset of the data in the logical address range [24 bits, 31 bits] in the storage system.
  • the data index operation unit includes a data index operation unit with a first length and a data index operation unit with a second length as an example.
  • the technical solutions provided in the embodiments of the present application can be applied to scenarios that include data index operation units with other granularities.
  • the embodiment of the present application does not elaborate on the specific implementation scheme under the data index operation unit including three or more different granularities. For example, based on the first embodiment above, in the process of generating the data index, if the logical address range of the data index to be generated cannot be aligned with the data index operation unit of the maximum granularity, it is determined whether the logical address range of the data index to be generated is the second largest The operating units are aligned, and so on.
  • the storage system tries to generate data indexes based on data index operation units with larger granularity.
  • the storage system tries its best to perform the complementation process based on the smaller-granularity data index operation unit.
  • Other examples will not be listed one by one.
  • the embodiments of the present application can divide the function modules of the data index management device in the storage system according to the above method examples.
  • each function module can be divided corresponding to each function, or two or more functions can be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 14 it is a schematic structural diagram of an apparatus for managing data indexes in a storage system provided by an embodiment of this application.
  • the data index management device 140 in the storage system can be used to perform the functions performed by the storage system in any of the above embodiments (the embodiment shown in FIG. 6 or FIG. 10).
  • the apparatus 140 for managing data indexes in the storage system may include: an obtaining unit 1401 and a generating unit 1402.
  • the acquiring unit 1401 is configured to acquire the first data to be written, and the logical address range of the first data to be written is the first logical address range.
  • the generating unit 1402 is configured to generate a data index based on the alignment between the logical address range of the data index to be generated in the first logical address range and the data index operation units of different lengths in the storage system; wherein the storage system includes a data index operation of the first length Unit and a data index operation unit of the second length; the first length is greater than the second length.
  • the acquiring unit 1401 may be used to perform S103 or S104, and the generating unit 1402 may be used to perform S106, S107, and S110.
  • the generating unit 1402 is specifically configured to: when the logical address range of the data index to be generated is aligned with the data index operation unit of the first length, generate the first data for the first data based on the data index operation unit of the first length Index; the first data is the data of the logical address range starting from the first logical address of the logical address range of the data index to be generated and the length is the first length in the first data to be written; wherein, the first data index includes the first data The location in the storage system.
  • the generating unit 1402 may be used to execute S107.
  • the generating unit 1402 is specifically configured to: when the logical address range of the data index to be generated is not aligned with the data index operation unit of the first length, generate a second data index operation unit for the second data based on the data index operation unit of the second length.
  • Data index wherein, the second data is data in the first logical address range of the logical address range to be generated from the logical address range of the data index to be generated and the length is the second length of the logical address range; the second data index includes the second The location of the data in the storage system.
  • the generating unit 1402 may be used to perform S110.
  • the logical address range of the data index to be generated is the same as The data index operation unit of the first length is aligned.
  • the device 140 for managing data indexes in the storage system further includes: a deleting unit 1403 and a setting unit 1404.
  • the deleting unit 1403 is configured to delete the first data index when the first target data index already exists in the storage system after the generating unit generates the first data index for the first data based on the data index operation unit of the first length.
  • Target data index wherein, the first target data index is a data index generated for the third data based on a data index operation unit of the second length; the logical address range of the first data includes the logical address range of the third data; the first target The data index contains the position of the third data in the storage system.
  • the deleting unit 1403 may be used to execute S109.
  • the setting unit 1404 is configured to: after the generating unit generates the second data index for the second data based on the data index operation unit of the second length, when the second target data index already exists in the storage system, the second target data index The status of the target part in the target data index is marked as invalid; among them, the second target data index is a data index generated for the fourth data based on the data index operation unit of the first length; the target part is the logical address of the fourth data The data index generated by the data of the second logical address range contained in the range; the second logical address range is the same as the logical address range of the second data.
  • the setting unit 1404 may be used to execute S112.
  • the deleting unit 1403 is configured to delete the second target data index when the states of all parts of the second target data index are marked as invalid.
  • the deleting unit 1403 is configured to generate a new data index for the valid parts of all parts when the number of invalid parts in all parts of the second target data index is greater than or equal to a threshold value, and Delete the second target data index.
  • the obtaining unit 1401 is further configured to obtain the logical address range to be queried.
  • the device also includes: an encoding unit 1405, a query unit 1406, and a determination unit 1407.
  • the encoding unit 1405 is configured to encode the logical address range to be queried based on the second length to obtain the first encoding result.
  • the query unit 1406 is configured to query the second data index based on the first encoding result; wherein, the second data index includes the first encoding result.
  • the determining unit 1407 is configured to determine the data of the logical address range to be queried based on the data indicated by the second data index.
  • the acquiring unit 1401 may be used to perform S201, and the encoding unit 1405 may be used to perform S202.
  • the query unit 1406 may be used to perform S203, and the determining unit 1407 may be used to perform S204.
  • the encoding unit 1405 is further configured to, when the first encoding result does not exist in the storage system, encode the logical address range to be queried based on the first length to obtain the second encoding result.
  • the query unit 1406 is further configured to query the first data index based on the second encoding result; wherein, the first data index includes the second encoding result.
  • the determining unit 1407 is further configured to determine the data of the logical address range to be queried based on the data indicated by the first data index. For example, in conjunction with FIG. 10, the encoding unit 1405 may be used to perform S205.
  • the query unit 1406 may be used to perform S206, and the determining unit 1407 may be used to perform S207.
  • the device 140 for managing data index in the storage system may be the storage controller 201.
  • some or all of the above-mentioned units can be called by the processor 301 to call the stored in the memory 302 Computer program implementation.
  • the device 140 for managing data indexes in the storage system may be a server.
  • some or all of the above-mentioned units can be called by the processor 501 to call the stored in the memory 502 Computer program implementation.
  • the embodiments of the present application also provide a computer-readable storage medium with computer program instructions stored on the computer-readable storage medium, and when the computer program instructions are executed on a computer, the computer executes any of the implementations provided above The actions or steps mentioned in the example.
  • the embodiment of the application also provides a chip.
  • the chip integrates a circuit and one or more interfaces for realizing the function of the data index management device 140 in the above-mentioned storage system.
  • the functions supported by the chip may include processing actions based on the embodiment described in FIG. 3 or FIG. 10, which will not be repeated here.
  • a person of ordinary skill in the art can understand that all or part of the steps for implementing the foregoing embodiments can be completed by instructing related hardware through computer program instructions.
  • the computer program instructions can be stored in a computer-readable storage medium.
  • the aforementioned storage medium may be a read-only memory, a random access memory, and the like.
  • the above-mentioned processing unit or processor may be a central processing unit, a general-purpose processor, an application specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field programmable gate array (field programmable gate array, FPGA) ) Or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • FPGA field programmable gate array
  • FPGA field programmable gate array
  • the embodiments of the present application also provide a computer program product containing computer program instructions.
  • the computer program instructions When the computer program instructions are executed on a computer, the computer executes any one of the methods in the foregoing embodiments.
  • the computer program product includes one or more computer program instructions.
  • the processes or functions described in the embodiments of the present application are generated in whole or in part.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • Computer program instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer program instructions can be transmitted from a website, computer, server, or data center through a cable (For example, coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (for example, infrared, wireless, microwave, etc.) to transmit to another website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or may include one or more data storage devices such as a server or a data center that can be integrated with the medium.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, an SSD).
  • the foregoing devices for storing computer program instructions provided in the embodiments of the present application are non-transitory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种存储系统中管理数据索引的方法和装置，涉及计算机技术领域，有助于实现使用合适粒度的操作单元创建数据索引，从而节省数据索引所占的存储空间。该方法包括：获取第一待写数据，第一待写数据的逻辑地址范围是第一逻辑地址范围；基于第一逻辑地址范围中待生成数据索引的逻辑地址范围与存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引；其中，存储系统包含第一长度的数据索引操作单元和第二长度的数据索引操作单元；第一长度大于第二长度。

Description

存储系统中管理数据索引的方法和装置
本申请要求于2019年12月23日提交国家知识产权局、申请号为201911342473.9、申请名称为“管理数据索引的方法和装置”的中国专利申请，以及于2019年12月26日提交国家知识产权局、申请号为201911369404.7、申请名称为“存储系统中管理数据索引的方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及存储系统中管理数据索引的方法和装置。
背景技术
数据索引(index)用于快速查询数据，使用数据索引可以屏蔽数据语义，提供一致性的高效的存储服务。将存储系统上数据的存储地址按照特定的数据索引操作单元为单位生成数据索引。
存储系统使用一种固态粒度的数据索引操作单元创建数据索引，根据存储系统中数据访问的多样性，无法满足访问数据访问需求。因此，如何使用合适粒度的数据索引操作单元创建数据索引成为存储系统要解决的问题。
发明内容
本申请实施例提供了存储系统中管理数据索引的方法和装置，有助于实现使用合适粒度的数据索引操作单元创建数据索引，从而节省数据索引所占的存储空间。
第一方面，提供了一种存储系统中管理数据索引的方法，包括：获取第一待写数据，第一待写数据的逻辑地址范围是第一逻辑地址范围。然后，基于第一逻辑地址范围中待生成数据索引的逻辑地址范围与存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引。其中，该存储系统包含第一长度的数据索引操作单元和第二长度的数据索引操作单元；第一长度大于第二长度。该技术方案中，支持基于不同长度的数据索引操作单元生成数据索引。由于对于同一逻辑地址范围来说，在保证与相应数据索引操作单元对齐的情况下，基于越大长度的数据索引操作单元生成数据索引，数据索引量越少，越节省数据索引所占的存储空间。因此，使用本技术方案有助于节省数据索引所占的存储空间。作为示例，待生成数据索引的逻辑地址范围，可以理解为，第一逻辑地址范围中的未参与生成数据索引的操作的逻辑地址范围。
在一种可能的设计中，基于第一逻辑地址范围中待生成数据索引的逻辑地址范围与存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引，具体包括：当待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元对齐时，基于第一长度的数据索引操作单元，为第一数据生成第一数据索引；第一数据是第一待写数据中从待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是第一长度的逻辑地址范围的数据。其中，第一数据索引包括第一数据在存储系统中的位置。
在一种可能的设计中，基于第一逻辑地址范围中待生成数据索引的逻辑地址范围与存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引，具体包括：当待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元不对齐时，基于第二长度的数据索引操作单元，为第二数据生成第二数据索引。其中，第二数据是第一待 写数据中从待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是第二长度的逻辑地址范围的数据；第二数据索引包括第二数据在存储系统中的位置。
这样，优先基于较大长度的数据索引操作单元生成数据索引，数据索引的数量越少，有助于节省数据索引所占的存储空间。也就是说，在生成数据索引的流程中，越大长度的数据索引操作单元，具有越高的优先级。
在一种可能的设计中，当待生成数据索引的逻辑地址范围的首个逻辑地址能被第一长度整除，且待生成数据索引的逻辑地址范围大于等于所述第一长度时，待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元对齐。
在一种可能的设计中，当待生成数据索引的逻辑地址范围的首个逻辑地址不能被第一长度整除，和/或，待生成数据索引的逻辑地址范围小于所述第一长度时，待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元不对齐。
在一种可能的设计中，在基于第一长度的数据索引操作单元，为第一数据生成第一数据索引之后，该方法还包括：当存储系统中已存在第一目标数据索引时，删除第一目标数据索引。第一目标数据索引是基于第二长度的数据索引操作单元，为第三数据生成的数据索引。第一数据的逻辑地址范围包括第三数据的逻辑地址范围。第一目标数据索引包含第三数据在存储系统中的位置。这样，有助于节省数据索引所占的存储空间，并且有助于在垃圾回收时，提高垃圾回收的效率。
在一种可能的设计中，在基于第一长度的数据索引操作单元，为第一数据生成第一数据索引之后，该方法还包括：当存储系统中已存在第一目标数据索引时，存储系统将第一目标数据索引标记为无效。
在一种可能的设计中，在基于第二长度的数据索引操作单元，为第二数据生成第二数据索引之后，该方法还包括：当存储系统中已存在第二目标数据索引时，将第二目标数据索引中的目标部分的状态标记为无效。其中，第二目标数据索引是基于第一长度的数据索引操作单元，为第四数据生成的数据索引。目标部分是为第四数据的逻辑地址范围中包含的第二逻辑地址范围的数据生成的数据索引。第二逻辑地址范围与第二数据的逻辑地址范围相同。这样，有助于在数据查询过程和垃圾回收过程中，区分新数据和旧数据，从而提高数据查询效率和垃圾回收效率。
在一种可能的设计中，存储系统可以使用状态位来标记第二目标数据索引中的每一部分(section)。
在一种可能的设计中，该方法还包括：当第二目标数据索引的所有部分的状态均被标记为无效时，删除第二目标数据索引。这样，有助于在垃圾回收过程中，区分新数据和旧数据，从而提高垃圾回收效率。
在一种可能的设计中，该方法还包括：当第二目标数据索引的所有部分中状态为无效的部分的个数大于或等于阈值时，将所该有部分中状态为有效的部分生成新的数据索引，并删除第二目标数据索引。这样，有助于进一步节省数据索引所占的存储空间；并且，在垃圾回收过程中，区分新数据和旧数据，从而提高垃圾回收效率。
在一种可能的设计中，该方法还包括：获取待查询逻辑地址范围；然后，基于第二长度对待查询逻辑地址范围进行编码，得到第一编码结果；接着，基于第一编码结果查询到第二数据索引。其中，第二数据索引包括第一编码结果。最后，基于第二数 据索引所指示的数据，确定待查询逻辑地址范围的数据。
在一种可能的设计中，该方法还包括：当存储系统中不存在第一编码结果时，基于第一长度对待查询逻辑地址范围进行编码，得到第二编码结果；然后，基于第二编码结果查询到第一数据索引；其中，第一数据索引包括第二编码结果；接着，基于第一数据索引所指示的数据，确定待查询逻辑地址范围的数据。
这样，优先基于较小长度的数据索引操作单元得到的编码结果，从而基于该编码结果确定待查询逻辑地址范围的数据，有助于保证查询到的待查询逻辑地址范围的数据是新数据。也就是说，在数据查询流程中，越小长度的数据索引操作单元，具有越高的优先级。
在一种可能的设计中，第一逻辑地址范围中的首个逻辑地址能被第二长度整除，第一逻辑地址范围的长度是第二长度的整数倍。
在一种可能的设计中，获取第一待写数据，具体包括：接收数据写请求；该数据写请求包括第一待写数据。或者，接收数据写请求，该数据写请求包括初始待写数据和初始逻辑地址范围；基于第二长度补齐初始待写数据，得到第一待写数据。也就是说，第一逻辑地址范围可以是写请求中携带的逻辑地址范围，或者是基于该逻辑地址范围进行补齐后的逻辑地址范围。并且，基于较小长度的数据索引操作单元进行数据补齐，这样，有助于降低读放大。
在一种可能的设计中，基于第一长度的数据索引操作单元为第一数据生成第一数据索引，包括：基于第一长度的数据索引操作单元对第一数据的逻辑地址范围进行编码，得到第一编码结果；第一数据索引(如第一数据索引的键)包括第一编码结果。
在一种可能的设计中，基于第二长度的数据索引操作单元为第二数据生成第二数据索引，包括：基于第二长度的数据索引操作单元对第二数据的逻辑地址范围进行编码，得到第一编码结果；第二数据索引(如第二数据索引的键)包括第二编码结果。
本申请实施例对具体的编码方法不进行限定，例如可以参考现有技术。
在一种可能的设计中，存储系统中包括第一前缀树和第二前缀树；第一前缀树的生成时间在第二前缀树之前；第一前缀树包括第一数据索引；第二前缀树包括第二数据索引。第一数据索引所指示的数据的逻辑地址范围包含第二数据索引所指示的数据的逻辑地址范围。该情况下，该方法还包括：合并第一前缀树和第二前缀树，得到合并后的前缀树。该合并后的前缀树中包括第一数据索引和第二数据索引。第一数据索引包括多个部分，该多个部分中的目标部分被标记为无效，该目标部分是第二数据索引所指示的数据的逻辑地址范围所对应的部分。该可能的设计提供了一种前缀树合并的方式，这样有助于在查询数据的流程和垃圾回收流程中区分新旧数据，从而提高数据查询效率和垃圾回收效率。
在一种可能的设计中，存储系统中包括第一前缀树和第二前缀树；第一前缀树的生成时间在第二前缀树之前；第一前缀树包括第二数据索引；第二前缀树包括第一数据索引。第一数据索引所指示的数据的逻辑地址范围包含第二数据索引所指示的数据的逻辑地址范围。该情况下，该方法还包括：合并第一前缀树和第二前缀树，得到合并后的前缀树。该合并后的前缀树中包括第一数据索引，但不包括第二数据索引。该可能的设计提供了另一种前缀树合并的方式，这样有助于节省数据索引所占的存储资 源。
在一种可能的设计中，该方法还可以包括：如果存储系统的缓存中不存在第二编码结果，则从存储系统的硬盘中读取第一数据索引。其中，第一数据索引对应的逻辑地址范围包含待查询逻辑地址范围；如果第一数据索引中的与待查询逻辑地址范围对应的部分的状态被标记为有效，则基于第一数据索引指示的数据确定待查询逻辑地址范围的数据。如果第一数据索引中与待查询逻辑地址范围对应的部分的状态被标记为无效，则从硬盘中获取该部分对应的逻辑地址范围的第二数据索引。该方案中，优先基于较大长度的数据索引操作单元从缓存中读取数据。这样，有助于减少从硬盘中读取数据的次数，从而加快数据读取速率。
第二方面，提供了一种存储系统中管理数据索引的装置，用于执行上述第一方面或第一方面提供的任一种方法。该装置可以是存储控制器或服务器或芯片等。
在一种可能的设计中，该装置包括用于执行第一方面或其任一种可能的设计提供的方法的各个模块。
在一种可能的设计中，该装置包括存储器和处理器，存储器用于存储计算机程序指令，处理器用于调用该计算机程序指令，以执行第一方面或其任一种可能的设计提供的方法。
第三方面，提供了一种计算机可读存储介质，其上储存有计算机程序指令，当该计算机程序指令被计算机调用时，使得计算机执行第一方面或其任一种可能的设计提供的方法。
第四方面，提供了一种计算机程序产品，当该计算程序产品包含的计算机程序指令在计算机上运行时，使得第一方面或其任一种可能的设计提供的方法被执行。
可以理解的是，上述提供的任一种存储系统中管理数据索引的装置、计算机可读存储介质或计算机程序产品均用于执行上文所提供的对应的存储系统中管理数据索引的方法，因此，其所能达到的有益效果可参考对应的方法中的有益效果，此处不再赘述。
附图说明
图1为可适用于本申请实施例的一种前缀树的示意图；
图2为可适用于本申请实施例的一种存储系统的示意图；
图3为可适用于本申请实施例的一种存储控制器的结构示意图；
图4为可适用于本申请实施例的另一种存储系统的示意图；
图5为可适用于本申请实施例的一种服务器的结构示意图；
图6为本申请实施例提供的一种存储系统中管理数据索引的方法的流程示意图；
图7为本申请实施例提供的一种前缀树的插入过程的示意图；
图8为本申请实施例提供的一种前缀树的修改过程的示意图；
图9为本申请实施例提供的另一种前缀树的修改过程的示意图；
图10为本申请实施例提供的另一种存储系统中管理数据索引的方法的流程示意图；
图11为本申请实施例提供的一种前缀树合并的过程示意图；
图12为本申请实施例提供的另一种前缀树合并的过程示意图；
图13为本申请实施例提供的一种分解第一数据索引的过程示意图；
图14为本申请实施例提供的一种存储系统中管理数据索引的装置的结构示意图。
具体实施方式
以下，说明本申请实施例中涉及的相关术语或技术：
1)、数据索引
存储系统中，数据索引用于指示数据在存储系统中的位置。
存储系统接收数据写请求，该数据写请求包含数据的逻辑地址范围。存储系统将数据存储到存储系统之后，会为该数据建立数据索引。该数据索引通常包含数据在存储系统中的位置。数据在存储系统中的位置具体可以是数据在存储系统中硬盘的位置或数据在存储系统卷上的位置等。存储系统为方便对数据索引进行管理，减少数据索引量，通常可以基于数据的逻辑地址范围的编码结果确定数据的键(key)，用于构建数据的数据索引。在一种实现方式中，键可以包含数据的标识、数据所在卷的标识或数据所属的文件的标识。在另一种实现方式中，键可以包含数据的标识、数据所在卷的标识或数据所属的文件的标识编码结果。存储系统通常以前缀树的形式管理数据索引。
本发明实施例中，逻辑地址为存储系统向客户端提供的访问地址。数据在存储系统中的位置是存储系统内部与逻辑地址对应的存储地址，用于存储数据。客户端可以为主机、服务器、虚拟机或应用等。
存储系统接收数据查询或读取请求，根据数据查询或读取请求中携带的数据标识和数据的逻辑地址范围生成相应的键，查询数据索引中是否包含该键，从而确定该数据在存储系统中的位置。
以文件系统为例，键可以包含数据所在文件的标识信息(或数据所在文件的标识信息的编码结果)，以及该数据的逻辑地址范围的编码结果。文件的标识信息可以包括文件名和/或文件路径。本申请实施例对文件的标识信息的编码方式不进行限定。为了描述上的简洁，下文中均以“键包括数据所在文件的文件名，以及该数据的逻辑地址范围的编码结果”为例进行说明。本申请实施例对数据的逻辑地址范围的编码方式不进行限定，具体示例可参考下文。
本申请实施例中，数据的长度可以使用数据的逻辑地址的范围来表征。例如，逻辑地址范围[0比特,7比特]的数据的长度是8比特。长度的标识可以是任意一种唯一表示该长度的信息，例如，长度的标识可以是该长度本身，或者标识符。例如，使用二进制数“1”表示第一长度，使用二进制数“0”表示第二长度。
2)、数据索引操作单元(grain)
数据索引操作单元，也称为数据索引粒度，即数据索引所指示的数据的长度。数据索引操作单元对应数据在存储系统的最小存储粒度和最小操作粒度，即最小逻辑地址长度。
通常，存储系统使用数据索引进行数据读写时，一次读写数据时该数据的逻辑地址范围需要和数据索引操作单元对齐。这样，有助于减少数据索引所占的存储空间，并加快读写速率。作为示例，一次读写数据时该数据的逻辑地址范围和数据索引操作单元对齐，可以是一次读写数据时该数据的首个逻辑地址的偏移量(如数据的首个逻 辑地址相对其所在文件的首个逻辑地址的偏移量)能够被该数据索引操作单元的长度整除，且该数据的长度与数据索引操作单元的长度相等。
例如，当数据索引操作单元是8比特时，存储系统一次读写数据时数据的首个逻辑地址的偏移量为8比特的整数倍，如0比特，8比特，16比特、24比特……。存储系统一次读写数据时数据的逻辑地址范围可以包括：[0比特，7比特]、[8比特，15比特]、[16比特，23比特]……。
又如，当数据索引操作单元是32比特时，存储系统一次读写数据时数据的首个逻辑地址的偏移量为32比特的整数倍，如0比特、32比特、64比特……。存储系统一次读写数据时数据的逻辑地址范围可以包括：[0比特，31比特]、[32比特，63比特]、[64比特，95比特]……。
3)、前缀树(tries tree)
前缀树，又称字典树，是一种有序树，用于保存关联数组，其中的键通常是字符串。与二叉查找树不同，前缀树的键不是直接保存在节点中，而是由节点在树中的位置决定。一个节点的所有子孙都有相同的前缀，也就是这个节点对应的字符串，而根节点对应空字符串。一般情况下，不是所有的节点都有对应的值，只有叶子节点和部分内部节点所对应的键才有相关的值。在本申请的一些实施例中，可以使用前缀树管理数据索引。
如图1所示，为可适用于本申请实施例的一种前缀树的示意图。该前缀树包括根节点和与根节点连接的若干个中间节点，以及与每个中间节点连接的若干个叶子节点。具体的，一个数据索引可以由前缀树中的根节点和一个叶子节点，以及连接二者的中间节点构成。
图1中示意出了3个叶子节点，这3个叶子节点分别对应数据索引1-3。其中，数据索引1中的键为abc，该键对应的值为叶子节点c中存储的地址1，该地址是数据索引1所指示的数据在存储系统中的位置。数据索引2中的键为abd，该键对应的值为叶子节点d中存储的地址2，地址2是数据索引2所指示的数据在存储系统中的位置。数据索引3中的键为efg，该键对应的值为叶子节点g中存储的地址3，该地址是数据索引3所指示的数据在存储系统中的位置。
4)、数据索引对应的逻辑地址范围
数据索引对应的逻辑地址范围，是指该数据索引所指示的数据的逻辑地址范围。
5)、垃圾回收(garbage collection)
在采用追加写的方式写入数据时，当数据发生修改时，修改前的数据会在存储系统中变成无效数据，即垃圾数据。修改前的数据和修改后的数据具有相同的逻辑地址范围，但在存储系统中具有不同的位置。因此，用于指示修改前的数据的数据索引的状态会变为无效。因此，需要进行垃圾回收。
垃圾回收，可以理解为：对存储系统中的数据进行整理，以删除无效数据，并且搬迁有效数据使数据紧缩，以节省存储空间。
在具体实现的过程中，存储系统可以基于数据索引操作单元进行垃圾回收。如果数据索引包含多个部分(section)，即将数据索引划为多个子粒度的数据索引。例如，数据索引的第一部分包含数据在存储系统中的位置为地址A，由于地址A中的数据发 生修改，即地址A中的数据的逻辑地址范围再次写入数据，因此，数据索引的第一部分的状态变为无效。具体实现时，存储系统可以为数据索引的每一个部分分配状态位，用于表征该部分的状态。当数据索引中状态为无效的部分的数量达到(即大于或等于)阈值，则对数据索引所表征的数据进行垃圾回收。当数据索引中状态为无效的部分的数量达到(即大于或等于)阈值时，可以将该数据索引中状态为有效的部分组成新的数据索引。当数据索引所有部分均变为无效时，可以直接删除该数据索引。
6)、其他术语
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请实施例中，“至少一个”是指一个或多个。“多个”是指两个或两个以上。
在本申请实施例中，“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
本申请实施例中的存储系统可以为存储阵列(如
Figure PCTCN2020134025-appb-000001
的
Figure PCTCN2020134025-appb-000002
系列，
Figure PCTCN2020134025-appb-000003
系列)。如图2所示，存储阵列包括存储控制器201和多块硬盘。硬盘包含固态硬盘(solid state disk，SSD)或者磁盘等。
如图3所示，存储控制器201包含处理器301、存储器302和接口303，存储器302中存储有计算机程序指令，处理器301执行存储器302中的计算机程序指令，一对存储系统进行管理及数据访问操作。中央处理单元(central processing unit，CPU)、现场可编程门阵列(field programmable gate array，FPGA)或其他硬件可以作为处理器301，或者，FPGA或其他硬件与CPU共同作为处理器301，处理器301与接口303通信。本申请实施例中的存储器302可以为处理器301提供内存。接口303可以为网络接口卡(networkinginterface card，NIC)或主机总线适配器(host bus adaptor，HBA)等。
在如图2和图3所描述的存储阵列中，存储控制器201可以用于执行本申请实施例中的管理数据索引的方法。
本申请实施例的存储系统还可以为分布式存储系统(如
Figure PCTCN2020134025-appb-000004
的
Figure PCTCN2020134025-appb-000005
系列)等。以
Figure PCTCN2020134025-appb-000006
的
Figure PCTCN2020134025-appb-000007
系列。示例性的，如图4所示，分布式存储系统包括多台服务器，如服务器1、服务器2、服务器3，……，服务器6，服务器间通过无限带宽(InfiniBand)或以太网络等互相通信。在实际应用当中，分布式存储系统中服务器的数量可以根据实际需求增加或减少，本申请实施例对此不作限定。分布式存储系统中的服务器又称为存储节点。
分布式存储系统的服务器中可以包含如图5所示的结构。如图5所示，分布式存储系统中的每台服务器可以包含处理器501、存储器502、接口503、硬盘1、硬盘2和硬盘3等。存储器502中存储有计算机程序指令。处理器501执行存储器502中的 计算机程序指令，以执行相应的操作。CPU、FPGA或其他硬件可以作为处理器501，或者，FPGA(或其他硬件)与CPU的组合共同作为处理器501。本申请实施例中的存储器502可以为处理器501提供内存。接口503可以为硬件接口，如NIC或HBA等，也可以为程序接口模块等。硬盘包含SSD或者磁盘等。
在分布式存储系统中，服务器可以用于执行本申请实施例中的管理数据索引的方法。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚地描述。
实施例一
如图6所示，为本申请实施例提供的一种存储系统中管理数据索引的方法的流程示意图。本实施例具体描述了“在数据写请求的触发条件下”存储系统执行的管理数据索引的方法。该方法可以包括如下步骤：
S101：存储系统接收数据写请求。该数据写请求可以是来自客户端的数据写请求。该数据写请求包括初始待写数据和初始待写数据的逻辑地址范围(即初始逻辑地址范围)。
作为示例，初始逻辑地址范围可以是任意长度的，具体可以基于客户端的需求确定。
可选的，该数据写请求还可以包括：初始待写数据所在的文件的文件名，以及初始待写数据在其所在文件中的路径等。
S102：存储系统判断是否需要补齐初始待写数据。若是，则执行S103；若否，则执行S104。
当初始逻辑地址范围中的首个逻辑地址能被第二长度整除，且初始逻辑地址范围的长度是第二长度的整数倍，则不需要补齐初始待写数据。例如，当第二长度是8比特，初始逻辑地址范围是[0比特，7比特]或[8比特，23比特]等，则不需要补齐初始待写数据。
当初始逻辑地址范围中的首个逻辑地址不能被第二长度整除，和/或初始逻辑地址范围的长度不是第二长度的整数倍，则需要补齐初始待写数据。例如，若第二长度是8比特，则：
示例1：当初始逻辑地址范围是[1比特,7比特]时，初始逻辑地址范围的首个逻辑地址(即1比特)不能被第二长度(即8比特)整除，因此需要补齐初始待写数据。
示例2：当初始逻辑地址范围是[0比特,6比特]时，初始逻辑地址范围的长度(即7比特)不是第二长度(即8比特)的整数倍，因此需要补齐初始待写数据。
示例3：当初始逻辑地址范围是[1比特,9比特]时，初始逻辑地址范围的首个逻辑地址(即1比特)不能被第二长度(即8比特)整除，且初始逻辑地址范围的长度(即9比特)不是第二长度(即8比特)的整数倍，因此需要补齐初始待写数据。
S103：存储系统基于第二长度的数据索引操作单元补齐初始待写数据,得到第一待写数据，并将第一待写数据的逻辑地址范围作为第一逻辑地址范围。执行S103之后，执行S105。
具体的，基于第二长度的数据索引操作单元补齐初始待写数据，可以包括：
当初始逻辑地址范围中的首个逻辑地址不能被第二长度整除，则存储系统从硬盘中读取该首个逻辑地址所在的第二长度的逻辑地址范围的数据；然后，基于初始待写数据和所读取的数据，得到第一待写数据。例如，基于上述示例1，初始逻辑地址范围是[1比特,7比特]，该情况下，存储系统可以从硬盘中读取逻辑地址范围[0比特,7比特]的数据，并将所读取的数据中逻辑地址范围是[1比特,7比特]的数据替换为初始待写数据，得到第一待写数据。此时，第一待写数据包括从硬盘中读取的逻辑地址是[0比特,1比特]的数据，以及数据写请求中的逻辑地址范围是[1比特,7比特]的初始待写数据，第一逻辑地址范围是[0比特,7比特]。
当初始逻辑地址范围中的首个逻辑地址能被第二长度整除，但是初始逻辑地址范围的长度不是第二长度的整数倍，则存储系统从硬盘中读取初始逻辑地址范围中的最后一个逻辑地址所在的第二长度的逻辑地址范围的数据；然后，基于初始待写数据和所读取的数据，得到第一待写数据。例如，基于上述示例2，初始逻辑地址范围是[0比特,6比特]，该情况下，存储系统可以从硬盘中读取逻辑地址范围[0比特，7比特]的数据，并将所读取的数据中逻辑地址范围是[0比特，6比特]的数据替换为初始待写数据，得到第一待写数据。此时，第一待写数据包括数据写请求中的逻辑地址范围是[0比特,6比特]的初始待写数据，以及从硬盘中读取逻辑地址是[6比特,7比特]的数据，第一逻辑地址范围是[0比特,7比特]。
当初始逻辑地址范围中的首个逻辑地址不能被第二长度整除，且初始逻辑地址范围的长度不是第二长度的整数倍，则从硬盘中读取初始逻辑地址范围中的首个逻辑地址所在的第二长度的逻辑地址范围的数据和最后一个逻辑地址所在的第二长度的逻辑地址范围的数据，然后，基于初始待写数据和所读取的数据，得到第一待写数据。
本申请实施例支持使用较小粒度的数据索引操作单元补齐初始待写数据，而非使用较大粒度的数据索引操作单元补齐初始待写数据，这有助于减小写数据过程中产生的读放大，从而提高写数据的过程中从硬盘中读取数据的速率，进而提高写数据的效率。其中，读放大是指从硬盘中读取数据的数据量大于待写数据的数据量。
S104：存储系统将初始待写数据作为第一待写数据，将初始逻辑地址范围作为第一逻辑地址范围。执行S104之后，执行S105。
S105：存储系统存储第一待写数据，并记录第一待写数据在存储系统中的位置。其中，存储系统向硬盘中写数据时，可以采用追加写的方式，当然不限于此。
存储系统存储第一待写数据，具体包含为第一待写数据在存储系统中分配存储位置，将第一待写数据写入该位置，并记录该第一待写数据在存储系统中的位置。该位置可以是数据在存储系统中硬盘的位置或数据在存储系统卷上的位置等。
执行S105之后，存储系统可以执行一次或多次S106-S113，直至基于第一逻辑地址范围中的每个子逻辑地址范围，均执行生成数据索引的操作为止。其中，子逻辑地址范围是与第一长度的数据索引操作单元对齐的逻辑地址范围，或者与第二长度的数据索引操作单元对齐的逻辑地址范围。
S106：存储系统判断待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元是否对齐。第1次执行S106-S113时，待生成数据索引的逻辑地址范围是第一逻辑地址范围。
若是，则执行S107；若否，则执行S110。
当待生成数据索引的逻辑地址范围中的首个逻辑地址能被第一长度整除，且待生成数据索引的逻辑地址范围的长度大于等于第一长度，则待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元对齐。
当待生成数据索引的逻辑地址范围中的首个逻辑地址不能被第一长度整除，和/或，待生成数据索引的逻辑地址范围的长度小于第一长度，则待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元不对齐。
以第一长度是32比特为例，当第一逻辑地址范围是[0比特,31比特]，则第一次执行S106时，待生成数据索引的逻辑地址范围中的首个逻辑地址是0比特，且待生成数据索引的逻辑地址范围的长度是32比特，满足上述对齐条件，则生成数据索引的逻辑地址范围与第一长度的数据索引操作单元对齐。如果第一逻辑地址范围是[0比特,12比特]，则第一次执行S106时，待生成数据索引的逻辑地址范围中的首个逻辑地址是0比特，但待生成数据索引的逻辑地址范围的长度是13比特，不满足上述对齐条件，则待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元不对齐。
S107：存储系统基于第一长度的数据索引操作单元，为第一数据生成第一数据索引。
第一数据索引包括第一键和第一数据在存储系统中的位置。例如，第一数据在硬盘中的偏移量和第一数据的长度(即第一长度)的标识信息。第一键可以包括数据写请求中携带的文件的标识信息的编码结果(下文中标记为文件A)和第一数据的逻辑地址范围的编码结果。第一数据在存储系统中的位置可以包括第一数据是从待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是第一长度的逻辑地址范围的数据。
可以理解的事，存储系统生成的前缀树先被存储在缓存中，随着生成的数据索引越来越多，前缀树种插入的节点会越来越多；当缓存中的前缀树中的节点的个数大于或等于某一阈值时，存储系统会将该前缀树写入硬盘。后续，当有数据写入时，存储系统会生成一个新的前缀树，并在该前缀树种继续插入数据索引。一种实现，S107中，还包括将第一数据索引插入前缀树。S107中所描述的前缀树可以是存储系统中的缓存中存储的前缀树。存储系统向前缀树中插入数据索引的具体示例可以参考下述图7。
可选的，S107可以包括：存储系统基于第一长度的数据索引操作单元，对第一数据的逻辑地址范围进行编码，得到第一数据的逻辑地址范围的编码结果。例如，使用“a”作为第一数据的逻辑地址范围的编码结果，其中，“a”表示第一数据的逻辑地址范围是文件A中的第a个第一长度的逻辑地址范围，a是整数。或者，使用“ax”作为第一数据的逻辑地址范围的编码结果，其中，“a”的含义参见上文，x可以是预设字符串。
例如，若第一长度是32比特，第二长度是8比特，第一数据的逻辑地址范围是[32比特,63比特]，则[32比特,63比特]的编码结果，可以是“1ff”，具体可以根据该逻辑地址范围的首个逻辑地址32比特与第一长度进行求模运算，即“32比特mod32比特＝1”，以及预设字符x具体是“ff”得到。
S108：存储系统查询是否包含第一目标数据索引。第一目标数据索引是基于第二长度的数据索引操作单元为第三数据生成的数据索引，第一数据的逻辑地址范围包括 第三数据的逻辑地址范围；第一目标数据索引包含第三数据在存储系统中的位置。
例如，假设第一长度是32比特，第二长度是8比特，第一数据是逻辑地址范围[0比特，31比特]的数据，则第一目标数据索引可以是存储系统分别基于逻辑地址范围[0比特，7比特]、[8比特，15比特]、[16比特，23比特]或[24比特，31比特]生成的数据索引。
基于步骤S108的查询，若否，说明存储系统首次写入“第一目标数据索引对应的逻辑地址范围(即第三数据的逻辑地址范围)”的数据，则执行S113。该情况下的示例可以参考图7。
基于步骤S108的查询，若是，说明存储系统并非首次写入“第一目标数据索引对应的逻辑地址范围(即第三数据的逻辑地址范围)”的数据，则执行S109。该情况下的示例可以参考图8。
基于前缀树实现的数据索引中，S108具体为存储系统查询该前缀树中是否包含第一目标数据索引。
S109：存储系统删除第一目标数据索引，执行S113。
对于“第一目标数据索引对应的逻辑地址范围”来说，第三数据是新数据，而第一目标数据索引所指示的数据是旧数据。删除旧数据的数据索引，有助于节省数据索引所占的存储空间，并且有助于在垃圾回收时，提高垃圾回收的效率。在基于前缀树实现的数据索引中，S109具体为存储系统删除该前缀树中的第一目标数据索引。
存储系统删除前缀树中已存在的数据索引的具体示例可以参考下述图8。
另外一种实现，存储系统将第一目标数据索引标记为无效。
S110：存储系统基于第二长度的数据索引操作单元，为第二数据生成第二数据索引。第二数据索引包括第二键和第二数据在存储系统中的位置。其中，第二键可以包括数据写请求中携带的文件的标识信息的编码结果(下文称为文件A)以及第二数据的逻辑地址范围的编码结果。第二数据是是第一待写数据中从待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是第二长度的逻辑地址范围的数据。在基于前缀树实现的数据索引中，S110还包括，将第二数据索引插入前缀树。
例如，第二数据在存储系统中的位置可以包括：第二数据在硬盘中的偏移量和第二数据的长度(即第二长度)的标识信息。
本发明实施例中，优先使用第一长度的数据索引操作单元生成数据索引，当待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元不对齐，则在一起第二长度的数据索引操作单元创建数据索引。
可选的，S110可以包括：存储系统基于第二长度的数据索引操作单元，对第二数据的逻辑地址范围进行编码，得到第二数据的逻辑地址范围的编码结果。例如，使用“ab”作为第二数据的逻辑地址范围的编码结果，“ab”表示第二数据的逻辑地址范围是文件A中的第a个第一长度的逻辑地址范围中的第b个第二长度的逻辑地址范围。a和b均为整数。
例如，若第一长度是32比特，第二长度是8比特，第二数据的逻辑地址范围是[8比特,15比特]，则[8比特，15比特]的编码结果，可以是“01”，用于表示[8比特，15比特]是文件A中的第0个第一长度的逻辑地址范围中的第1个第二长度的逻辑地 址范围。该编码结果“01”可以由该逻辑地址范围的首个逻辑地址8比特分别与第一长度和第二长度进行求模运算(即8比特mod32比特＝0和8比特mod8比特＝1)得到。
S111：存储系统查询是否包含第二目标数据索引。第二目标数据索引是基于第一长度的数据索引操作单元为第四数据生成的数据索引。第四数据的逻辑地址范围包括第二数据的逻辑地址范围。第二目标数据索引包含第四数据在存储系统中的位置。
在基于前缀树实现的数据索引中，S111具体为存储系统查询该前缀树中是否包含第二目标数据索引。
例如，假设第一长度是32比特，第二长度是8比特，第二数据是逻辑地址范围[8比特，15比特]的数据，则第二目标数据索引可以是存储系统基于“[8比特，15比特]的数据所在的长度为第一长度的逻辑地址范围(即逻辑地址范围[0比特，31比特])”生成的数据索引。
存储系统查询该前缀树中是否包含第二目标数据索引,若是，说明存储系统首次写入“第二数据索引对应的逻辑地址范围”的数据，则执行S113。该情况下的示例可以参考图7。
存储系统查询该前缀树中是否包含第二目标数据索引,若否，说明存储系统并非首次写入“第二数据索引对应的逻辑地址范围”的数据，则执行S112。该情况下的示例可以参考图9。
可选的，第二目标数据索引可以包括多个部分(Section)，每个部分的状态可以被标记为有效或无效，一个部分对应“第二目标数据索引对应的逻辑地址范围”中的一个与第二长度的数据索引操作单元对齐的逻辑地址范围。本申请实施例对如何标记每个部分的状态为是否无效(或是否有效)不进行限定。可选的，第二目标数据索引包括的状态位可以采用bitmap方式。作为示例，一个状态位可以是一个比特位。
例如，假设第一长度是32比特，第二长度是8比特，第二目标数据索引对应的逻辑地址范围是[0比特，31比特]，那么，第二目标数据索引可以包括4个状态位，这4个状态位分别与逻辑地址范围[0比特,7比特]、[8比特，15比特]、[16比特，23比特]和[24比特，31比特]对应，分别用于指示第二目标数据索引所指示的数据中的逻辑地址范围[0比特，7比特]、[8比特，15比特]、[16比特，23比特]和[24比特，31比特]的数据是否有效。
在一种实现方式中，存储系统可以在第二目标数据索引生成之后，生成多个部分的状态，并将该多个部分的状态标记为有效；然后，在需要将该多个状态中的某个或某些部分的状态标记为无效时，将该一个或多个部分的状态标记为无效。
在另一种实现方式中，存储系统可以在第二目标数据索引的某个或某些部分的状态需要标记为无效时，生成该多个部分的状态，并将该某个或某些部分的状态标记为无效。这样，有助于节省存储资源。
S112：存储系统将第二目标数据索引的目标部分的状态标记为无效。其中，目标部分是为第四数据的逻辑地址范围中包含的第二逻辑地址范围的数据生成的数据索引；第二逻辑地址范围与第二数据的逻辑地址范围相同。
对于“第二数据的逻辑地址范围(称为逻辑地址范围B)”来说，第二数据是新数据，第二目标数据索引对应的逻辑地址范围中的逻辑地址范围B的数据是旧数据。 执行S111和S112有助于在数据查询过程和垃圾回收过程中，区分新数据和旧数据，从而提高数据查询效率和垃圾回收效率。
可选的，第二目标数据索引的所有状态为均标记为无效时，说明第二目标数据索引所指示的所有数据均时无效的数据，因此，第二目标数据索引可以被删除，以节省数据索引所占的存储空间。
S113：存储系统判断第一逻辑地址范围中的每个子逻辑地址范围是否均参与生成操作。
若是，则本次数据写请求的操作流程结束。若否，则返回执行S106。
本申请实施例提供的存储系统中管理数据索引的方法，支持基于不同长度的数据索引操作单元对数据索引进行管理，并且，可以基于数据写操作携带的逻辑地址范围选择不同长度的操作单元生成数据索引。由于对于同一逻辑地址范围来说，在保证与相应操作单元对齐的情况下，基于越大长度的操作单元生成数据索引，则数据索引数量越少，越节省数据索引所占的存储空间。因此，使用本申请实施例提供的技术方案有助于节省数据索引所占的存储空间。
以下，通过描述前缀树的插入和修改过程，对上文中描述的存储系统中管理数据索引的方法进行说明。
实施例1
如图7所示，为本实施例提供的一种前缀树的插入过程的示意图。本实施例中，“数据写请求包括文件A的文件名“A”、第一待写数据和第一逻辑地址范围[8比特，71比特]，且存储系统首次写入第一逻辑地址范围的数据。第一长度是32比特，第二长度是8比特。
图7中的a图示意出了第一逻辑地址范围所分成的多个子逻辑地址范围，一个子逻辑地址范围表示：一次执行S106-S113时，参与生成操作的逻辑地址范围。
图7中的b图示意出了每个子逻辑地址范围的数据在存储系统中的位置，其中，位置包括偏移量和数据长度。其中，各子逻辑地址范围对应的偏移量分别标记为偏移量1-5。
图7中的c图示意出了执行数据写请求之前和之后的前缀树。执行数据写请求之前，前缀树中不存在数据索引，只存在一个根节点。执行数据写请求之后，前缀树中存在数据索引1-5。
具体分析过程如下：
第1次执行上述S106-S113时，待生成数据索引的逻辑地址范围是[8比特，71比特]，不能与第一长度的数据索引操作单元对齐，因此，存储系统基于第二长度的数据索引操作单元生成数据索引1。本次执行S106-S113时，参与生成操作的逻辑地址范围是[8比特,15比特]。数据索引1中的键可以是“A01”，表示[8比特,15比特]是文件A中的第0个第一长度的逻辑地址范围中的第1个第二长度的逻辑地址范围。数据索引1中的位置可以为“偏移量1，8比特”。
第2次执行上述S106-S113时，待生成数据索引的逻辑地址范围是[16比特，71比特]，不能与第一长度的数据索引操作单元对齐，因此，存储系统基于第二长度的数据索引操作单元生成数据索引2。本次执行S106-S113时，参与生成操作的逻辑地址 范围是[16比特,23比特]。数据索引2中的键可以是“A02”。数据索引2中的位置可以为“偏移量2，8比特”。
第3次执行上述S106-S113时，待生成数据索引的逻辑地址范围是[24比特，71比特]，不能与第一长度的数据索引操作单元对齐，因此，存储系统基于第二长度的数据索引操作单元生成数据索引3。本次执行S106-S113时，参与生成操作的逻辑地址范围是[24比特,31比特]。数据索引3中的键可以是“A03”。数据索引3中的位置可以为“偏移量3，8比特”。
第4次执行上述S106-S113时，待生成数据索引的逻辑地址范围是[32比特，71比特],能与第一长度的数据索引操作单元对齐，因此，存储系统基于第一长度的数据索引操作单元生成数据索引4。本次执行S106-S113时，参与生成操作的逻辑地址范围是[32比特,63比特]。数据索引4中的键可以是“A1ff”，表示[32比特,63比特]是文件A中的第1个第一长度的逻辑地址范围，“ff”是预设字符。数据索引4中的位置可以为“偏移量4，32比特”。
第5次执行上述S106-S113时，待生成数据索引的逻辑地址范围是[64比特，71比特]，不能与第一长度的数据索引操作单元对齐，因此，基于第二长度的数据索引操作单元生成数据索引5。本次执行S106-S113时，参与生成操作的逻辑地址范围是[64比特,71比特]。数据索引3中的键可以是“A20”。数据索引5中的位置可以为“偏移量5，8比特”。
至此，第一逻辑地址范围中的每个子逻辑地址范围均已参与生成操作。
由于对于同一逻辑地址范围来说，在保证对齐的情况下，基于越大粒度的数据索引操作单元生成数据索引，数据索引的数量越少，越节省数据索引所占的存储空间。例如，如果基于第二长度的数据索引操作单元生成数据索引，则对于上述数据索引4来说，需要生成4个数据索引，由此可知，本申请实施例提供的技术方案，有助于节省数据索引所占的存储空间。
实施例2
本实施例描述了前缀树的修改流程。本实施例中，第一长度是32比特，第二长度是8比特。
如图8所示，为本申请实施例提供的一种前缀树的修改流程的示意图。该示例中，是以先写入“文件A中逻辑地址范围[0比特，7比特]”的数据，再写入“文件A中逻辑地址范围[0比特，31比特]”为例进行说明的。具体的：
写入逻辑地址范围[0比特，7比特]的数据之后，存储系统生成的数据索引中的键可以为“A00”，位置可以为“偏移量x，8比特”，如图8中的a图所示。
写入逻辑地址范围[0比特，31比特]的数据之后，存储系统生成的数据索引中的键可以为“A0ff”，位置可以为“偏移量y，32比特”。存储系统可以先将该数据索引插入前缀树；接着，由于前缀树中包含第一目标数据索引(即如图8中的a图所示的数据索引)，因此，存储系统可以删除该第一目标数据索引，从而得到如图8中的b图所示的前缀树。
如图9所示，为本申请实施例提供的另一种前缀树的修改流程的示意图。该示例中，是以先写入“文件A中逻辑地址范围[0比特，31比特]”的数据，再写入“文件 A中逻辑地址范围[0比特，7比特]”为例进行说明的。具体的：
写入逻辑地址范围[0比特，31比特]的数据之后，存储系统生成的数据索引1中的键可以为“A0ff”，位置可以为“偏移量x，32比特”，如图9中的a图所示。
写入逻辑地址范围[0比特，7比特]的数据之后，存储系统生成的数据索引2中的键可以为“A00”，位置可以为“偏移量y，8比特”。存储系统可以先将数据索引2插入前缀树；接着，由于前缀树中包含第二目标数据索引(即图9中的a图所示的数据索引)，因此，存储系统可以在图9中的a图所示的数据索引设置多个部分的状态，并将该多个部分中的与逻辑地址范围[0比特，7比特]对应的部分的状态标记为无效，从而得到如图9中的b图所示的前缀树。
实施例二
如图10所示，为本申请实施例提供的一种存储系统中管理数据索引的方法的流程示意图。本实施例具体描述了数据查询过程中执行的管理数据索引的方法。该方法可以包括：
S201：存储系统获取待查询逻辑地址。其中，待查询逻辑地址范围与第一长度的数据索引操作单元对齐或与第二长度的数据索引操作单元对齐。也就是说，本实施例中的待查询逻辑地址范围可以是上文中所描述的子逻辑地址范围。
待查询逻辑地址范围与第一长度的数据索引操作单元对齐可以包括：待查询逻辑地址范围的首个逻辑地址能够被第一长度整除，且长度与第一长度相同。
待查询逻辑地址范围与第二长度的数据索引操作单元对齐可以包括：待查询逻辑地址范围的首个逻辑地址能够被第二长度整除，且长度与二长度相同。
S201可以扩展为：存储系统接收数据查询请求，并基于该数据查询请求中携带的逻辑地址范围确定待查询逻辑地址范围。关于存储系统确定待查询逻辑地址范围的具体实现方式，本申请实施例不进行限定。例如，可以参考上文中的获得子逻辑地址范围的相关描述。
S202：存储系统基于第二长度的数据索引操作单元，对待查询逻辑地址范围进行编码，得到第一编码结果。
例如，假设待查询逻辑地址范围是[0比特,7比特]，那么，存储系统可以基于第二长度的数据索引操作单元，对[0比特,7比特]进行编码，得到第一编码结果“00”。
S203：存储系统判断存储系统的缓存中是否存在第一编码结果。例如，存储系统查询前缀树的键中是否包含第一编码结果。若是，则执行S204。若否，则执行S205。
示例1：基于S202中的示例，假设前缀树如图9中的b图所示，则存储系统经查询，可以得到前缀树中包含“键包含00”的数据索引(即数据索引1)，即命中。
示例2：基于S202中的示例，假设前缀树如图8中的b图所示，则存储系统经查询，可以得到前缀树中不包含“键包含00”的数据索引，即没有命中。
S204：存储系统基于第二数据索引所指示的数据，确定待查询逻辑地址的数据。其中，第二数据索引的键包括第一编码结果。执行S204之后，则本次查询过程结束。
基于S203中的示例1，存储系统可以将数据索引1所指示的数据作为待查询逻辑地址的数据。该示例中，数据索引1所指示的数据为待查询逻辑地址的数据。
S205：存储系统基于第一长度的数据索引操作单元，对待查询逻辑地址范围进行 编码，得到第二编码结果。
S206：存储系统判断存储系统的缓存中是否存在第二编码结果。例如，存储系统查询前缀树的键中是否包含第二编码结果。若是，则执行S207。若否，说明存储系统的缓存中不存在待查询逻辑地址的数据，则执行S208。
例如，基于S203中的示例2，存储系统可以基于第一长度的数据索引操作单元，对[0比特,7比特]进行编码，得到第二编码结果“0ff”。
S207：存储系统基于第一数据索引所指示的数据，确定待查询逻辑地址的数据；其中，第一数据索引的键包括第二编码结果。执行S207之后，则本次查询过程结束。
例如，基于S206中的示例，存储系统可以将图9中的b图中的键包括“00”的数据索引所指示的数据中的待查询逻辑地址的数据作为数据查询请求所请求的数据。
在另一示例中，执行S207时，存储系统可以基于第一数据索引和第二数据索引，获取待查询逻辑地址的的数据。例如，基于图9中的b图所示的前缀树，假设待查询逻辑地址范围是[0比特，31比特]，那么，第一编码结果和第二编码结果均会被命中，该情况下，待查询逻辑地址的数据可以包括：第二数据索引所指示的数据，即逻辑地址范围是[0比特,7比特]的数据；以及，第一数据索引所指示的部分数据中逻辑地址范围是[8比特，31比特]的数据。
上述S201-S207中，在基于多种粒度的数据索引操作单元对待查询逻辑地址进行编码之后，如果编码结果均被命中，说明基于较小粒度的数据索引操作单元生成的数据索引所指示的”逻辑地址范围是待查询逻辑地址范围的数据是新数据，而基于较大粒度的数据索引操作单元生成的数据索引所指示的逻辑地址范围是待查询逻辑地址的数据是旧数据，具体示例可以参考上述对图9所示的示例的分析。该情况下，优先基于较小粒度的数据索引操作单元得到的编码结果，确定待查询逻辑地址的的数据，有助于使得查询到的数据是新数据。也就是说，在查询流程中，越小粒度的数据索引操作单元，具有越高的优先级。
S208：存储系统从硬盘中读取第一数据索引。第一数据索引对应的逻辑地址范围包含待查询逻辑地址范围。
S209：如果第一数据索引中与待查询逻辑地址范围对应的部分的状态被标记为有效，则基于第一数据索引指示的数据确定待查询逻辑地址范围的数据。当第一数据索引中与待查询逻辑地址范围对应的部分的状态被标记为无效，则从硬盘中获取该部分对应的逻辑地址范围的第二数据索引。
例如，假设待查询逻辑地址范围是[0比特,7比特]，存储系统可以先从硬盘中读取[0比特,7比特]所在的第一长度的数据索引操作单元的逻辑地址范围(即[0比特,31比特])对应的第一数据索引。如果第一数据索引中[0比特,7比特]对应的部分的状态被标记为无效，则再从硬盘中获取逻辑地址范围[0比特,7比特]对应的第二数据索引。
上述S208～S209是本申请实施例提供的一种从硬盘中查询数据索引的方案，该方案中，优先从基于较大粒度的数据索引操作单元从缓存中读取数据。这样，有助于减少从硬盘中读取数据的次数。由于从硬盘中读取数据的速率较慢，因此，执行S208-S209，有助于加快数据读取速率。
实施例三
存储系统可以对存入硬盘的多个前缀树进行合并，本实施例中说明合并前缀树的流程。
合并前缀树所遵循的原则与上文中前缀树插入过程和修改过程所遵循的原则类似，区别在于，上文中所描述的前缀树的插入过程和修改过程是针对一个前缀树的更新，而本实施例中主要描述多个前缀树的合并。以下以对第一前缀树和第二前缀树合并为例进行具体说明：
对于第一前缀树和第二前缀树中的根节点来说，存储系统将二者的根节点合并为同一根节点。
对于第一前缀树和第二前缀树中的其中一个前缀树中的一个中间节点来说，如果在另一个前缀树中找到了与之相同的中间节点，则存储系统将这两个中间节点合并为同一个中间节点。否则，存储系统在合并后的前缀树中保留该中间节点。其中，假设第一前缀树中包括第一中间节点，第二前缀树中包括第二中间节点，如果第一前缀树的根节点(标记为第一根节点)和第二前缀树的根节点(标记为第二根节点)相同，且第一根节点到第一中间节点的路径与第二根节点到第二中间节点的路径相同，则认为第一中间节点与第二中间节点相同。
对于第一前缀树和第二前缀树中的任一个叶子节点(即对于任一个数据索引)来说，如果在另一个前缀树中没有找到与之对应的逻辑地址范围重叠的数据索引，则存储系统在合并后的前缀树中保留该叶子节点。如果在另一个前缀树中找到了与之对应的逻辑地址范围重叠的数据索引，则基于以下场景1至场景3中的一种或多种合并前缀树：
场景1：假设第一前缀树的生成时间在第二前缀树之前，第一前缀树包括第一数据索引，第一数据索引是基于第一长度的数据索引操作单元生成的。第二前缀树包括第二数据索引，第二数据索引是基于第二长度的数据索引操作单元生成的。第一数据索引对应的逻辑地址范围包含第二数据索引对应的逻辑地址范围。该情况下，合并后的前缀树中包含第一数据索引和第二数据索引，且第一数据索引包含多个部分的状态，该多个部分中的与“第二数据索引对应的逻辑地址范围”对应的部分的状态被标记为无效。
例如，如图11所示，为该场景下的一种前缀树合并的过程示意图。
图11中的a图示意出了第一前缀树。第一前缀树包括第一数据索引，第一数据索引对应的逻辑地址范围是[0比特，31比特]，第一数据索引的键为“A0ff”。
图11中的b图示意出了第二前缀树。第二前缀树包括第二数据索引，第二数据索引对应的逻辑地址范围是[0比特，7比特]，第二数据索引的键为“A00”。
图11中的c图示意出了合并后的前缀树。该合并后的前缀树中包括第一数据索引和第二数据索引，且第一数据索引包括多个部分的状态，且与[0比特，7比特]对应的部分的状态被标记为无效。
场景2：假设第一前缀树的生成时间在第二前缀树之前，第一前缀树包括第二数据索引，第二数据索引是基于第二长度的数据索引操作单元生成的；第二前缀树包括第一数据索引，第一数据索引是基于第一长度的数据索引操作单元生成的。第一数据 索引对应的逻辑地址范围包含第二数据索引对应的逻辑地址范围。该情况下，合并后的前缀树中包含第一数据索引，但不包含第二数据索引。
例如，如图12所示，为该场景下的一种前缀树合并的过程示意图。
图12中的a图示意出了第一前缀树。第一前缀树包括第二数据索引，第二数据索引对应的逻辑地址范围是[0比特，7比特]，第二数据索引的键为“A00”。
图12中的b图示意出了第二前缀树。第二前缀树包括第一数据索引，第一数据索引对应的逻辑地址范围是[0比特，31比特]，第一数据索引的键为“A0ff”。
图12中的c图示意出了合并后的前缀树。该合并后的前缀树中包括第一数据索引，但不包括第二数据索引。
这是在考虑到“第二数据索引所指示的数据是旧数据”而提出的技术方案。这样，有助于节省数据索引的存储空间，且有助于提高垃圾回收的效率。
场景3：基于上述场景1和/或场景2进行合并前缀树，如果合并后的前缀树中的某一数据索引的各状态位均被标记为无效，则存储系统删除该数据索引。这是在考虑到“该数据索引所指示的数据是旧数据”而提出的技术方案。这样，有助于节省数据索引的存储空间，且有助于提高垃圾回收的效率。
实施例四
对于基于第一长度的数据索引操作单元生成的任一数据索引(标记为第一数据索引)来说，当其被标记为无效的部分的个数大于或等于阈值，则基于没有被标记为无效的部分对应的逻辑地址范围分别生成一个数据索引，并删除该第一数据索引。这样，有助于在数据查询过程中区分新数据和旧数据，从而提高数据查询效率；以及，在垃圾回收过程中，区分新数据和旧数据，从而提高垃圾回收效率。
该方案等价于：当基于第一长度的数据索引操作单元生成的数据索引中的被标记为无效的部分的个数较多时，将基于第一长度的数据索引操作单元生成的数据索引，分解为基于第二长度的数据索引操作单元生成的数据索引，并删除无效的基于第二长度的数据索引操作单元生成的数据索引(即基于第二长度生成的且所指示的数据是旧数据的数据索引)。这样，垃圾回收过程中，可以将更多的垃圾数据删除，从而提高垃圾回收效率。
该方案可以应用在垃圾回收流程中。当然本申请实施例不限于此。例如，该方案还可以应用于合并前缀树的流程中，或者写操作流程中等。
如图13所示，为本申请实施例提供的一种分解第一数据索引的过程示意图。图13中的a图中示意出了第一数据索引，其中，叶子节点中包含第一数据索引所指示的第一数据的偏移量x、第一数据的长度32比特和状态位0011，其中，“0011”表示逻辑地址范围[0比特，7比特]、[8比特，15比特]、[16比特，23比特]和[24比特，31比特]对应的状态位分别为无效、无效、有效、有效。假设此时需要将第一数据索引进行分解，则分解后可以得到如图13中的b图所示的2个数据索引。其中一个数据索引是逻辑地址[16比特，23比特]对应的数据索引，该数据索引的键为“A02”，位置为“偏移量y和长度8比特”。另外一个数据索引是逻辑地址[24比特，31比特]对应的数据索引，该数据索引的键为“A03”，位置为“偏移量z和长度8比特”。其中，该示例中的偏移量x是指第一数据索引所指示的数据在存储系统中的偏移量， 偏移量y是指逻辑地址范围[16比特，23比特]的数据在存储系统中的偏移量，偏移量z是逻辑地址范围[24比特，31比特]的数据在存储系统中的偏移量。
上文中均是以数据索引操作单元包括第一长度的数据索引操作单元和第二长度的数据索引操作单元为例进行说明的。可扩展地，本申请实施例提供的技术方案可以应用于包含其他粒度的数据索引操作单元的场景。本申请实施例对包括三种或三种以上不同粒度的数据索引操作单元下的具体实现方案不进行详述。例如，基于上述实施例一，在生成数据索引的过程中，如果待生成数据索引的逻辑地址范围不能与最大粒度的数据索引操作单元对齐，则判断待生成数据索引的逻辑地址范围是否与次大操作单元对齐，依此类推。也就是说，存储系统尽量基于粒度较大的数据索引操作单元生成数据索引。另外，存储系统尽量基于较小粒度的数据索引操作单元执行补齐流程。其他示例不再一一列举。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的方法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对存储系统中管理数据索引的装置进行功能模块的划分，例如可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图14所示，为本申请实施例提供的一种存储系统中管理数据索引的装置的结构示意图。该存储系统中管理数据索引的装置140可以用于执行上文中任意一个实施例(如图6或图10所示的实施例)中存储系统所执行的功能。该存储系统中管理数据索引的装置140可以包括：获取单元1401和生成单元1402。
获取单元1401用于获取第一待写数据，第一待写数据的逻辑地址范围是第一逻辑地址范围。生成单元1402用于基于第一逻辑地址范围中待生成数据索引的逻辑地址范围与存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引；其中，存储系统包含第一长度的数据索引操作单元和第二长度的数据索引操作单元；第一长度大于第二长度。例如，结合图6，获取单元1401可以用于执行S103或S104，生成单元1402可以用于执行S106、S107和S110。
可选的，生成单元1402具体用于：当待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元对齐时，基于第一长度的数据索引操作单元，为第一数据生成第一数据索引；第一数据是第一待写数据中从待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是第一长度的逻辑地址范围的数据；其中，第一数据索引包括第一数据在存储系统中的位置。例如，结合图6，生成单元1402可以用于执行S107。
可选的，生成单元1402具体用于：当待生成数据索引的逻辑地址范围与第一长度 的数据索引操作单元不对齐时，基于第二长度的数据索引操作单元，为第二数据生成第二数据索引；其中，第二数据是第一待写数据中从待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是第二长度的逻辑地址范围的数据；第二数据索引包括第二数据在存储系统中的位置。例如，结合图6，生成单元1402可以用于执行S110。
可选的，当待生成数据索引的逻辑地址范围的首个逻辑地址能被第一长度整除，且待生成数据索引的逻辑地址范围大于等于第一长度时，待生成数据索引的逻辑地址范围与第一长度的数据索引操作单元对齐。
可选的，该存储系统中管理数据索引的装置140还包括：删除单元1403和设置单元1404。
可选的，删除单元1403，用于在生成单元基于第一长度的数据索引操作单元，为第一数据生成第一数据索引之后，当存储系统中已存在第一目标数据索引时，删除第一目标数据索引；其中，第一目标数据索引是基于第二长度的数据索引操作单元，为第三数据生成的数据索引；第一数据的逻辑地址范围包括第三数据的逻辑地址范围；第一目标数据索引包含第三数据在存储系统中的位置。例如，结合图6，删除单元1403可以用于执行S109。
可选的，设置单元1404，用于在生成单元基于第二长度的数据索引操作单元，为第二数据生成第二数据索引之后，当存储系统中已存在第二目标数据索引时，将第二目标数据索引中的目标部分的状态标记为无效；其中，第二目标数据索引是基于第一长度的数据索引操作单元，为第四数据生成的数据索引；目标部分是为第四数据的逻辑地址范围中包含的第二逻辑地址范围的数据生成的数据索引；第二逻辑地址范围与第二数据的逻辑地址范围相同。例如，结合图6，设置单元1404可以用于执行S112。
可选的，删除单元1403，用于当第二目标数据索引的所有部分的状态均被标记为无效时，删除第二目标数据索引。
可选的，删除单元1403，用于当第二目标数据索引的所有部分中状态为无效的部分的个数大于或等于阈值时，将所有部分中状态为有效的部分生成新的数据索引，并删除第二目标数据索引。
可选的，获取单元1401还用于，获取待查询逻辑地址范围。该装置还包括：编码单元1405、查询单元1406和确定单元1407。编码单元1405，用于基于第二长度对待查询逻辑地址范围进行编码，得到第一编码结果。查询单元1406，用于基于第一编码结果查询到第二数据索引；其中，第二数据索引包括第一编码结果。确定单元1407，用于基于第二数据索引所指示的数据，确定待查询逻辑地址范围的数据。例如，结合图10，获取单元1401可以用于执行S201，编码单元1405可以用于执行S202。查询单元1406可以用于执行S203，确定单元1407可以用于执行S204。
可选的，编码单元1405还用于，当存储系统中不存在第一编码结果时，基于第一长度对待查询逻辑地址范围进行编码，得到第二编码结果。查询单元1406还用于，基于第二编码结果查询到第一数据索引；其中，第一数据索引包括第二编码结果。确定单元1407还用于，基于第一数据索引所指示的数据，确定待查询逻辑地址范围的数据。例如，结合图10，编码单元1405可以用于执行S205。查询单元1406可以用于执行S206，确定单元1407可以用于执行S207。
在一个示例中，参见图2和图3，存储系统中管理数据索引的装置140可以是存储控制器201。参见图3，上述各单元(如获取单元1401、生成单元1402、删除单元1403、设置单元1404、编码单元1405和查询单元1406)中的部分或全部均可以由处理器301调用存储器302中存储的计算机程序实现。
在另一个示例中，参见图4，存储系统中管理数据索引的装置140可以是服务器。参见图5，上述各单元(如获取单元1401、生成单元1402、删除单元1403、设置单元1404、编码单元1405和查询单元1406)中的部分或全部均可以由处理器501调用存储器502中存储的计算机程序实现。
关于上述可选方式的具体描述参见前述的方法实施例，此处不再赘述。此外，上述提供的任一种存储系统中管理数据索引的装置140的解释以及有益效果的描述均可参考上述对应的方法实施例，不再赘述。
需要说明的是，上述各个模块对应执行的动作仅是具体举例，各个模块实际执行的动作参照上述基于图6或图10所述的实施例的描述中提及的动作或步骤。
本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序指令，当该计算机程序指令在计算机上运行时，使得该计算机执行上文提供的任一实施例中提及的动作或步骤。
本申请实施例还提供了一种芯片。该芯片中集成了用于实现上述存储系统中管理数据索引的装置140的功能的电路和一个或者多个接口。可选的，该芯片支持的功能可以包括基于图3或图10所述的实施例中的处理动作，此处不再赘述。本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可通过计算机程序指令来指令相关的硬件完成。所述的计算机程序指令可以存储于一种计算机可读存储介质中。上述提到的存储介质可以是只读存储器，随机接入存储器等。上述处理单元或处理器可以是中央处理器，通用处理器、特定集成电路(application specificintegrated circuit，ASIC)、微处理器(digital signal processor，DSP)，现场可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。
本申请实施例还提供了一种包含计算机程序指令的计算机程序产品，当该计算机程序指令在计算机上运行时，使得计算机执行上述实施例中的任意一种方法。该计算机程序产品包括一个或多个计算机程序指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机程序指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机程序指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如，软盘、硬盘、磁带)，光介质(例如，DVD)、或者半导体介质(例如SSD)等。
应注意，本申请实施例提供的上述用于存储计算机程序指令器件，例如但不限于，上述存储器、计算机可读存储介质和通信芯片等，均具有非易失性(non-transitory)。
在实施所要求保护的本申请过程中，本领域技术人员通过查看附图、公开内容、以及所附权利要求书，可理解并实现公开实施例的其他变化。在权利要求中，“包括”(comprising)一词不排除其他组成部分或步骤，“一”或“一个”不排除多个的情况。单个处理器或其他单元可以实现权利要求中列举的若干项功能。相互不同的从属权利要求中记载了某些措施，但这并不表示这些措施不能组合起来产生良好的效果。

Claims (23)

  1. 一种存储系统中管理数据索引的方法，其特征在于，所述方法包括：
    获取第一待写数据，所述第一待写数据的逻辑地址范围是第一逻辑地址范围；
    基于所述第一逻辑地址范围中待生成数据索引的逻辑地址范围与所述存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引；其中，所述存储系统包含第一长度的数据索引操作单元和第二长度的数据索引操作单元；所述第一长度大于所述第二长度。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述第一逻辑地址范围中待生成数据索引的逻辑地址范围与所述存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引，具体包括：
    当所述待生成数据索引的逻辑地址范围与所述第一长度的数据索引操作单元对齐时，基于所述第一长度的数据索引操作单元，为第一数据生成第一数据索引；所述第一数据是所述第一待写数据中从所述待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是所述第一长度的逻辑地址范围的数据；其中，所述第一数据索引包括所述第一数据在所述存储系统中的位置。
  3. 根据权利要求2所述的方法，其特征在于，所述基于所述第一逻辑地址范围中待生成数据索引的逻辑地址范围与所述存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引，具体包括：
    当所述待生成数据索引的逻辑地址范围与所述第一长度的数据索引操作单元不对齐时，基于所述第二长度的数据索引操作单元，为第二数据生成第二数据索引；其中，所述第二数据是所述第一待写数据中从所述待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是所述第二长度的逻辑地址范围的数据；所述第二数据索引包括所述第二数据在所述存储系统中的位置。
  4. 根据权利要求2或3所述的方法，其特征在于，当所述待生成数据索引的逻辑地址范围的首个逻辑地址能被所述第一长度整除，且所述待生成数据索引的逻辑地址范围大于等于所述第一长度时，所述待生成数据索引的逻辑地址范围与所述第一长度的数据索引操作单元对齐。
  5. 根据权利要求2所述的方法，其特征在于，在所述基于所述第一长度的数据索引操作单元，为第一数据生成第一数据索引之后，所述方法还包括：
    当所述存储系统中已存在第一目标数据索引时，删除所述第一目标数据索引；其中，所述第一目标数据索引是基于所述第二长度的数据索引操作单元，为第三数据生成的数据索引；所述第一数据的逻辑地址范围包括所述第三数据的逻辑地址范围；所述第一目标数据索引包含所述第三数据在所述存储系统中的位置。
  6. 根据权利要求3所述的方法，其特征在于，在所述基于所述第二长度的数据索引操作单元，为第二数据生成第二数据索引之后，所述方法还包括：
    当所述存储系统中已存在第二目标数据索引时，将所述第二目标数据索引中的目标部分的状态标记为无效；其中，所述第二目标数据索引是基于所述第一长度的数据索引操作单元，为第四数据生成的数据索引；所述目标部分是为所述第四数据的逻辑地址范围中包含的第二逻辑地址范围的数据生成的数据索引；所述第二逻辑地址范围 与所述第二数据的逻辑地址范围相同。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    当所述第二目标数据索引的所有部分的状态均被标记为无效时，删除所述第二目标数据索引。
  8. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    当所述第二目标数据索引的所有部分中状态为无效的部分的个数大于或等于阈值时，将所述所有部分中状态为有效的部分生成新的数据索引，并删除所述第二目标数据索引。
  9. 根据权利要求3所述的方法，其特征在于，所述方法还包括：
    获取待查询逻辑地址范围；
    基于所述第二长度对所述待查询逻辑地址范围进行编码，得到第一编码结果；
    基于所述第一编码结果查询到所述第二数据索引；其中，所述第二数据索引包括所述第一编码结果；
    基于所述第二数据索引所指示的数据，确定所述待查询逻辑地址范围的数据。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    当所述存储系统中不存在所述第一编码结果时，基于所述第一长度对所述待查询逻辑地址范围进行编码，得到第二编码结果；
    基于所述第二编码结果查询到所述第一数据索引；其中，所述第一数据索引包括所述第二编码结果；
    基于所述第一数据索引所指示的数据，确定所述待查询逻辑地址范围的数据。
  11. 一种存储系统中管理数据索引的装置，其特征在于，所述装置包括：
    获取单元，用于获取第一待写数据，所述第一待写数据的逻辑地址范围是第一逻辑地址范围；
    生成单元，用于基于所述第一逻辑地址范围中待生成数据索引的逻辑地址范围与所述存储系统中不同长度的数据索引操作单元的对齐情况生成数据索引；其中，所述存储系统包含第一长度的数据索引操作单元和第二长度的数据索引操作单元；所述第一长度大于所述第二长度。
  12. 根据权利要求11所述的装置，其特征在于，
    所述生成单元具体用于：当所述待生成数据索引的逻辑地址范围与所述第一长度的数据索引操作单元对齐时，基于所述第一长度的数据索引操作单元，为第一数据生成第一数据索引；所述第一数据是所述第一待写数据中从所述待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是所述第一长度的逻辑地址范围的数据；其中，所述第一数据索引包括所述第一数据在所述存储系统中的位置。
  13. 根据权利要求12所述的装置，其特征在于，
    所述生成单元具体用于：当所述待生成数据索引的逻辑地址范围与所述第一长度的数据索引操作单元不对齐时，基于所述第二长度的数据索引操作单元，为第二数据生成第二数据索引；其中，所述第二数据是所述第一待写数据中从所述待生成数据索引的逻辑地址范围的首个逻辑地址开始且长度是所述第二长度的逻辑地址范围的数据；所述第二数据索引包括所述第二数据在所述存储系统中的位置。
  14. 根据权利要求12或13所述的装置，其特征在于，当所述待生成数据索引的逻辑地址范围的首个逻辑地址能被所述第一长度整除，且所述待生成数据索引的逻辑地址范围大于等于所述第一长度时，所述待生成数据索引的逻辑地址范围与所述第一长度的数据索引操作单元对齐。
  15. 根据权利要求12所述的装置，其特征在于，所述装置还包括：
    删除单元，用于在所述生成单元基于所述第一长度的数据索引操作单元，为第一数据生成第一数据索引之后，当所述存储系统中已存在第一目标数据索引时，删除所述第一目标数据索引；其中，所述第一目标数据索引是基于所述第二长度的数据索引操作单元，为第三数据生成的数据索引；所述第一数据的逻辑地址范围包括所述第三数据的逻辑地址范围；所述第一目标数据索引包含所述第三数据在所述存储系统中的位置。
  16. 根据权利要求13所述的装置，其特征在于，所述装置还包括：
    设置单元，用于在所述生成单元基于所述第二长度的数据索引操作单元，为第二数据生成第二数据索引之后，当所述存储系统中已存在第二目标数据索引时，将所述第二目标数据索引中的目标部分的状态标记为无效；其中，所述第二目标数据索引是基于所述第一长度的数据索引操作单元，为第四数据生成的数据索引；所述目标部分是为所述第四数据的逻辑地址范围中包含的第二逻辑地址范围的数据生成的数据索引；所述第二逻辑地址范围与所述第二数据的逻辑地址范围相同。
  17. 根据权利要求16所述的装置，其特征在于，所述装置还包括：
    删除单元，用于当所述第二目标数据索引的所有部分的状态均被标记为无效时，删除所述第二目标数据索引。
  18. 根据权利要求16所述的装置，其特征在于，所述装置还包括：
    删除单元，用于当所述第二目标数据索引的所有部分中状态为无效的部分的个数大于或等于阈值时，将所述所有部分中状态为有效的部分生成新的数据索引，并删除所述第二目标数据索引。
  19. 根据权利要求13所述的装置，其特征在于，
    所述获取单元还用于，获取待查询逻辑地址范围；
    所述装置还包括：
    编码单元，用于基于所述第二长度对所述待查询逻辑地址范围进行编码，得到第一编码结果；
    查询单元，用于基于所述第一编码结果查询到所述第二数据索引；其中，所述第二数据索引包括所述第一编码结果；
    确定单元，用于基于所述第二数据索引所指示的数据，确定所述待查询逻辑地址范围的数据。
  20. 根据权利要求19所述的装置，其特征在于，
    所述编码单元还用于，当所述存储系统中不存在所述第一编码结果时，基于所述第一长度对所述待查询逻辑地址范围进行编码，得到第二编码结果；
    所述查询单元还用于，基于所述第二编码结果查询到所述第一数据索引；其中，所述第一数据索引包括所述第二编码结果；
    所述确定单元还用于，基于所述第一数据索引所指示的数据，确定所述待查询逻辑地址范围的数据。
  21. 一种存储系统中管理数据索引的装置，其特征在于，所述装置包括：存储器和处理器，所述存储器用于存储计算机程序指令，所述处理器用于调用所述计算机程序指令，以执行权利要求1-10任一项所述的方法。
  22. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序指令，当所述计算机程序指令在计算机上运行时，使得所述计算机执行权利要求1-10任一项所述的方法。
  23. 一种计算机程序产品，其特征在于，当计算程序产品包含的计算机程序指令在计算机上运行时，使得所述计算机执行权利要求1-10任一项所述的方法。
PCT/CN2020/134025 2019-12-23 2020-12-04 存储系统中管理数据索引的方法和装置 WO2021129354A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20907053.1A EP4068111A4 (en) 2019-12-23 2020-12-04 DATA INDEX MANAGEMENT METHOD AND APPARATUS IN A STORAGE SYSTEM
US17/846,208 US20220327102A1 (en) 2019-12-23 2022-06-22 Data index management method and apparatus in storage system

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201911342473 2019-12-23
CN201911342473.9 2019-12-23
CN201911369404.7A CN111241090B (zh) 2019-12-23 2019-12-26 存储系统中管理数据索引的方法和装置
CN201911369404.7 2019-12-26

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/846,208 Continuation US20220327102A1 (en) 2019-12-23 2022-06-22 Data index management method and apparatus in storage system

Publications (1)

Publication Number Publication Date
WO2021129354A1 true WO2021129354A1 (zh) 2021-07-01

Family

ID=70863893

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/134025 WO2021129354A1 (zh) 2019-12-23 2020-12-04 存储系统中管理数据索引的方法和装置

Country Status (4)

Country Link
US (1) US20220327102A1 (zh)
EP (1) EP4068111A4 (zh)
CN (1) CN111241090B (zh)
WO (1) WO2021129354A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241090B (zh) * 2019-12-23 2023-11-10 华为技术有限公司 存储系统中管理数据索引的方法和装置
CN112131140B (zh) * 2020-09-24 2023-07-14 北京计算机技术及应用研究所 基于ssd的支持高效存储空间管理的键值分离存储方法
CN113420035A (zh) * 2021-02-05 2021-09-21 阿里巴巴集团控股有限公司 数据处理方法、系统、装置、电子设备及计算机存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103440208A (zh) * 2013-08-12 2013-12-11 华为技术有限公司 一种数据存储的方法及装置
CN103942114A (zh) * 2013-01-22 2014-07-23 Lsi公司 Nvm地址、跨度及长度映射/转换的存储地址空间
US20170090754A1 (en) * 2015-09-30 2017-03-30 Western Digital Technologies, Inc. Mapping system selection for data storage device
CN110471861A (zh) * 2019-07-10 2019-11-19 华为技术有限公司 一种闪存设备中的数据存储方法及闪存设备
CN111241090A (zh) * 2019-12-23 2020-06-05 华为技术有限公司 存储系统中管理数据索引的方法和装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3607107B2 (ja) * 1998-03-13 2005-01-05 株式会社東芝 データ管理装置
CN103034677B (zh) * 2012-11-12 2015-06-24 武大吉奥信息技术有限公司 多时相瓦片数据集组织和游程编码索引方法
US10114674B2 (en) * 2014-03-06 2018-10-30 International Business Machines Corporation Sorting database collections for parallel processing
US20150347477A1 (en) * 2014-05-30 2015-12-03 John Esmet Streaming File System
US10437785B2 (en) * 2016-03-29 2019-10-08 Samsung Electronics Co., Ltd. Method and apparatus for maximized dedupable memory
JP6725120B2 (ja) * 2016-06-24 2020-07-15 ホアウェイ・テクノロジーズ・カンパニー・リミテッド データ処理方法、記憶装置、ソリッドステートディスク及び記憶システム
CN107329704B (zh) * 2017-06-30 2020-03-24 杭州宏杉科技股份有限公司 一种缓存镜像方法及控制器
WO2020041928A1 (zh) * 2018-08-27 2020-03-05 深圳市锐明技术股份有限公司 数据存储方法、系统及终端设备
US10901912B2 (en) * 2019-01-28 2021-01-26 Western Digital Technologies, Inc. Flash translation layer table for unaligned host writes

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103942114A (zh) * 2013-01-22 2014-07-23 Lsi公司 Nvm地址、跨度及长度映射/转换的存储地址空间
CN103440208A (zh) * 2013-08-12 2013-12-11 华为技术有限公司 一种数据存储的方法及装置
US20170090754A1 (en) * 2015-09-30 2017-03-30 Western Digital Technologies, Inc. Mapping system selection for data storage device
CN110471861A (zh) * 2019-07-10 2019-11-19 华为技术有限公司 一种闪存设备中的数据存储方法及闪存设备
CN111241090A (zh) * 2019-12-23 2020-06-05 华为技术有限公司 存储系统中管理数据索引的方法和装置

Also Published As

Publication number Publication date
EP4068111A1 (en) 2022-10-05
CN111241090A (zh) 2020-06-05
CN111241090B (zh) 2023-11-10
US20220327102A1 (en) 2022-10-13
EP4068111A4 (en) 2022-12-28

Similar Documents

Publication Publication Date Title
WO2021129354A1 (zh) 存储系统中管理数据索引的方法和装置
JP5996088B2 (ja) 暗号ハッシュ・データベース
US10331641B2 (en) Hash database configuration method and apparatus
WO2019062574A1 (zh) 一种元数据查询方法及装置
US20180349285A1 (en) Managing Multiple Namespaces in a Non-Volatile Memory (NVM)
KR20190134115A (ko) 효율적인 인덱싱을 제공하기 위한 방법, 장치 및 컴퓨터-판독가능 매체에 포함된 컴퓨터 프로그램
KR20140042522A (ko) 디렉토리 엔트리 조회 장치, 그 방법 및 디렉토리 엔트리 조회 프로그램이 기록된 기록 매체
CN110168532B (zh) 数据更新方法和存储装置
WO2019127018A1 (zh) 存储系统访问方法及装置
US11347711B2 (en) Sparse infrastructure for tracking ad-hoc operation timestamps
WO2022241813A1 (zh) 一种基于图压缩的图数据库构建方法、装置及相关组件
WO2021027340A1 (zh) 一种键值kv的存储方法、装置及存储设备
CN109407985B (zh) 一种数据管理的方法以及相关装置
WO2023273659A1 (zh) 缓存文件的管理方法和管理装置
CN112615917A (zh) 存储系统中存储设备的管理方法及存储系统
CN109947667B (zh) 数据访问预测方法和装置
CN110199270B (zh) 存储系统中存储设备的管理方法及装置
KR20230026946A (ko) 해싱을 사용하는 키 밸류 스토리지 장치 및 그것의 동작 방법
KR101806394B1 (ko) 모바일 dbms환경에서 트랜잭션에 특화된 색인 캐시의 구조를 갖는 데이터 처리 방법
US20220342888A1 (en) Object tagging
WO2024119797A1 (zh) 一种数据处理方法、系统、设备以及存储介质
CN116627359B (zh) 内存管理方法、装置、可读存储介质及电子设备
WO2024093090A1 (zh) 一种元数据管理方法、装置、计算机设备及可读存储介质
US11163693B2 (en) Hash-based one-level mapping for storage clusters
EP4307129A1 (en) Method for writing data into solid-state hard disk

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20907053

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020907053

Country of ref document: EP

Effective date: 20220629

NENP Non-entry into the national phase

Ref country code: DE