WO2014101420A1

WO2014101420A1 - System for constructing metadata and method therefor

Info

Publication number: WO2014101420A1
Application number: PCT/CN2013/080710
Authority: WO
Inventors: 代海军
Original assignee: 华为技术有限公司
Priority date: 2012-12-28
Filing date: 2013-08-02
Publication date: 2014-07-03
Also published as: CN103019953B; CN103019953A

Abstract

Disclosed are a system for constructing metadata and a method therefor. The method comprises the following steps: receiving a request for reading data or writing data (101); querying an index area of metadata according to a logical address in the request, and judging whether the logical address exists in the index area of the metadata (102); if so, querying a mapping relationship between the logical address and a physical address in a mapping relationship area of logical addresses and physical addresses, acquiring the physical address corresponding to the logical address, and reading data or writing data according to the physical address (103); otherwise, when the request is a data reading request, all the read data being zero (105); and when the request is a data writing request, allocating a physical address for the logical address, and writing the data according to the physical address (106). The method and system can meet the requirements for querying metadata under various conditions, and support setting different index types according to different application scenarios to enable the performance to be better, thereby increasing the flexibility of the metadata.

Description

Metadata construction system and method thereof

Technical field

The present invention relates to the field of storage technologies, and in particular, to a metadata construction system and method thereof.

Background technique

With the continuous development of information science and technology, the widespread use of the Internet, and the continuous popularization of cloud computing, the amount of stored data is growing rapidly, and higher requirements are placed on the rational allocation and utilization of storage system resources. In the prior art, Thin Provisioning (TP) aggregates all physical storage space into a thin pool for unified management, and reduces the pool by using a thin LUN (Logical Unit Number) created on the thin pool as needed. Obtain physical storage space, thin LUNs, that is, thin volumes, also known as Thin LUNs. The use of thin provisioning technology simplifies configuration management of storage resources and conserves physical storage resources. In the thin provisioning technology, metadata is mainly used to store the Logic Address and Physical Address Mapping (LPM) of the reduced LUN.

In the prior art, the automatic thin provisioning technology uses the B-tree to store metadata, which can effectively improve the access efficiency of metadata and has good scalability. However, in some scenarios (such as when the amount of metadata is small), it is not appropriate to use B-tree to store metadata, and the key value of the metadata is fixed to the logical address of the thin LUN, which cannot satisfy the metadata based on other information. The need to find, the existing metadata storage method lacks flexibility. Summary of the invention

In view of this, embodiments of the present invention provide a metadata construction system and method thereof to solve the problem of lack of flexibility in metadata storage.

A first aspect provides a method for constructing metadata, comprising the steps of: receiving a request to read data or writing data; querying an index area of the metadata according to a logical address in the request, and determining an index of the logical address in the metadata Whether there is a zone, where the metadata includes an index area, a logical address And the physical address mapping relationship area, the index area records the storage address of the logical address and the physical address mapping relationship in the logical address and the physical address mapping relationship area; if yes, according to the logical address and the physical address corresponding to the logical address in the index area The mapping relationship between the logical address and the physical address mapping relationship area, the logical address and the physical address mapping relationship area are searched for the mapping relationship between the logical address and the physical address, and the physical address corresponding to the logical address is obtained, and the data is obtained. The physical address reads data or writes data; if it does not exist, when the request is a read data request, the read data is all 0; when the request is a write data request, the physical address is assigned to the logical address, and The mapping relationship between the logical address and the assigned physical address is stored in the logical address and the physical address mapping relationship area, and the corresponding index record is established in the index area for the mapping relationship between the newly added logical address and the allocated physical address, according to the physical address. data input.

In combination with the first aspect, in a first possible implementation, the method further includes: setting an index type of the index area when creating the thin volume Thin LUN.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner, a corresponding index record is established in the index area for the mapping relationship between the newly added logical address and the allocated physical address, including: The mapping relationship between the newly added logical address and the assigned physical address is established according to the set index type.

With reference to the first aspect or the first or second possible implementation manner of the first aspect, in a third possible implementation manner, different key values KEY are established in the index area for the mapping relationship between the same logical address and the physical address. At least two indexes.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, establishing, in the index area, a corresponding index record for the mapping relationship between the newly added logical address and the allocated physical address includes: A mapping relationship between the newly added logical address and the assigned physical address is sequentially established in at least two indexes.

A second aspect provides a metadata construction system, comprising: 10 units, configured to receive a request for reading data or writing data; and a processing unit, configured to query metadata according to a logical address in a request acquired by 10 units The index area determines whether the logical address exists in the index area of the metadata, wherein the metadata includes an index area, a logical address, and a physical address mapping relationship area, and the index area records a mapping relationship between the logical address and the physical address at the logical address and The physical address maps the storage address in the relational area; if the logical address exists in the index area of the metadata, the processing unit is further configured to map the logical address and the physical address according to the logical address and the physical address in the index area in the logical address and the physical Address Mapping the storage address in the relational area, querying the mapping relationship between the logical address and the physical address in the logical address and the physical address mapping relationship area, acquiring the physical address corresponding to the logical address, and reading data or writing data according to the physical address; The logical address does not exist in the index area of the metadata. When the request is a read data request, the 10 unit is also used to read the data as all 0s; when the request is a write data request, the processing unit also uses The physical address is assigned to the logical address, and the mapping relationship between the logical address and the assigned physical address is stored in the logical address and the physical address mapping relationship area, and the newly added logical address and the assigned physical address are mapped in the index area. The relationship establishes the corresponding index record; 10 units are also used to write data based on the physical address.

In conjunction with the second aspect, in a first possible implementation, the processing unit is further configured to set an index type of the index area when creating the thin volume.

With reference to the first possible implementation of the second aspect, in a second possible implementation manner, a corresponding index record is established in the index area for the mapping relationship between the newly added logical address and the allocated physical address, specifically:

The mapping relationship between the newly added logical address and the assigned physical address in the index area is based on the set index type to establish a corresponding index record.

With reference to the second aspect or the first or second possible implementation manner of the second aspect, in a third possible implementation, the mapping between the same logical address and the physical address in the index area establishes at least a different key value KEY Two indexes.

With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, a corresponding index record is established in the index area for the mapping relationship between the newly added logical address and the allocated physical address, specifically:

The mapping relationship between the newly added logical address and the assigned physical address in the index area sequentially establishes corresponding index records in at least two indexes.

Through the foregoing solution, the beneficial effects of the present invention are: The present invention records the logical relationship between the logical address and the physical address in the index area by dividing the metadata into an index area, a logical address, and a physical address mapping relationship area. The physical address maps the storage address in the relational area, so that the query metadata can be satisfied under various search conditions, and the performance of the index is improved according to the different application scenarios, and the flexibility of the metadata is increased.

DRAWINGS In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in view of the drawings. among them:

1 is a flow chart showing a method of constructing metadata according to a first embodiment of the present invention;

2 is a schematic structural diagram of metadata in FIG. 1;

3 is a flow chart showing setting an index type of an index area in a method of constructing metadata according to a second embodiment of the present invention;

4 is a schematic structural diagram of an index area in a method for constructing metadata according to a third embodiment of the present invention; FIG. 5 is a schematic structural diagram of a metadata construction system according to a first embodiment of the present invention;

Fig. 6 is a block diagram showing the structure of a metadata construction system in accordance with a fourth embodiment of the present invention. detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1, FIG. 1 is a flowchart of a method of constructing metadata according to a first embodiment of the present invention. As shown in FIG. 1, the construction method disclosed in this embodiment includes the following steps:

Step 101: Receive a request to read data or write data.

Step 102: Query the index area of the metadata according to the logical address in the request, and determine whether the logical address exists in the index area of the metadata.

Preferably, the metadata includes a logical address and a physical address mapping area (MP) and an index part (IP), as shown in FIG. 2. The mapping between the logical address and the physical address includes a mapping relationship between multiple logical addresses and physical addresses. Preferably, the LPM can be stored in the first segment of each physical space of the physical disk. The index area IP records logical addresses and physical addresses. Address mapping relationship The storage address of the LPM in the logical address and physical address mapping relationship area MP.

Step 103: If yes, according to the mapping relationship between the logical address and the physical address corresponding to the logical address in the index area, the storage address in the logical address and the physical address mapping relationship area, in the logical address Query the mapping relationship between the logical address and the physical address in the physical address mapping relationship area, obtain the physical address corresponding to the logical address, and read data or write data according to the physical address.

Preferably, if the logical address exists in the index area IP of the metadata, the mapping relationship between the logical address and the physical address corresponding to the logical address is searched in the index area IP according to the logical address. The logical address and the physical address mapping relationship area MP The storage address in the logical address and the physical address mapping relationship area MP reads the mapping relationship LPM of the corresponding logical address and the physical address according to the storage address, to obtain the physical address corresponding to the logical address, and reads according to the physical address Data or write data.

Step 104: If it does not exist, determine that the request is a read data request or a write data request. Step 105: When the request is a data read request, the read data is all 0;

Step 106: When the request is a write data request, assign a physical address to the logical address, and store the mapping relationship between the logical address and the allocated physical address in the logical address and physical address mapping relationship area of the metadata, in the metadata In the index area, a corresponding index record is established for the mapping relationship between the newly added logical address and the assigned physical address, and data is written according to the physical address.

In this embodiment, the present embodiment divides the metadata into an index area IP, a logical address, and a physical address mapping relationship area MP, and records a mapping relationship between the logical address and the physical address in the index area IP at the logical address and The physical address maps the storage address in the relational area MP, so that the query metadata can be satisfied under various search conditions, and the flexibility of the metadata is increased.

The present invention also provides a method of constructing metadata of the second embodiment, which is described in detail based on the method disclosed in the first embodiment. The method disclosed in this embodiment further includes: when creating a thin volume, setting an index type of the index area. Correspondingly, in step 106, a corresponding index record is established in the index area of the metadata for the mapping relationship between the newly added logical address and the allocated physical address, including: a newly added logical address and a location in the index area of the metadata The mapping relationship of the assigned physical addresses establishes a corresponding index record according to the index type.

The method disclosed in this embodiment further includes a method of setting an index type of an index area. As shown in Figure 3, when creating a thin volume, setting the index type of the index area includes the following steps:

Step 301: Set the LUN-related parameters; for example, the size of the LUN, the write-back policy, and other parameters.

Step 302: prompt the user to select a type of the index;

Step 303: Create a thin LUN and set the index type according to the user selection. In step 302, the user selects the type of the index according to different application scenarios and the size of the thinned volume. For example, when the thinned volume is small, and the index of the metadata can be stored in the memory, the structure of the balanced binary tree can be selected to store the index; The thin volume is large, and when the write data is small, the hash table structure can be selected to store the index; when the thin volume is large, and the write data is large, the linear table structure can be selected to store the index; Larger, when the amount of write data is uncertain, you can choose the structure of the B-tree and its variants to store the index.

In this embodiment, different index types are set according to the unused application scenarios and the size of the thinned volume, so that the performance of the thinned volume in various application scenarios is optimized.

The present invention also provides a method of constructing metadata of the third embodiment, which is described in detail based on the method disclosed in the second embodiment. The method disclosed in this embodiment further includes: establishing at least two indexes of different key values KEY for the mapping relationship between the same logical address and the physical address in the index area. That is, for the mapping relationship between the same logical address and the physical address LPM, the user can select multiple indexes to improve performance according to the needs of the service, and preferably, establish indexes of different key values KEY. As shown in FIG. 4, the index area 400 includes an index 401 of a logical address and an index 402 of a physical address. Among them, the use of different indexes for different query requirements of metadata can significantly improve the performance of query metadata. For example, when the logical address is queried according to the physical address, if there is only the index 401 of the logical address, all the metadata must be queried to obtain the logical address; if the index area includes the index 402 of the physical address, the index 402 of the physical address can be directly Query metadata to obtain logical addresses, which in turn improves the performance of query metadata. Further, the types of the multiple indexes may be different or the same, for example, one index is a B-tree with a logical address as a key value, and the other index is a B-tree with a physical address as a key value.

Then, in step 106, a corresponding index record is established in the index area of the metadata for the mapping relationship between the newly added logical address and the allocated physical address, including: a newly added logical address in the index area of the metadata The mapping relationship with the assigned physical address sequentially establishes corresponding index records in at least two indexes, that is, the corresponding index records are respectively established in the index 401 of the logical address and the index 402 of the physical address.

In this embodiment, at least two indexes of different key values KEY are established by mapping the same logical address and the physical address in the index area, which can improve the performance of the query metadata, and increase the application scenarios and metadata of the thin volume. Flexibility.

The present invention also provides a metadata construction system of the first embodiment, which is disclosed in the first embodiment The method is described in detail based on the method. As shown in FIG. 5, the construction system 50 disclosed in this embodiment includes: 10 (Input/Output) unit 501 and processing unit 502.

In this embodiment, 10 unit 501 is configured to receive a request to read data or write data. The processing unit 502 is configured to obtain a request from the 10 unit 501, query the index area IP of the metadata according to the logical address in the request, and determine whether the logical address exists in the index area IP of the metadata.

If the logical address exists in the index area of the metadata, the processing unit 502 is configured to search, in the index area IP, the mapping relationship between the logical address and the physical address corresponding to the logical address according to the logical address, the logical address and the physical Addressing the storage address in the relational area MP, and further searching the mapping relationship LPM of the corresponding logical address and the physical address in the logical address and the physical address mapping relationship area MP according to the storage address, to obtain the physical address corresponding to the logical address, and Read data or write data based on physical address.

If the logical address does not exist in the index area of the metadata, when the request is a read data request, the 10 unit 501 is also used to read data of all 0s; when the request is a write data request, the processing unit 502. The physical address is further allocated to the logical address, and the mapping relationship between the logical address and the allocated physical address is stored in the logical address and the physical address mapping relationship area of the metadata, and is newly added in the index area of the metadata. The mapping relationship between the logical address and the assigned physical address establishes a corresponding index record, and the 10 unit 501 is also used to write data according to the physical address.

In this embodiment, the metadata includes a logical address and a physical address mapping relationship area and an index area, as shown in FIG. 2. The mapping between the logical address and the physical address includes a mapping relationship between multiple logical addresses and physical addresses. The LPM is stored in the first segment of each physical space of the physical disk. The IP address of the index area records the mapping between logical addresses and physical addresses. The storage address of the relational LPM in the logical address and physical address mapping relationship area MP.

It should be noted that in other embodiments of the present invention, those skilled in the art can fully set 10 units 501 as read units and write units.

In this embodiment, the processing unit 502 divides the metadata into an index area IP, a logical address, and a physical address mapping relationship area MP, and records a mapping relationship between the logical address and the physical address in the index area IP. The logical address and the physical address map the storage address in the relationship area MP, so that the query metadata can be satisfied under various search conditions, and the flexibility of the metadata is increased.

The present invention also provides a metadata construction system of the second embodiment, which is described in detail based on the system 50 disclosed in the first embodiment. In the system disclosed in this embodiment, the processing unit 502, Also used to set the index type of the index area when creating a thin volume. Preferably, the user can select the type of the index according to different application scenarios and the size of the reduced LU. For example, when the reduced LUN is small, the index of the metadata can be stored in the memory, and the structure of the balanced binary tree can be selected to store the index; The LUN is large. When the data is written less, the Hash table structure can be selected to store the index. When the thin LUN is large, when the data is written, the linear table structure can be selected to store the index. In the thin LUN, the data is written. When the amount is uncertain, you can choose the structure of the B-tree and its variants to store the index.

In this embodiment, if the request is to write data, the processing unit 502 establishes a corresponding index record in the index area for the mapping relationship between the newly added logical address and the assigned physical address, specifically: the processing unit 502 is in the metadata. The index area is a mapping relationship between the newly added logical address and the assigned physical address, and the corresponding index record is established according to the index type.

The present invention also provides a metadata construction system of the third embodiment, which is different from the system disclosed in the second embodiment in that: in the index area, different key values KEY are established for the mapping relationship between the same logical address and the physical address. At least two indexes. The processing unit 502 can significantly improve the performance of the query metadata by establishing multiple indexes and using different indexes for different query requirements of the metadata. In addition, the processing unit 502 establishes a corresponding index record for the mapping relationship between the newly added logical address and the allocated physical address in the index area, specifically: the processing unit 502 is a mapping of the newly added logical address and the allocated physical address in the index area. The relationship in turn establishes a corresponding index record in at least two indexes.

Referring to FIG. 6, FIG. 6 is a system for constructing metadata according to a fourth embodiment of the present invention. As shown in FIG. 6, the system 60 disclosed in this embodiment includes a 10 interface 603 and a processor 601, a memory 604, and a communication bus 602. Wherein, the 10 interface 603 is for receiving a request for reading data or writing data, the memory 604 stores execution instructions, when the construction system 60 is running, the processor 601 is in communication with the memory 604, and the processor 601 executes execution instructions to construct the system. 60 is a method for constructing metadata of the first to third embodiments of the present invention. The implementation principle and technical effects are similar to those described in the previous embodiments, and are not described herein again.

In summary, the present invention records the storage address in the logical address and the physical address mapping relationship area by recording the mapping relationship between the logical address and the physical address in the index area, thereby satisfying the query metadata under various search conditions, and supporting the basis Setting different indexes without using the application scenario makes the performance better, and can include multiple indexes in the index area, so that the query metadata can be satisfied under various conditions, and And increased the flexibility of metadata.

The above is only the embodiment of the present invention, and is not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the specification and the drawings of the present invention may be directly or indirectly applied to other related technologies. The scope of the invention is included in the scope of patent protection of the present invention.

Claims

Rights request

1. A method for constructing metadata, characterized in that the method includes the following steps: receiving a request to read data or write data;

Query the index area of metadata according to the logical address in the request, and determine whether the logical address exists in the index area of the metadata, where the metadata includes the index area, logical address and physical address mapping The relationship area, the index area records the storage address of the mapping relationship between the logical address and the physical address in the mapping relationship area between the logical address and the physical address; the mapping relationship of the address in the mapping relationship between the logical address and the physical address. To store an address, query the mapping relationship between the logical address and the physical address in the mapping relationship area between the logical address and the physical address, obtain the physical address corresponding to the logical address, and read or write data according to the physical address. data;

If it does not exist, then when the request is a read data request, the read data is all 0; when the request is a write data request, a physical address is assigned to the logical address, and the logical address is The mapping relationship between the address and the allocated physical address is stored in the mapping relationship area between the logical address and the physical address, and a mapping relationship between the newly added logical address and the allocated physical address is established in the index area. The corresponding index record writes data according to the physical address.

2. The method according to claim 1, characterized in that, the method further includes:

When creating a thin volume Thin LUN, set the index type of the index area.

3. The method according to claim 2, characterized in that: establishing a corresponding index record in the index area for the mapping relationship between the newly added logical address and the allocated physical address includes:

A corresponding index record is established in the index area for the mapping relationship between the newly added logical address and the allocated physical address according to the set index type.

4. The method according to any one of claims 1 to 3, characterized in that,

In the index area, at least two indexes of different key values KEY are established for the mapping relationship between the same logical address and physical address.

5. The method according to claim 4, wherein establishing a corresponding index record in the index area for the mapping relationship between the newly added logical address and the allocated physical address includes: In the index area, corresponding index records are respectively established in the at least two indexes for the mapping relationship between the newly added logical address and the allocated physical address.

6. A metadata construction system, characterized by including:

10 units, used to receive requests to read data or write data;

The processing unit is configured to query the index area of metadata according to the logical address in the request obtained by the 10 unit, and determine whether the logical address exists in the index area of the metadata, wherein the metadata includes The index area, the logical address and the physical address mapping relationship area, the index area records the storage address of the mapping relationship between the logical address and the physical address in the logical address and physical address mapping relationship area;

If the logical address exists in the index area of the metadata, the processing unit also uses the storage address in the mapping relationship area between the logical address and the physical address to map the logical address and the physical address. Query the mapping relationship between the logical address and the physical address in the relationship area, obtain the physical address corresponding to the logical address, and read or write data according to the physical address; if the logical address is in the metadata does not exist in the index area, then

When the request is a read data request,

The 10 units are also used to read data that is all 0;

When the request is a write data request,

The processing unit is further configured to allocate a physical address to the logical address, store the mapping relationship between the logical address and the allocated physical address in the logical address and physical address mapping relationship area, and store the mapping relationship between the logical address and the physical address in the mapping relationship area. Establish a corresponding index record in the index area for the mapping relationship between the newly added logical address and the allocated physical address;

The 10 unit is also used to write data according to the physical address.

7. The system according to claim 6, characterized in that,

The processing unit is also used to set the index type of the index area when creating a thin volume.

8. The system according to claim 7, characterized in that, in the index area, a corresponding index record is established for the mapping relationship between the newly added logical address and the allocated physical address, specifically:

The mapping relationship between the newly added logical address and the allocated physical address in the index area is to establish a corresponding index record according to the set index type.

9. The system according to any one of claims 6 to 8, characterized in that,

10. The system according to claim 9, characterized in that, in the index area, a corresponding index record is established for the mapping relationship between the newly added logical address and the allocated physical address, specifically:

In the index area, corresponding index records are respectively established in the at least two indexes for the mapping relationship between the newly added logical address and the allocated physical address.