CN107155307A

CN107155307A - Access method, system and the main frame of file

Info

Publication number: CN107155307A
Application number: CN201480037250.0A
Authority: CN
Inventors: 朱冠宇; 徐君; 赵俊峰
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2014-10-31
Filing date: 2014-10-31
Publication date: 2017-09-12
Anticipated expiration: 2034-10-31
Also published as: WO2016065612A1; CN107155307B

Abstract

Disclose a kind of method, system and main frame for accessing file.This method includes：Main frame obtains the information of the first Virtual Space of file destination, and first Virtual Space is used to manage the file destination in distributed memory system, and the information of first Virtual Space is used for the LVA local virtual address space (S210) for pointing to the first memory node；It is file destination distribution the second Virtual Space (S220) in the LVA local virtual address space of the main frame according to the information of first Virtual Space；Second Virtual Space is mapped to physical vlan space (S230)；Set up the corresponding relation (S240) of first Virtual Space and the physical vlan space；When it is determined that the second access request accesses the physical vlan space, second access request is converted to the first access request (S250) for accessing first Virtual Space；First access request is sent to the network equipment, so that first access request is routed to first memory node (S260) by the network equipment.Method, system and main frame can improve the efficiency of file access.

Description

Method, system and host for accessing file

Technical Field

The invention relates to the technical field of storage, in particular to a method, a system and a host for accessing files.

Background

A new type of non-volatile Memory (NVM) medium, such as Phase Change Memory (PCM), variable resistance Random Access Memory (Resistive RAM, ReRAM), Magnetic Random Access Memory (MRAM), etc., has read/write characteristics similar to Dynamic Random Access Memory (DRAM), and can be hung on a Memory bus for Access. Currently, storage systems based on non-volatile storage media are the hot spot of research in the industry.

The existing method for remotely accessing the File System includes using Network File System (NFS) or Common Internet File System (CIFS) protocol. These conventional file access methods are based on Remote Procedure Call (RPC) and are based on Transmission Control Protocol/Internet Protocol (TCP/IP). For the NVM-based storage system, if the existing access method for remotely accessing the file system is still used, the access request still needs to be processed through the kernel TCP/IP protocol stack of the host, and the access efficiency is low.

Disclosure of Invention

The embodiment of the invention provides a method, a system and a host for accessing files, which can improve the efficiency of file access.

In a first aspect, a method for accessing a file is provided, where the method is applied in a distributed storage system, where the distributed storage system includes a management node, a plurality of storage nodes, and a network device, and the method includes:

the method comprises the steps that a host acquires information of a first virtual space of a target file, wherein the first virtual space is used for managing the target file in the distributed storage system, the information of the first virtual space is used for pointing to a local virtual address space of a first storage node, and the first storage node is a storage node which is used for storing the target file and is in a plurality of storage nodes;

the host allocates a second virtual space for the target file in the local virtual address space of the host according to the information of the first virtual space;

the host maps the second virtual space to a virtual physical space, wherein the virtual physical space is a storage space which is locally virtualized in the host and used for storing the target file;

the host machine establishes a corresponding relation between the first virtual space and the virtual physical space;

when the host determines that a second access request accesses the virtual physical space, converting the second access request into a first access request for accessing the first virtual space according to the corresponding relation between the first virtual space and the virtual physical space, wherein the first access request comprises the equipment information of the first storage node;

the host sends the first access request to the network device, so that the network device routes the first access request to the first storage node according to the device information of the first storage node.

With reference to the first aspect, in a first possible implementation manner, the obtaining, by the host, information of a first virtual space of a target file includes:

the host computer obtains the metadata of the target file and obtains the information of the first virtual space of the target file from the metadata of the target file.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner, the information of the first virtual space includes device information of the first storage node and a start address and a size of a local virtual address space used for storing the target file in the first storage node.

With reference to the first aspect or the first possible implementation manner of the first aspect, in a third possible implementation manner, the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and information of the first virtual space includes a start address and a size of the first virtual space.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the number of bits of the address in the first virtual space is 2N bits, N is the number of bits of the address in the local virtual address space of the first storage node, the upper N bits of the address in the first virtual space include the device information of the first storage node, and the lower N bits of the address in the first virtual space are the address in the local virtual address space of the first storage node, where the address is used to store the target file.

With reference to any one possible implementation manner of the second to the fourth possible implementation manners of the first aspect, in a fifth possible implementation manner, the allocating, by the host, a second virtual space to the target file in the local virtual address space of the host according to the information of the first virtual space includes:

the host allocates the second virtual space with the same size as the first virtual space for the target file in the local virtual address space of the host.

In a second aspect, a system is provided, comprising: a host and a distributed storage system;

the distributed storage system comprises a management node, a plurality of storage nodes and network equipment;

the host is used for:

acquiring information of a first virtual space of a target file, wherein the first virtual space is used for managing the target file in the distributed storage system, the information of the first virtual space is used for pointing to a local virtual address space of a first storage node, and the first storage node is a storage node used for storing the target file in the plurality of storage nodes;

according to the information of the first virtual space, distributing a second virtual space for the target file in a local virtual address space of the host;

mapping the second virtual space to a virtual physical space, wherein the virtual physical space is a storage space which is locally virtualized in the host and used for storing the target file;

establishing a corresponding relation between the first virtual space and the virtual physical space;

when a second access request is determined to access the virtual physical space, converting the second access request into a first access request for accessing the first virtual space according to the corresponding relation between the first virtual space and the virtual physical space, wherein the first access request comprises equipment information of the first storage node;

and sending the first access request to the network equipment so that the network equipment routes the first access request to the first storage node according to the equipment information of the first storage node.

With reference to the second aspect, in a first possible implementation manner, the host is specifically configured to obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.

With reference to the second aspect or the first possible implementation manner of the second aspect, in a second possible implementation manner, the information of the first virtual space includes device information of the first storage node and a start address and a size of a local virtual address space used for storing the target file in the first storage node.

With reference to the second aspect or the first possible implementation manner of the second aspect, in a third possible implementation manner, the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and information of the first virtual space includes a start address and a size of the first virtual space.

With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, the number of bits of the address in the first virtual space is 2N bits, N is the number of bits of the address in the local virtual address space of the first storage node, the upper N bits of the address in the first virtual space include the device information of the first storage node, and the lower N bits of the address in the first virtual space are the address in the local virtual address space of the first storage node, where the address is used to store the target file.

With reference to any one of the second to the fourth possible implementation manners of the second aspect, in a fifth possible implementation manner, the host is specifically configured to allocate, in a local virtual address space of the host, the second virtual space, which is the same as the first virtual space, to the target file.

In a third aspect, a host is provided, including:

a processing module, configured to obtain information of a first virtual space of a target file, where the first virtual space is used to manage the target file in a distributed storage system, the information of the first virtual space is used to point to a local virtual address space of a first storage node, the first storage node represents a storage node of the distributed storage system, the storage node is used to store the target file, according to the information of the first virtual space, a second virtual space is allocated to the target file in the local virtual address space of the host, the second virtual space is mapped to a virtual physical space, the virtual physical space is a storage space locally virtualized at the host, the storage space is used to store the target file, a corresponding relationship between the first virtual space and the virtual physical space is established, and when a second access request is determined to access the virtual physical space, the second access request is converted into an access request according to the corresponding relationship between the first virtual space and the virtual physical space Asking for a first access request of the first virtual space, wherein the first access request comprises equipment information of the first storage node;

a sending module, configured to send the first access request to a network device in the distributed storage system, so that the network device routes the first access request to the first storage node according to the device information of the first storage node.

With reference to the third aspect, in a first possible implementation manner, the processing module is specifically configured to obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.

With reference to the third aspect or the first possible implementation manner of the third aspect, in a second possible implementation manner, the information of the first virtual space includes device information of the first storage node and a start address and a size of a local virtual address space used for storing the target file in the first storage node.

With reference to the third aspect or the first possible implementation manner of the third aspect, in a third possible implementation manner, the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and information of the first virtual space includes a start address and a size of the first virtual space.

With reference to the third possible implementation manner of the third aspect, in a fourth possible implementation manner, the number of bits of the address in the first virtual space is 2N bits, N is the number of bits of the address in the local virtual address space of the first storage node, the upper N bits of the address in the first virtual space include device information of the first storage node, and the lower N bits of the address in the first virtual space are the address in the local virtual address space of the first storage node, where the address is used to store the target file.

With reference to any one of the second to the fourth possible implementation manners of the third aspect, in a fifth possible implementation manner, the processing module is specifically configured to allocate, in the local virtual address space of the host, the second virtual space, which is the same as the first virtual space, to the target file.

In a fourth aspect, there is provided a computer program product comprising a computer readable storage medium storing program code comprising instructions for performing the method of the first aspect.

Based on the above technical solution, in the method, the system, and the host for accessing a file in the distributed storage system according to the embodiments of the present invention, when accessing a file in the distributed storage system, the second virtual space is allocated to the file in the local virtual address space of the host and mapped to the local virtual physical space of the host, and when accessing the virtual physical space, the first access request for accessing the first virtual space of the file in the distributed storage system is sent to the network, so that the physical storage space where the file in the distributed storage system is located can be directly accessed without passing through the kernel of the host, thereby reducing latency and improving file access efficiency.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It should be apparent that the drawings in the following description are only drawings of some embodiments of the invention.

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present invention.

FIG. 2 is a schematic flow chart diagram of a method of accessing a file of an embodiment of the present invention.

Fig. 3 is a schematic diagram of the spatial correspondence of files according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a spatial correspondence relationship of files according to another embodiment of the present invention.

FIG. 5 is a schematic block diagram of a system of an embodiment of the present invention.

FIG. 6 is a schematic block diagram of a host of an embodiment of the present invention.

Fig. 7 is a schematic structural diagram of a host according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention.

It should be understood that the technical solution of the embodiments of the present invention can be applied to various storage systems, in particular to a distributed storage system based on a non-volatile storage medium.

It should also be understood that, in the embodiments of the present invention, the term "virtual address space" may also be expressed as "virtual space", that is, "virtual address space" and "virtual space" are used generically; similarly, "physical storage space" and "physical space" are common.

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present invention. In fig. 1, the host 130, the storage node 120, and the metadata management node 110 are interconnected via a network 140. The storage node 120, the management node 110, and the network 140 constitute a distributed storage system.

The host 130 may be a computer or other user device, and the host 130 may communicate with the distributed storage system via a network. The host 130 may comprise any computing device known in the art, such as a server, a desktop computer, etc., in which an operating system and other applications are installed in the host 130, and there may be multiple hosts 130. The host 130 may access files of the distributed storage system. For example, the host 130 may provide a file access interface to an application on the host through a file interface module.

The storage node 120 is used for storing file data and processing access requests for files. Storage node 120 may include at least one type of memory, for example, may include PCM, ReRAM, MRAM, and the like, new types of NVM having byte addressing and access capabilities. In the embodiment of the present invention, the file is stored in the NVM memory, and the storage node 120 can access the new NVM through the memory bus.

The management node 110 is used for managing metadata of a file, and processing a metadata query, an update request, and the like of a host to the file. The metadata of the file may include information such as a start address of the file, an allocated space size, and an actual size of the file.

The management node 110 and the storage node 120 may also be combined, that is, one node may have both functions of a metadata management node and a storage node.

In the embodiment of the present invention, the management node may be independently configured (for example, the management node 110 in fig. 1), or may be a node having functions of both the management node and the storage node. For convenience of description, the management node 110 is described as an example.

The physical layer, link layer of the network 140 may be based on ethernet, PCIE switching network, InfiniBand network, optical network, and the like. The network 140 is used for routing access requests and access responses, and in particular, routing may be performed by a network device (e.g., a router or a switch). For convenience of description, the following description will be made by taking a network device as an example.

FIG. 2 shows a schematic flow diagram of a method 200 of accessing a file according to an embodiment of the invention.

The method 200 is applied to a distributed storage system including a management node, a plurality of storage nodes, and a network device. The method 200 is performed by a host.

S210, a host obtains information of a first virtual space of a target file, where the first virtual space is used to manage the target file in the distributed storage system, the information of the first virtual space is used to point to a local virtual address space of a first storage node, and the first storage node is a storage node used to store the target file in the plurality of storage nodes.

In an embodiment of the present invention, the first virtual space is used to manage the target file in the distributed storage system. The first virtual space may be mapped to a local virtual address space of the storage node. That is, the target file is ultimately stored by a storage node (denoted as the first storage node) of the distributed storage system. The information of the first virtual space is used to point to a local virtual address space of the first storage node.

Optionally, as an embodiment of the present invention, the information of the first virtual space includes device information of the first storage node and a start address and a size of a local virtual address space in the first storage node, where the local virtual address space is used for storing the target file.

The information of the first virtual space may identify the first virtual space of the target file. For example, the device information of the first storage node may be an ID (or an ID and a node type) of the first storage node, and the first virtual space of the target file may be identified using the storage node ID and an address of a local virtual address space in the storage node.

Optionally, as another embodiment of the present invention, the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and the information of the first virtual space includes a start address and a size of the first virtual space.

The global virtual address space of a distributed storage system is the space that is virtualized for managing files in the distributed storage system. The local virtual address spaces of different storage nodes in the distributed storage system correspond to different portions of the global virtual address space of the distributed storage system. For example, a first portion of the global virtual address space is mapped to a local virtual address space of a first storage node in the distributed storage system, and a second portion of the global virtual address space is mapped to a local virtual address space of a second storage node in the distributed storage system, wherein the first portion of space is different from the second portion of space. In this embodiment, a global virtual address space of the distributed storage system is used to allocate a space (i.e., a first virtual space) for the target file, and since a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, the first virtual space may be mapped to a local virtual address space of the storage node. The address of the global virtual address space is a global virtual address. The information of the first virtual space includes a starting global virtual address and a space size of the first virtual space. The number of bits of the global virtual address may be greater than the number of bits of the local virtual address. Alternatively, the number of bits of the global virtual address may be 2N bits, where N is the number of bits of the address of the local virtual address space of the storage node (i.e., the number of bits of the local virtual address) in the distributed storage system. That is, the number of bits of the address of the first virtual space may be 2N bits, the upper N bits of the address of the first virtual space include the device information of the first storage node, and the lower N bits of the address of the first virtual space are the address of the local virtual address space in the first storage node for storing the target file. The device information of the first storage node may be an ID (or an ID and a node type) of the first storage node. The first virtual space of the target file may be identified using the address of the first virtual space.

The number of bits of the local virtual address is the number of bits of an Operating System (OS). For example, for a 64-bit OS, the number of bits of the local virtual address is 64 bits, and the number of bits of the global virtual address may be 128 bits, in which case the upper 64 bits of the global virtual address may contain device information of the storage node, and the lower 64 bits of the global virtual address may correspond to the local virtual address. It should be understood that only a portion of the upper 64 bits may be used to carry the device information for the storage node (e.g., the top 8 bits are the first storage node's ID), and the remaining bits may be reserved or left for other purposes.

Optionally, as an embodiment of the present invention, the host may obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.

The case of host access to a target file may be to create, open, and access the target file, or to open and access an already created target file. The host may obtain metadata of the target file from the management node when the target file is opened, and obtain a start address and a size of the first virtual space of the target file from the metadata of the target file.

S220, the host allocates a second virtual space for the target file in the local virtual address space of the host according to the information of the first virtual space.

In the embodiment of the present invention, optionally, the local virtual address space may adopt an unused section of the virtual address space of the OS. For example, the size of the virtual address space of the 64-bit OS is a power of 64 of 2, and a reserved virtual address space is reserved in addition to the user-state virtual address space and the kernel virtual address space, and the local virtual address space may be the virtual address space reserved by the OS. Or, the virtual address space of the operating system can be divided again according to the requirement, and a section of virtual address space is divided to be used as the local virtual address space.

The host allocates a section of virtual address space (denoted as the second virtual space) in the local virtual address space for the target file to be accessed. Optionally, the host allocates a second virtual space having the same size as the first virtual space for the target file in the local virtual address space of the host.

S230, the host maps the second virtual space to a virtual physical space, where the virtual physical space is a storage space that is virtualized locally in the host and is used for storing the target file.

In the embodiment of the present invention, the "virtual physical space" may also be expressed as a "virtual physical storage space" which represents a virtual storage space and does not exist physically.

For a file local to the host, the host may map the local virtual address space of the file to a local physical space. For a file in a distributed storage system, a host may map the local virtual address space (i.e., the second virtual space) of the file to a virtual physical space.

S240, the host establishes a corresponding relationship between the first virtual space and the virtual physical space.

Since the virtual physical space does not really store the target file, the target file is actually stored in the distributed storage system, and in order to realize access to the target file, the host may establish a corresponding relationship between the first virtual space and the virtual physical space.

Optionally, the corresponding relationship between the first virtual space and the virtual physical space may be a direct corresponding relationship, that is, the first virtual space directly corresponds to the virtual physical space; or may be in indirect correspondence, that is, the first virtual space corresponds to the second virtual space, and the second virtual space corresponds to the virtual physical space.

S250, when the host determines that the second access request accesses the virtual physical space, the host converts the second access request into a first access request for accessing the first virtual space according to the corresponding relationship between the first virtual space and the virtual physical space, wherein the first access request includes the device information of the first storage node.

An application in the host machine accesses the target file by accessing the second virtual space. For example, the host gives the application a file handle pointing to the starting address of the second virtual space, and the application accesses the second virtual space through the file handle. Since the second virtual space maps to the virtual physical space, the virtual physical space is accessed. And when the host determines that the accessed virtual physical space is the virtual physical space, converting the second access request into a first access request for accessing the first virtual space according to the corresponding relation between the first virtual space and the virtual physical space. The first access request carries information of a first virtual space, and the information of the first virtual space comprises equipment information of the first storage node. For example, if a global virtual address is used, the first access request carries an address of a first virtual space, where the address of the first virtual space includes the device information of the first storage node and a local virtual address in the first storage node; if the global virtual address is not adopted, the first access request directly carries the equipment information of the first storage node and the local virtual address in the first storage node.

S260, the host sends the first access request to the network device, so that the network device routes the first access request to the first storage node according to the device information of the first storage node.

The host sends the converted first access request to the network device, and the network device may route the first access request to the first storage node according to the information of the first virtual space in the first access request. Specifically, the network device may route the first access request to the first storage node according to the device information of the first storage node included in the information of the first virtual space.

After receiving a first access request sent by the network device, the first storage node accesses the target file according to the first access request. Specifically, the first storage node may determine, according to the address of the local virtual address space in the first access request, an address of a local physical storage space for storing the target file, and then access the local physical storage space. The first storage node then sends an access response to the network device in response to the first access request. The network device sends an access response of the first storage node to the host.

The method for accessing the file of the embodiment of the invention can directly access the physical storage space where the file in the distributed storage system is located without passing through the kernel of the host, and does not need to negotiate the directly accessed address, thereby reducing the delay and improving the efficiency of file access.

Therefore, in the method for accessing a file according to the embodiment of the present invention, when accessing a file in the distributed storage system, the second virtual space is allocated to the file in the local virtual address space of the host and mapped to the local virtual physical space of the host, and the first access request for accessing the first virtual space of the file in the distributed storage system is sent to the network when accessing the virtual physical space, so that the physical storage space where the file in the distributed storage system is located can be directly accessed without passing through the kernel of the host, thereby reducing latency and improving file access efficiency.

Fig. 3 is a diagram illustrating a spatial correspondence of files according to an embodiment of the present invention. As shown in fig. 3, the distributed storage system has M storage nodes in total. A first virtual space of a file in the distributed storage system corresponds to a segment of a local virtual address space of the first storage node and maps to a local physical storage space (i.e., a physical page, such as an NVM page) of the first storage node. When the host opens the file, a second virtual space is allocated in the local virtual address space of the host for the file, the second virtual space is mapped to the virtual physical space, and meanwhile, the corresponding relation between the first virtual space and the virtual physical space is established. In this way, when accessing a file, if it is determined to access the virtual physical space, a first access request for accessing the first virtual space is sent to the network, so that access to the file in the distributed storage system can be realized.

Fig. 4 is a schematic diagram illustrating a spatial correspondence relationship of files according to another embodiment of the present invention. The global virtual address space is used in fig. 4. As shown in fig. 4, a first virtual space of a file in the distributed storage system in the global virtual space of the distributed storage system corresponds to a segment of the local virtual address space of the first storage node and is mapped to the local physical storage space (i.e., a physical page, e.g., an NVM page) of the first storage node. When the host opens the file, a second virtual space is allocated in the local virtual address space of the host for the file, the second virtual space is mapped to the virtual physical space, and meanwhile, the corresponding relation between the first virtual space and the virtual physical space is established. In this way, when accessing a file, if it is determined to access the virtual physical space, a first access request for accessing the first virtual space is sent to the network, so that access to the file in the distributed storage system can be realized.

It should be understood that the specific examples in the embodiments of the present invention are provided only to help those skilled in the art better understand the embodiments of the present invention, and do not limit the scope of the embodiments of the present invention.

It should also be understood that, in various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Having described the method of accessing a file according to an embodiment of the present invention in detail above, a system and a host according to an embodiment of the present invention will be described below.

Fig. 5 shows a schematic block diagram of a system 500 according to an embodiment of the invention. As shown in fig. 5, the system 500 includes: a host 510 and a distributed storage system 520.

Distributed storage system 520 includes a management node 521, a plurality of storage nodes 522, and a network device 523.

The host 510 is configured to:

acquiring information of a first virtual space of a target file, wherein the first virtual space is used for managing the target file in the distributed storage system 520, the information of the first virtual space is used for pointing to a local virtual address space of a first storage node, and the first storage node is a storage node used for storing the target file in the plurality of storage nodes 522;

allocating a second virtual space for the target file in the local virtual address space of the host 510 according to the information of the first virtual space;

mapping the second virtual space to a virtual physical space, where the virtual physical space is a storage space that is locally virtualized on the host 510 for storing the target file;

the first access request is sent to the network device 523, so that the network device 523 routes the first access request to the first storage node according to the device information of the first storage node.

According to the system provided by the embodiment of the invention, when a file in the distributed storage system is accessed, the second virtual space is allocated to the file in the local virtual address space of the host and is mapped to the local virtual physical space of the host, and the first access request for accessing the first virtual space of the file in the distributed storage system is sent to the network when the virtual physical space is accessed, so that the physical storage space where the file in the distributed storage system is located can be directly accessed without passing through the kernel of the host, thereby reducing the delay and improving the file access efficiency.

In this embodiment of the present invention, optionally, the host 510 is specifically configured to obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.

In this embodiment of the present invention, optionally, the information of the first virtual space includes device information of the first storage node and a start address and a size of a local virtual address space in the first storage node, where the local virtual address space is used for storing the target file.

In this embodiment of the present invention, optionally, the first virtual space is a space allocated by the management node 521 for the target file in the global virtual address space of the distributed storage system 520, the local virtual address space of each storage node 522 in the distributed storage system 520 corresponds to a portion of the global virtual address space of the distributed storage system 520, and the information of the first virtual space includes a start address and a size of the first virtual space.

In this embodiment of the present invention, optionally, bits of the address of the first virtual space are 2N bits, N is a bit of the address of the local virtual address space of the first storage node, an upper N bit of the address of the first virtual space includes the device information of the first storage node, and a lower N bit of the address of the first virtual space is an address of the local virtual address space used for storing the target file in the first storage node.

In this embodiment of the present invention, optionally, the host 510 is specifically configured to allocate the second virtual space with the same size as the first virtual space for the target file in the local virtual address space of the host 510.

The host 510 and the distributed storage system 520 in the system 500 according to the embodiment of the present invention may correspond to the host and the distributed storage system in the foregoing method embodiment, and the host 510 may perform corresponding processes of the foregoing methods, which is not described herein again for brevity.

Fig. 6 shows a schematic block diagram of a host 600 according to an embodiment of the invention. As shown in fig. 6, the host 600 includes:

a processing module 610, configured to obtain information of a first virtual space of a target file, where the first virtual space is used to manage the target file in a distributed storage system, the information of the first virtual space is used to point to a local virtual address space of a first storage node, the first storage node is a storage node of the distributed storage system, the storage node is used to store the target file, according to the information of the first virtual space, a second virtual space is allocated to the target file in the local virtual address space of the host, the second virtual space is mapped to a virtual physical space, the virtual physical space is a storage space which is locally virtualized in the host and is used to store the target file, a corresponding relationship between the first virtual space and the virtual physical space is established, and when a second access request is determined to access the virtual physical space, the second access request is converted into an access request according to the corresponding relationship between the first virtual space and the virtual physical space A first access request of the first virtual space, wherein the first access request includes the device information of the first storage node;

a sending module 620, configured to send the first access request to a network device in the distributed storage system, so that the network device routes the first access request to the first storage node according to the device information of the first storage node.

According to the host of the embodiment of the invention, when accessing the file in the distributed storage system, the second virtual space is allocated for the file in the local virtual address space of the host and mapped to the virtual physical space, and the first access request for accessing the first virtual space of the file in the distributed storage system is sent to the network when accessing the virtual physical space, so that the physical storage space where the file in the distributed storage system is located can be directly accessed without passing through the kernel of the host, thereby reducing the delay and improving the efficiency of file access.

In this embodiment of the present invention, optionally, the processing module 610 is specifically configured to obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.

In this embodiment of the present invention, optionally, the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, a local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and the information of the first virtual space includes a start address and a size of the first virtual space.

In this embodiment of the present invention, optionally, the processing module 610 is specifically configured to allocate the second virtual space having the same size as the first virtual space to the target file in the local virtual address space of the host.

The host 600 according to the embodiment of the present invention may correspond to an execution main body of the method 200 for accessing a file according to the embodiment of the present invention, and the above and other operations and/or functions of each module in the host 600 are respectively for implementing corresponding flows of the foregoing methods, and are not described herein again for brevity.

Fig. 7 shows an architecture of a host provided by yet another embodiment of the invention, including at least one processor 702 (e.g., CPU), at least one network interface 705 or other communication interface, memory 706, and at least one communication bus 703. Wherein:

the communication bus 703 is used to enable connection communication between these devices.

The processor 702 is configured to execute executable modules, such as computer programs, stored in the memory 706.

The Memory 706 may comprise a high speed Random Access Memory (RAM) and may also include an NVM, such as at least one disk Memory.

The host computer is communicatively coupled to the network via at least one network interface 705 (which may be wired or wireless).

In some embodiments, the memory 706 stores the program 7061, and the processor 702 is configured to execute the program 7061 to implement the methods of the foregoing method embodiments.

It should be understood that, in the embodiment of the present invention, the term "and/or" is only one kind of association relation describing an associated object, and means that three kinds of relations may exist. For example, a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto.

Claims

A method for accessing files, the method being applied to a distributed storage system, the distributed storage system comprising a management node, a plurality of storage nodes and a network device, the method comprising:

a host acquires information of a first virtual space of a target file, wherein the first virtual space is used for managing the target file in the distributed storage system, the information of the first virtual space is used for pointing to a local virtual address space of a first storage node, and the first storage node is a storage node used for storing the target file in the plurality of storage nodes;

the host allocates a second virtual space for the target file in a local virtual address space of the host according to the information of the first virtual space;

the host maps the second virtual space to a virtual physical space, wherein the virtual physical space is a storage space which is locally virtualized in the host and used for storing the target file;

the host machine establishes a corresponding relation between the first virtual space and the virtual physical space;

when the host determines that a second access request accesses the virtual physical space, converting the second access request into a first access request for accessing the first virtual space according to the corresponding relation between the first virtual space and the virtual physical space, wherein the first access request comprises equipment information of the first storage node;

the host sends the first access request to the network device, so that the network device routes the first access request to the first storage node according to the device information of the first storage node.
The method of claim 1, wherein the host obtaining information of the first virtual space of the target file comprises:

and the host acquires the metadata of the target file and acquires the information of the first virtual space of the target file from the metadata of the target file.
The method according to claim 1 or 2, wherein the information of the first virtual space comprises device information of the first storage node and a start address and a size of a local virtual address space in the first storage node for storing the target file.
The method according to claim 1 or 2, wherein the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, the local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and the information of the first virtual space includes a start address and a size of the first virtual space.
The method according to claim 4, wherein the number of bits of the address of the first virtual space is 2N bits, N is the number of bits of the address of the local virtual address space of the first storage node, the upper N bits of the address of the first virtual space contain the device information of the first storage node, and the lower N bits of the address of the first virtual space are the address of the local virtual address space of the first storage node for storing the target file.
The method according to any one of claims 3 to 5, wherein the host allocates a second virtual space for the target file in a local virtual address space of the host according to the information of the first virtual space, comprising:

and the host allocates the second virtual space with the same size as the first virtual space for the target file in the local virtual address space of the host.
A system, comprising: a host and a distributed storage system;

the distributed storage system comprises a management node, a plurality of storage nodes and network equipment;

the host is used for:

acquiring information of a first virtual space of a target file, wherein the first virtual space is used for managing the target file in the distributed storage system, the information of the first virtual space is used for pointing to a local virtual address space of a first storage node, and the first storage node is a storage node used for storing the target file in the plurality of storage nodes;

according to the information of the first virtual space, distributing a second virtual space for the target file in a local virtual address space of the host;

mapping the second virtual space to a virtual physical space, wherein the virtual physical space is a storage space which is locally virtualized in the host and used for storing the target file;

establishing a corresponding relation between the first virtual space and the virtual physical space;

when a second access request is determined to access the virtual physical space, converting the second access request into a first access request for accessing the first virtual space according to the corresponding relation between the first virtual space and the virtual physical space, wherein the first access request comprises equipment information of the first storage node;

and sending the first access request to the network equipment so that the network equipment routes the first access request to the first storage node according to the equipment information of the first storage node.
The system of claim 7, wherein the host is specifically configured to obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.
The system according to claim 7 or 8, wherein the information of the first virtual space comprises device information of the first storage node and a start address and a size of a local virtual address space in the first storage node for storing the target file.
The system according to claim 7 or 8, wherein the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, the local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and the information of the first virtual space includes a start address and a size of the first virtual space.
The system according to claim 10, wherein the number of bits of the address of the first virtual space is 2N bits, N is the number of bits of the address of the local virtual address space of the first storage node, the upper N bits of the address of the first virtual space contain the device information of the first storage node, and the lower N bits of the address of the first virtual space are the address of the local virtual address space of the first storage node for storing the target file.
The system according to any of claims 9 to 11, wherein the host is specifically configured to allocate the second virtual space of the same size as the first virtual space for the target file in a local virtual address space of the host.
A host, comprising:

a processing module, configured to obtain information of a first virtual space of a target file, where the first virtual space is used to manage the target file in a distributed storage system, the information of the first virtual space is used to point to a local virtual address space of a first storage node, the first storage node is a storage node of the distributed storage system, the storage node is used to store the target file, according to the information of the first virtual space, a second virtual space is allocated to the target file in the local virtual address space of the host, the second virtual space is mapped to a virtual physical space, the virtual physical space is a storage space which is locally virtualized in the host and is used to store the target file, a corresponding relationship between the first virtual space and the virtual physical space is established, and when a second access request is determined to access the virtual physical space, converting the second access request into a first access request for accessing the first virtual space according to the corresponding relation between the first virtual space and the virtual physical space, wherein the first access request comprises the equipment information of the first storage node;

a sending module, configured to send the first access request to a network device in the distributed storage system, so that the network device routes the first access request to the first storage node according to the device information of the first storage node.
The host of claim 13, wherein the processing module is specifically configured to obtain metadata of the target file, and obtain information of the first virtual space of the target file from the metadata of the target file.
The host according to claim 13 or 14, wherein the information of the first virtual space includes device information of the first storage node and a start address and a size of a local virtual address space in the first storage node for storing the target file.
The host according to claim 13 or 14, wherein the first virtual space is a space allocated by the management node for the target file in a global virtual address space of the distributed storage system, the local virtual address space of each storage node in the distributed storage system corresponds to a part of the global virtual address space of the distributed storage system, and the information of the first virtual space includes a start address and a size of the first virtual space.
The host computer according to claim 16, wherein the number of bits of the address of the first virtual space is 2N bits, N is the number of bits of the address of the local virtual address space of the first storage node, the upper N bits of the address of the first virtual space contain the device information of the first storage node, and the lower N bits of the address of the first virtual space are the address of the local virtual address space of the first storage node for storing the target file.
The host machine according to any one of claims 15 to 17, wherein the processing module is specifically configured to allocate the second virtual space of the same size as the first virtual space for the target file in a local virtual address space of the host machine.