WO2022121385A1 - 一种文件访问方法、存储节点以及网卡 - Google Patents

一种文件访问方法、存储节点以及网卡 Download PDF

Info

Publication number
WO2022121385A1
WO2022121385A1 PCT/CN2021/115931 CN2021115931W WO2022121385A1 WO 2022121385 A1 WO2022121385 A1 WO 2022121385A1 CN 2021115931 W CN2021115931 W CN 2021115931W WO 2022121385 A1 WO2022121385 A1 WO 2022121385A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
data
storage area
network card
index
Prior art date
Application number
PCT/CN2021/115931
Other languages
English (en)
French (fr)
Inventor
罗先强
韩兆皎
李楚
王�锋
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21902093.0A priority Critical patent/EP4258122A1/en
Publication of WO2022121385A1 publication Critical patent/WO2022121385A1/zh
Priority to US18/331,274 priority patent/US20230315683A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14Handling requests for interconnection or transfer
    • G06F13/20Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • G06F16/1824Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • G06F16/183Provision of network file services by network file servers, e.g. by using NFS, CIFS

Definitions

  • the present application relates to the field of storage technologies, and in particular, to a file access method, a storage node, and a network card.
  • RDMA remote direct memory access
  • the client needs to store the file to the server, it needs to store various information of the file, such as the data in the file, the file log, and the file index.
  • the RDMA unilateral operation only stores one kind of information at a time, so multiple interactions between the client and the server are required to store various information of the file to the server respectively.
  • the present application provides a file access method, a storage node and a network card to improve file access efficiency.
  • an embodiment of the present application provides a file access method, where a second device (which can be used as a client) can initiate a file access request to the first device (which can be used as a server), and the file access request is used to request access to a storage device.
  • a file in a device the file access request can be used to request to read the first data in the file (that is, a file read request), and the file access request can also be used to request to write the second data in the file ( That is, a file write request).
  • the file access request includes a file identifier of the file; the file identifier may be allocated to the file by the first device (eg, a network card or processor of the first device) and notified to the second device in advance.
  • the network card of the first device When the file access request arrives at the first device, the network card of the first device first receives the file access request, and the network card of the first device can directly parse the file access request, and obtain the file identifier carried in the file access request. The file identifier accesses the file in the memory of the first device, and sends a file access response to the second device.
  • the network card of the first device can directly process the file access request without the intervention of the processor of the first device, which can improve file access efficiency.
  • only one interaction between the first device and the second device that is, the second device sends a file access request, and the first device feeds back a file access response
  • the network card of the first device may first send the file identifier to the second device.
  • the processor of the first device can open the file according to the file open request, and can also notify the network card of the first device to store the file in the memory area of the first device. .
  • the network card of the first device obtains the storage area of the file from the processor of the first device, a memory registration operation is performed for the storage area of the file, and a file identifier is allocated to the file; after that, the network card of the first device can send the file to the second device. logo.
  • the network card of the first device obtains the storage area of the file from the processor of the first device, and the file identifier is allocated by the network card as an example.
  • the embodiment of the present application does not limit the scenario in which the network card of the first device acquires the storage area of the file from the processor of the first device.
  • the file identifier may also be allocated to the file by the processor of the first device and notified to the network card of the first device.
  • the network card of the first device can obtain the storage area of the file from the processor of the first device, so that the network card of the first device can determine the storage location of the file, and after performing the memory registration, can store the file.
  • the area performs operations, such as reading data or writing data, to provide the possibility for subsequent access to the file by the network card of the first device.
  • the file in addition to the data in the file, the file also has file-related information, such as a file index, file information, and a file log.
  • the storage area of the file can be used to store the data in the file and related information of the file, that is to say, the storage area of the file can include one of the following: the data storage area of the file (used to store the data in the file), the index of the file Storage area (used to store file indexes), file information storage area (used to store file information), file log storage area (used to store file logs).
  • the storage area of the file may include an area for storing data in the file and file-related information, so that the network card of the first device can access the data in the file and file-related information.
  • the network card of the first device may determine the file index of the file according to the file identifier; after that, obtain the first data according to the file index , sending a file access response to the second device, where the file access response includes the first data.
  • the network card of the first device can read the first data from the file according to the file identifier, no processor is required to participate, and no need to interact with the second device for many times, thus ensuring the file reading efficiency.
  • the network card of the first device when the network card of the first device determines the file index of the file according to the file identifier, it may first determine the index storage area of the file according to the file identifier, and read the file index from the index storage area of the file.
  • the network card of the first device obtains the storage area of the file index from the processor of the first device in advance, the network card of the first device can obtain the file index from the index storage area of the file. The way is simpler and more convenient.
  • the file index may indicate the location of the data in the file in the data storage area of the file.
  • the network card of the first device can know the location of the data in the file in the data storage area of the file, and can also acquire the first data in the file from the data storage area of the file.
  • the network card of the first device can parse the file index and determine the position of the data in the file in the data storage area in the file. Because the network card of the first device obtains the storage area of the file from the processor of the first device in advance, and performs the memory registration operation, the network card of the first device can read the first data from the data storage area of the file. The data reading process only needs to be performed by the network card of the first device, which ensures the file reading efficiency.
  • the file access request when the file access request is for requesting to write the second data in the file, the file access request includes the second data.
  • the network card of the first device may write the second data in the file according to the file access request. After writing the second data, the network card of the first device may send a file access response to the second device, where the file access response is used to indicate that the second data is successfully written.
  • the network card of the first device can write the second data in the file according to the file access request, without requiring the participation of the processor or interacting with the second device for many times, thus ensuring the efficiency of file writing.
  • the network card of the first device can determine the file index according to the file identifier, and update the file index, the updated file index may indicate the storage location of the data in the file after the second data is written in the file, that is, the updated file index may indicate the storage location of the second data.
  • the updated file index can accurately indicate the storage location of the data in the file in the data storage area of the file after the second data is written.
  • the first type the file access request appends the second data to the file. That is, the writing method is append writing (append).
  • the network card of the first device When the network card of the first device writes the second data in the file, it first applies for the first storage area, and writes the second data into the first storage area.
  • the first storage area is located in the data storage area of the file.
  • the starting position of the area points to the end of the file, ie the second data written should be next to the data already stored in the file.
  • the second type The file access request is used to request to replace the data of the file with the second data, and keep the original data. That is, the writing method is rewrite.
  • the network card of the first device When the network card of the first device writes the second data in the file, it can first apply for the second storage area, and then write the second data into the second storage area, and the second storage area is located in the data storage area of the file.
  • the third type the file access request is used to request to replace the data of the file with the second data. That is, the writing method is rewriting (replace).
  • the file access request also includes the data length of the second data and the offset of the second data in the file.
  • the offset of the data of the second data in the file determines the data to be replaced from the data storage area of the file, and the second data is used to overwrite the data to be replaced.
  • the network card of the first device can implement data writing in different ways, which are suitable for different application scenarios.
  • the network card of the first device determines the file index according to the file identifier, when updating the file index, the index storage area of the file may be determined according to the file identifier, and then the file index is read from the index storage area of the file, Update the file index.
  • the network card of the first device can more conveniently obtain the file index and update the file index. It is ensured that the file index can accurately indicate the storage location of the data in the file in the data storage area of the file.
  • an embodiment of the present application provides a file registration method.
  • the method is applied to a first device that stores a file.
  • the second device can send a file opening request to the first device, and the file opening request file uses When the file opening request arrives at the first device, it is first received by the network card of the first device, and then the network card of the first device sends the file opening request to the processor of the first device.
  • the processor of the first device can open the file according to the file open request, and can also determine the storage area of the file in the memory of the first device, and notify the network card of the first device of the storage area of the file.
  • the network card of the first device can send a file opening response to the second device, the file opening response is used to indicate that the file is opened successfully, and the file opening response includes a file identifier of the file, and the file identifier is that the network card of the first device is allocated for the file (such as After the processor of the first device notifies the network card of the first device of the storage area of the file, the network card of the first device may perform a memory registration operation for the storage area of the file, and may also assign a file identifier to the file), or The file is allocated by the processor of the first device (the processor of the first device may notify the network card of the first device of the file identification).
  • the processor of the first device can notify the network card of the first device of the storage area of the file to ensure that the network card of the first device can know the storage area of the file, so that the network card of the first device can access the file subsequently.
  • the processor of the first device may allocate a storage area for the file when determining the storage area of the file.
  • the processor of the first device can allocate a storage area for the file, so that the file can be stored in the memory of the first device, so that the network card of the first device can access the file subsequently.
  • the file storage area includes at least one of the following: a file data storage area, a file index storage area, a file information storage area, and a file log storage area.
  • the storage area of the file may include an area for storing data in the file and file-related information, ensuring that the network card of the first device can access the data in the file and file-related information.
  • an embodiment of the present application provides a storage node, which can implement file registration and file access.
  • Storage nodes include network cards and memory.
  • the network card is used to receive the file access request from the client.
  • the file access request is used to request access to the file, and the file access request includes the file ID of the file; according to the file ID, the file in the memory of the storage node is accessed, and the file access request is sent to the client response;
  • the memory may be a volatile memory such as random access memory (RAM) or dynamic random access memory (DRAM) or a combination of both. It can also be non-volatile memory, such as storage class memory (SCM), etc., or a combination of volatile memory and non-volatile memory.
  • RAM random access memory
  • DRAM dynamic random access memory
  • SCM storage class memory
  • the storage node further includes a processor
  • the processor can determine the storage area of the file in the memory, and notify the network card of the storage area of the file. For example, when the processor node receives a file open request from the client, the processor can open the file according to the file open request, and can also determine the file open request. The storage area of the file in the memory, and notify the network card of the storage area of the file;
  • the network card can obtain the storage area of the file from the processor, perform a memory registration operation for the storage area of the file, and assign a file identifier to the file; send the file identifier to the client, for example, the network card can send a file open response to the client, the file open response Include the file ID.
  • the processor may allocate a storage area for the file when determining the storage area of the file.
  • the file storage area includes at least one of the following: a file data storage area, a file index storage area, a file information storage area, and a file log storage area.
  • the network card can also access the file. Specifically, the network card can receive a file access request from the client, the file access request is used to request access to the file, and the file read request includes the file identifier; after that, the network card A file can be accessed based on the file ID and a file access response is sent to the client.
  • the file access request may be a file read request for requesting to read the first data of the file.
  • the network card accesses the file and sends the file access response to the client, it can determine the file index of the file according to the file identifier; after that, it obtains the first data according to the file index, and sends the file access response to the client.
  • the file access response includes the first data. a data.
  • the network card when the network card determines the file index of the file according to the file identifier, the network card may first determine the index storage area of the file according to the file identifier, and then read the file index from the index storage area of the file.
  • the file index is used to indicate the location of the data in the file in the data storage area of the file
  • the network card can determine the location of the data in the file in the data storage area of the file according to the file index, read the first data in the data storage area.
  • the file access request is a file write request for requesting to write the second data in the file; the file access request includes the second data.
  • the network card accesses the file according to the file identifier and sends a file access response to the client, it can write second data in the file and send the file access response to the client, where the file access response is used to indicate that the second data is successfully written.
  • the network card may further determine the file index of the file according to the file identifier, update the file index, and the updated file index indicates the storage location of the second data.
  • the first type the file access request is used to request to additionally write the second data in the file.
  • the network card can first apply for the first storage area, and write the second data into the first storage area.
  • the first storage area is located in the data storage area of the file, and the first storage area starts with The position points to the end of the file.
  • the second type the file access request is used to request to replace the data in the file with the second data, and keep the original data.
  • the network card When the network card writes the second data in the file, it can apply for a second storage area, write the second data into the second storage area, and the second storage area is located in the data storage area of the file.
  • the third type the file access request is used to request to replace the data in the file with the second data.
  • the file access request also includes the data length of the second data and the offset of the second data in the file.
  • the offset of the data in the file determines the data to be replaced from the data storage area of the file, and overwrites the data to be replaced with the second data.
  • the network card when the network card determines the file index according to the file identifier and updates the file index, the network card may determine the file index storage area according to the file identifier, read the file index from the file index storage area, and update the file index.
  • an embodiment of the present application further provides a file access device.
  • the file access device is deployed on a network card and has the function of implementing the behavior in the method example of the first aspect.
  • the functions can be implemented by hardware, or can be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more units corresponding to the above functions.
  • the structure of the apparatus includes a receiving unit, a processing unit, and a sending unit, and these units can perform the corresponding functions of the network card in the method example of the first aspect.
  • the structure of the apparatus includes a receiving unit, a processing unit, and a sending unit, and these units can perform the corresponding functions of the network card in the method example of the first aspect.
  • an embodiment of the present application further provides a file registration device.
  • the file registration device is deployed on a processor and has the function of implementing the behavior in the method example of the second aspect.
  • the functions can be implemented by hardware, or can be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more units corresponding to the above functions.
  • the structure of the apparatus includes a receiving unit, a processing unit, and a sending unit, and these units can perform the corresponding functions of the processor in the method example of the second aspect. For details, please refer to the detailed description in the method example. No further elaboration here.
  • an embodiment of the present application further provides a network card, the network card has a function of implementing the behavior in the method example of the first aspect, and the beneficial effects can be referred to the description of the first aspect and will not be repeated here.
  • the structure of the network card may include a processor and a memory, and the processor is configured to support the network card to perform the corresponding functions of the network card in the method of the first aspect or the second aspect.
  • the memory is coupled to the processor and stores necessary program instructions and data for the network card.
  • the network card may also include an interface for communicating with other devices, for example, receiving a file access request, sending a file access response, and the like.
  • the structure of the network card may also include a processor and an interface, and the processor is configured to support the network card to perform the corresponding functions in the method of the first aspect.
  • the processor may also transmit data through the interface, for example, receive file access requests, send file access responses, and the like.
  • the present application further provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer enables the computer to execute the first aspect and each possibility of the first aspect.
  • the present application also provides a computer program product comprising instructions, which, when run on a computer, causes the computer to execute the method described in the above-mentioned first aspect and each possible implementation manner of the first aspect or cause the computer to The methods described in the above second aspect and various possible implementations of the second aspect are performed.
  • the present application further provides a computer chip, the chip is connected to a memory, the chip is used to read and execute a software program stored in the memory, and execute the above-mentioned first aspect and various possibilities of the first aspect
  • the method described in the implementation manner of the above-mentioned second aspect and the method described in each possible implementation manner of the second aspect are performed.
  • FIG. 1 is a schematic diagram of the architecture of a system provided by the application.
  • FIG. 2 is a schematic structural diagram of a server provided by the present application.
  • Fig. 3 is a kind of method schematic diagram of file opening and closing provided by this application.
  • Fig. 4 is the schematic diagram of a kind of data table that this application provides
  • FIG. 5 is a schematic diagram of a file access method provided by the present application.
  • FIG. 6 is a schematic diagram of a file reading method provided by the application.
  • FIG. 7 is a schematic structural diagram of a data packet header provided by the present application.
  • FIG. 8 is a schematic diagram of a file writing method provided by the application.
  • FIG. 9 is a schematic diagram of a file additional writing method provided by the application.
  • FIG. 10 is a schematic structural diagram of a data packet header provided by the application.
  • FIG. 11 is a schematic diagram of a file replacement writing method provided by the application.
  • FIG. 12 is a schematic structural diagram of a data packet header provided by the application.
  • FIG. 13 is a schematic structural diagram of a file access device provided by the application.
  • FIG. 14 is a schematic structural diagram of a file registration device provided by the present application.
  • the embodiments of the present application relate to file reading. In order to facilitate the understanding of the embodiments of the present application, first, some terms involved in the present application are explained.
  • a file is a form of presenting data.
  • the file may be a video file, a voice file or a text file.
  • the file can include the same type of data, for example, the data in the file are all voice data or pictures.
  • the data in the file may also include different types of data, for example, the file may be a communication file transmitted between computing devices, and the file may include data such as pictures and texts.
  • Files can be stored in different memories. When storing a file, in addition to storing the data in the file, it is also necessary to store related information of the file, and the related information of the file is used to describe the file.
  • the relevant information of the file includes some or all of the following: file index (index), file information (file information), file log (file log), and file identification.
  • File IDs are used to uniquely identify a file.
  • the client such as the network card of the client
  • the server the network card of the server
  • the file identifier is provided by the server to the client (for example, in a scenario where the client initiates a file open request to the server, the processor or network card of the server assigns a file identifier to the file, and then the network card provides to the client).
  • the embodiment of this application does not limit the type of the file identifier.
  • the file identifier can be the remote key assigned to the file when the network card of the server performs memory registration, or the network card or processor of the server can assign the file to the file. number and other information. Any information that can uniquely identify a file is applicable to the embodiments of the present application.
  • the file index is used to indicate the storage location of the data in the file in the server memory.
  • the file index may indicate the location of the data in the file in the data storage area of the file.
  • the data storage area for this file is located in the server's memory.
  • the file information is attribute information of the file, and the file information may include the file size and related information describing the file being accessed, such as the time when the file is accessed, the data size of the file being accessed, and the offset of the data.
  • the file log is the information required for file recovery (such as data recovery in the file), and the file log can ensure the reliability of file storage to a certain extent.
  • the contents recorded in the file log are different in different application scenarios.
  • RDMA is a direct memory access method. Based on RDMA, a computing device can directly read and write data to the memory of other computing devices without processing by the processor.
  • the processor of the server can register the file to the network card of the server, that is, the processor of the server can notify the network card of the server of the storage area of the file, so that the network card can obtain the storage area of the file.
  • the storage area of the file is a storage area for storing data in the file and related information of the file.
  • the file storage area may include some or all of the following: a file index storage area, a file information storage area, a file log storage area, and a file data storage area.
  • the file index storage area is the area where the file index is stored in the server memory
  • the file information storage area is the area where the file information is stored in the server memory
  • the file log storage area is the area where the file log is stored in the server memory.
  • the data storage area of the file is the area in the server memory where the data in the file is stored.
  • the data storage area of the file is the storage area allocated for the file by the processor of the server and used for storing data.
  • the data storage area of the file may include an area where data has been stored and a blank area, and the blank area may be used to store data that needs to be newly written when the file is subsequently written.
  • the file index indicates the storage location of the data in the current file. Every time data is written in the file, the file index needs to be updated so that the file index can indicate the storage location of the data in the file after the data is written.
  • RDMA can directly read and write data to memory.
  • the premise of reading and writing data to memory is to realize memory registration.
  • Memory registration can be understood as obtaining the control right of the memory, so as to realize the data reading and writing of the memory.
  • Memory registration is a protection measure for memory in RDMA. Only by registering memory in the RDMA memory region (MR), this memory can be operated by RDMA. After the memory registration is completed, data can be read and written to the memory.
  • MR RDMA memory region
  • the processor of the server can register the file to the network card of the server (the network card has an RDMA engine), that is, the processor of the server can notify the network card of the server of the storage area of the file, and the network card is informed of the storage of the file.
  • a memory registration operation can be performed for the storage area of the file to register the storage area of the file in the RDMA memory domain.
  • the network card After the network card performs the memory registration operation, it can operate the data in the file, the file index, the file information, and the log of the file, which is convenient for subsequent reading or writing of the file, for example, reading the data in the file, or in the file. Write data in the file, update the file index, file information, and file log, etc.
  • RDMA supports a total of three queues, send queues (send queues, SQ), receive queues (receive queues, RQ), and complete queues (complete queues, CQ).
  • SQ and RQ are usually created in pairs, which are called queue pairs (QP).
  • RDMA is a message-based transmission protocol, and data transmission is an asynchronous operation.
  • the operation process of RDMA is as follows:
  • the processor of the host submits a work request (work request, WR) to the network card, and the network card configures the work request to the work queue (work queues, WQ), and the work queue includes a send queue (SQ) and a receive queue. (RQ).
  • Each element of the work queue is called (work queues element, WQE), and an element corresponds to a WR.
  • the processor of the host can obtain the work completion (work complete, WC) from the completion queue (CQ) through the network card.
  • work completion work complete
  • CQ completion queue
  • Each element in the completion queue is called (complete queues element, CQE), and one element corresponds to one WC.
  • hardware with an RDMA engine such as a network card
  • RDMA engine can be regarded as a queue element processing module.
  • the hardware continuously fetches the work request (WR) from the work queue (WQ) for execution, and places the work completion (WC) in the completion queue (CQ) after execution.
  • RDMA is a message-based data transfer protocol, and all packet assembly is done on hardware with an RDMA engine.
  • the send/receive operation in RDMA is a bilateral operation, that is, the sending and receiving must be completed with the participation of the remote application awareness.
  • both A and B must create and initialize their own QP and CQ.
  • a and B respectively add WQEs to their respective WQs.
  • A's network card asynchronously schedules A's WQE, resolves that this is a send message, and directly sends the data pointed to by WQE to B. After the data stream reaches B's network card, B's WQE is consumed, and B's network card directly stores the data in the cache pointed to by WQE.
  • a completion message CQE will be generated in the CQ of A to indicate that the transmission is completed.
  • a completion message CQE is also generated in the CQ of B to indicate that the reception is completed.
  • FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application. As shown in FIG. 1 , the system includes a server 100 and a client 200 .
  • the client 200 is deployed on the user side, and is used to initiate a request to the server 100 under the trigger of the user.
  • the embodiment of the present application does not limit the specific form of the client 200, for example, the client 200 may be a user's local computing device or a dedicated client.
  • a file system client (file system client, FS client) may be deployed on the client 200 , and the file system client may be software running on the client 200 .
  • the user can access the files stored in the server 100 through the FS client set in the local computing device, and instruct the server 100 (such as the network card of the server 100) to read and write the files stored in the server 100 (in this Writing in the application embodiment includes operations such as append writing and replacement writing).
  • a network card 210 can also be deployed on the client 200.
  • the FS client can trigger the network card 210 to generate a file access request (such as a file read request, a file write request), and send the file through the network card.
  • the access request is sent to the server 100 .
  • the server 100 can store the user's file.
  • the server 100 is essentially a storage node and has a file storage function.
  • the embodiment of the present application does not limit the deployment location of the server 100.
  • the server 100 may be deployed in an edge data center, a cloud data center, or a terminal computing device.
  • the server 100 may also be distributed in some or all of the environments of edge data centers, cloud data centers, and terminal computing devices.
  • a file system runs on the server 100, and the file system may be software running on the processor 120 of the server 100.
  • the file system manages the files stored on the server 100, and can allocate a storage area for the file in the memory 130 of the server 100, and can also notify the network card 140 of the file storage area, that is, register the file to the network card 140, triggering
  • the network card 140 performs memory registration operations. After the network card 140 knows the storage area of the file, it can read and write the file. When the network card 140 needs to read or write the file, the network card 140 can create a file handle. The operation of the file handle realizes the reading and writing of files.
  • the interaction between the client 200 and the server 100 may implement operations such as reading and writing files.
  • the network card 140 of the server 100 can obtain the storage area of the file from the processor 120 of the server 100, perform a memory registration operation, and can also assign a file identifier to the file.
  • the network card 140 of the server 100 can directly determine the storage area of the file (eg, the index storage area of the file) according to the file identifier.
  • the server 100 (such as the network card of the server 100) can determine the storage area of the file according to the file identifier carried in the file access request, and according to the file access request The file reading or file writing is performed, and an access response (such as a file reading response and a file writing response) may also be sent to the client 200 after the file reading and file writing.
  • an access response such as a file reading response and a file writing response
  • the access operation of the server 100 to the file may not be performed by the processor of the server 100, but is performed by the network card 140 of the server 100.
  • the server 100 includes a bus 110 , a processor 120 , a memory 130 and a network card 140 .
  • the processor 120 , the memory 130 , and the network card 140 communicate through the bus 110 .
  • the processor 120 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (artificial intelligence, AI) chip, system on chip (SoC) or complex programmable logic device (CPLD), graphics processing unit (GPU), etc.
  • CPU central processing unit
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • AI artificial intelligence
  • SoC system on chip
  • CPLD complex programmable logic device
  • GPU graphics processing unit
  • the memory 130 may include volatile memory such as RAM, DRAM, and the like. It can also be a non-volatile memory (non-volatile memory), such as SCM, etc., or a combination of volatile memory and non-volatile memory, and the like.
  • the memory 130 may also include other software modules required for running processes such as an operating system.
  • the operating system can be LINUX TM , UNIX TM , WINDOWS TM and so on.
  • the processor 120 can execute the instructions by calling the computer stored in the memory 130, so that the server 100 can perform the operations performed by the processor 120 of the server 100 in the embodiment shown in FIG. 3 .
  • the server 100 may also include auxiliary memory, which may also be referred to as external memory, and the external memory may be a non-volatile memory, such as a read-only memory (read-only memory, ROM), hard disk drive (HDD) or solid state drive (solid state disk, SSD), etc.
  • auxiliary memory such as a read-only memory (read-only memory, ROM), hard disk drive (HDD) or solid state drive (solid state disk, SSD), etc.
  • ROM read-only memory
  • HDD hard disk drive
  • SSD solid state drive
  • the processor 120 may transfer the file from the external memory into the memory 130, and the processor 120 may allocate a storage area for the file in the memory 130, and the allocated storage area may be used to store the file.
  • the processor 120 may determine the storage area of the file in the memory 130, and notify the network card 140 of the storage area of the file. If the file to be opened is not loaded into the memory 130 in advance, when the processor 120 receives the file opening request, it may allocate a storage area for the file in the memory 130 to store the file, and store the allocated storage area for the file. The zone is notified to the network card 140 .
  • the processor 120 can log out the storage area of the file and delete the file stored in the memory 120 .
  • the network card 140 can not only perform data transmission (eg, receive file open/close requests, file access requests, send file open/close responses, and file access responses), but also read and write files.
  • the network card 140 includes a processor 141 and an interface 142.
  • the processor 141 enables the network card 140 in the server 100 to perform the operations performed by the network card 140 of the server 100 in the embodiments shown in FIGS. 3 , 5 , 8 , 9 and 11 , and the processor 141 can also Information is sent or received through the interface 142, such as receiving a file open request, file access request, file close request, etc., or sending a file open response, a file access response, and a file close response.
  • the file In the traditional file reading and writing process, the file is not registered on the network card, and the network card cannot directly read and write the file. Therefore, when reading and writing a file, the client needs to obtain the file index from the server first, and then the client The storage location of the data in the file is determined according to the file index, and the network card on the server side reads or writes data from the storage location indicated by the client under the instruction of the client (for example, the client initiates an RDMA unilateral operation). Requires multiple interactions between client and server. Or when reading and writing files, the processor on the server side needs to obtain the file data under the instruction of the client, and send the file data through the network card. In this way, the participation of the processor is required, and the efficiency is poor.
  • the processor 120 in the server 100 may first register the file to In the network card 140, the network card 140 is notified of the storage area of the file, and the network card 140 is triggered to perform a memory registration operation.
  • the network card 140 of the server 100 assigns a file identifier to the file, and feeds back the file identifier to the client 200 .
  • the network card 140 of the server 100 After receiving the file access request carrying the file identifier from the client 200, the network card 140 of the server 100 accesses the file according to the file identifier, and reads or writes the file.
  • the client 200 and the server 100 do not need to interact with each other for many times, and the processor 120 of the server 100 does not need to participate, which can effectively improve the file access efficiency.
  • the file access method provided by the embodiments of the present application can be divided into two processes, namely, a file registration and memory registration process, and a file access process, wherein the file access process can be further divided into a file reading process and a file writing process. Each of them will be described below.
  • the processor 120 of the server 100 may inform the server 100 of the storage area of the file in the network card 140, and trigger the network card 140 to perform a memory registration operation, so as to prompt the network card 140 to assign a file identifier to the file.
  • the client 200 wants to read or write a file, it can carry the file identifier in the request to identify the file that needs to be read or written. Since the network card 140 has acquired the storage area of the file and performed the memory registration operation, the network card 140 can read or write data from the storage area of the file according to the file identifier.
  • the method includes:
  • Step 301 the user initiates a file opening request to the server 100 through the client 200 .
  • the user can click the file on the interface provided by the client 200; after detecting the user's operation, the client 200 determines that the user needs to open the file, and initiates a file opening request to the server 100, and the file opening request carries the name of the file.
  • the name of the file may be assigned by the user for the file, and the name of the file is stored in the server 100 when the server 100 stores the file.
  • the network card 210 in the client 200 can send a file open request to the network card 140 in the server 100 based on RDMA, that is, the file open request is added as a WQE in the WQ to the SQ on the client 200 side, and the WQE Indicates that the file needs to be opened.
  • Step 302 the network card 140 in the server 100 requests after receiving the file open request, and sends the file open request to the processor 120 of the server 100 , and the processor 120 in the server 100 opens the file.
  • the network card 140 in the server 100 can obtain the file open request from the RQ on the server 100 side, and send the file open request to the processor 120 in the server 100, and the processor 120 in the server 100 can access the file according to the file.
  • the open request opens the file and determines the storage area for the file.
  • the processor 120 in the server 100 can also expand the data storage area of the file. In the case of expanding the data storage area of the file, the processor 120 in the server 100 may also notify the network card 140 of the expanded data storage area of the file.
  • the log storage area for files is also similar.
  • the file that the user needs to open is a file that is not stored in the memory 130 of the server 100
  • the user can also initiate a file opening request to the server 100 through the client 200 .
  • the processor 120 in the server 100 determines that the file is not currently stored in the memory 130 according to the name of the file carried in the file open request, and the processor 120 in the server 100 can store the file in the memory 130 for the file. Allocate storage area.
  • Step 303 The processor 120 in the server 100 registers the file with the network card 140 in the server 100, and notifies the network card 140 of the storage area of the file.
  • the processor 120 in the server 100 After the processor 120 in the server 100 determines or allocates the storage area of the file, it can register the file in the network card 140 in the server 100, that is, the processor 120 in the server 100 can store the file. The area is notified to the network card 140, so that the network card 140 obtains the storage area of the file. Specifically, the processor 120 in the server 100 can send the information used to describe the storage area of the file (herein referred to as the information of the storage area of the file) to the network card 140, and trigger the network card 140 to perform a memory registration operation to allocate the file to the network card 140. For the file identifier, the network card 140 can store the information of the storage area of the file and the file identifier.
  • the storage area of the file includes the index storage area of the file, the data storage area of the file, the information storage area of the file, and the log storage area of the file.
  • the information of the storage area of the file includes information of the index storage area of the file, information of the data storage area of the file, information of the information storage area of the file, and information of the log storage area of the file.
  • the information of the index storage area of the file is used to describe the index storage area of the file.
  • the information of the index storage area of the file may include the starting storage address of the file index and the ending storage address of the file index.
  • the information of the index storage area of the file may include the starting storage address of the file index and the size of the file index.
  • Step 304 After the network card 140 in the server 100 obtains the storage area of the file, the network card 140 in the server 100 performs a memory registration operation for the storage space of the file, and assigns a file identifier to the file.
  • the storage area of the file can be added to the RDMA memory domain to ensure that the network card 140 can operate the storage area of the file, such as reading Get the file index and data in the file in the storage area of the file, update the file information or file log, etc.; for another example, write data in the storage area of the file, update the file index, file information or file log, etc.
  • the network card 140 in the server 100 may also record the information of the storage space of the file and the file identifier.
  • the information of the storage area of the file and the file identifier saved by the network card 140 in the server 100 may be the data table shown in FIG. 4 .
  • the data table exemplarily shows the information of the index storage area of the file, the information of the log storage area of the file, the information storage area of the file and the file identifier.
  • the information in the storage area of the file information may be the address of the file information.
  • the information of the index storage area of the file includes an index address (index address) and an index length (index size).
  • index address is used to indicate the starting storage address of the file index.
  • index length is used to indicate the length of the file index.
  • the information in the log storage area of the file includes the log key (log mkey), log address, log length (log size), or some or all of the indications in the log offset.
  • log mkey log key
  • log address log address
  • log length log size
  • the log length is used to indicate the length of the log.
  • the file index can indicate the storage location of the data in the file in the data storage area of the file.
  • the network card 140 knows The data storage area of the file, the log storage area of the file, the information storage area of the file, and the index storage area of the file are listed.
  • the network card 140 may determine the file index corresponding to the file identifier according to the data table shown in FIG. 4 , and determine the data in the file according to the file index.
  • Step 305 The processor 120 in the server 100 sends a file open response to the client 200 through the network card 140 in the server 100, where the file open response carries the file identifier.
  • the network card 140 in the server 100 can send a file open response to the client 200 based on RDMA, that is, add the file open response as a WQE in the WQ to the SQ on the server 100 side, and the WQE instructs the file to open.
  • the network card 210 in the client 200 can obtain the file open response from the RQ on the client 200 side, and obtain the file identifier from the file open response.
  • the client 200 can trigger the server 100 to register the file in the network card 140 by sending a file open request.
  • the user when the user closes the file through the client 200 , the user can also trigger the server 100 to log out the file in the network card 140 through the client 200 .
  • the server 100 for details, refer to steps 306 to 309.
  • Step 306 the user initiates a file closing request to the server 100 through the client 200 .
  • the user can select the "close file” option on the interface provided by the client 200; after detecting the user's operation, the client 200 determines that the user needs to close the file, and initiates a file closing request to the server 100, which is in the file opening request. Carry the document ID.
  • the network card 210 of the client 200 can send a file close request to the server 100 based on RDMA, the client 200 sends a file close request to the server 100 based on RDMA, and the network card 140 of the client 200 sends a file open request to the server 100 based on RDMA.
  • RDMA RDMA close request
  • RDMA file close request
  • RDMA file close request
  • RDMA file close request
  • the network card 140 of the client 200 sends a file open request to the server 100 based on RDMA.
  • Step 307 After the processor 120 of the server 100 receives the file closing request, the processor 120 of the server 100 closes the file, for example, the processor 120 can release the resources occupied by the file.
  • Step 308 the processor 120 in the server 100 cancels the file from the network card 140 in the server 100 .
  • the processor 120 in the server 100 can cancel the storage area of the file from the network card 140 in the server 100, such as informing the network card 140 to delete the log storage area of the file, the information storage area of the file, and the information of the data storage area of the file. and the file ID.
  • Step 309 After the processor 120 in the server 100 closes the file, it may send a file closing response to the client 200 through the network card 140 in the server 100 .
  • the network card 140 in the server 100 can send a file close response to the client 200 based on RDMA, and the network card 140 in the server 100 sends a file close response to the client 200 based on RDMA.
  • the manner of sending the file open response to the client 200 is similar, for details, please refer to the foregoing content, which will not be repeated here.
  • the processor 120 in the server 100 registers the file to the network card 140 in the server 100 as an example in the scenario where the client 200 initiates a file open request.
  • the embodiments of the present application do not limit the scenario in which the processor 120 of the server 100 registers the file with the network card 140 in the server 100 .
  • the server 120 registers the file to the network card 140 of the server.
  • a file access method provided in an embodiment of the present application includes:
  • Step 501 the user initiates a file access request to the server 100 through the client 200 .
  • the file access request is used to request access to a file, and the file access request includes a file identifier.
  • the user can trigger the network card 210 of the client 200 to initiate a file access request through the file system client deployed by the client 200, and the file system client can send the file access request carrying the file identifier to the network card 210 of the client 200, instructing the network card 210 to send the file access request to the network card 210 of the client 200.
  • the server 100 initiates the file access request. In this way, the network card 210 of the client 200 may not perceive the information in the file access request.
  • the file access request may be used to request to read the first data in the file, that is, the file access request is a file read request.
  • the file access request can also be used to request to write the second data in the file, that is, the file access request is a file write request.
  • Step 502 The network card 140 of the server 100 receives the file access request from the second device, and accesses the file according to the file identifier.
  • the network card 140 of the server 100 can parse the file identifier from the file access request, determine the file index according to the file identifier, and then determine the storage location of the data in the file.
  • the network card 140 of the server 100 can read the first data from the file according to the file read request.
  • the network card 140 of the server 100 may write the second data in the file according to the file write request.
  • Step 503 the network card 140 of the server 100 sends a file access response to the client 200 .
  • the file access response is a file read response, and the file read response includes the first data.
  • the file access response is a file write response
  • the file write response indicates that the second data is successfully written.
  • the file access process is divided into a file reading process and a file writing process.
  • the following describes the file reading process and the file writing process respectively.
  • the user can read the data in the file from the server 100 through the client 200 .
  • a file reading method provided in an embodiment of the present application includes:
  • Step 601 The user initiates a file reading request to the server 100 through the client 200 (eg, the network card 210 of the client 200).
  • the file read request is used to request to read the first data in the file.
  • the first data may be all the data in the file, or may be part of the data in the file.
  • the user can select the first data to be read from the file on the interface provided by the client 200, such as selecting data in the file, such as clicking the "read” option; after the client 200 detects the user's operation, It is determined that the user needs to read the first data of the file, and a file read request is initiated to the server 100, where the file read request carries the file identifier and related information of the first data.
  • the relevant information of the first data may include the data length of the first data, the offset of the first data in the data in the file (for example, indicating that the first bit of the first data is the same as the first bit in the data in the file). one-bit offset), etc.
  • the data header of the file reading request may carry relevant information of the first data, and may also indicate the type of the file reading request.
  • the format of a data message header of a file read request provided by an embodiment of the present application, as shown in FIG. 7 , the data message header includes a file operation code (operation code, OP) , a file identifier, a data length (length) of the first data, and an offset (offset) of the first data in the data in the file.
  • operation code operation code
  • OP file operation code
  • the OP may be read, which is used to indicate that the request is a file read request, and is used to request to read the first data in the file.
  • the data packet header may further include a page index, where the page index is used to indicate the page where the first data is stored on the server 100 side (the page is a page in the data storage area of the file).
  • the page index may be calculated and determined by the client 200 according to the offset and the size of the page.
  • the page size is fixed, such as 4 kilobytes (kb), and the page of the first data can be determined by using the ratio of the offset to the page size, and then the page index can be determined.
  • the data packet header may not carry the page index
  • the network card 140 in the server 100 may calculate the page index according to the offset and the size of the page.
  • OP and offset can occupy 0 to 3 bytes
  • page index can occupy 4 to 7 bytes
  • file identifier can occupy 8 to 11 bytes
  • length can occupy 12 to 14 bytes.
  • the embodiment of the present application does not limit the number of bytes occupied by each information in the data packet header and the position of each information in the data packet header, as shown in FIG. 7 is only an example.
  • the file read request may not carry relevant information of the first data.
  • the file read request may carry an indicator, and the indicator may be the client 200 and the character predetermined by the server 100, the indicator is used to indicate that all data in the file needs to be read.
  • Step 602 After receiving the file read request, the network card 140 in the server 100 obtains the first data. When the file read request reaches the server 100, firstly, the network card 140 of the server 100 receives the file read request. In step 602, the network card 140 of the server 100 may directly process the file reading request and parse the file reading request.
  • the network card 140 in the server 100 may determine the index storage area of the file according to the file identifier, and obtain the file index therefrom. After acquiring the file index, the network card 140 in the server 100 can determine the storage location of the data in the file according to the file index, and then according to the relevant information of the first data, such as the length of the first data and the first data in the file. The offset of all data, the first data is read from the storage address of the data in the file.
  • the network card 140 in the server 100 can also perform validity verification on the client 200 according to the file identifier.
  • the network card 140 in the server 100 will generate a remote key (remote key) and a local key (local key) for the file, and the remote key can be sent to the client 200 as a file identifier, and the local key can be saved locally.
  • the network card 140 in the server 100 can compare the file identifier with the locally stored local key, and if they are consistent, pass the validity verification of the client 200; otherwise, the verification fails and the service
  • the network card 140 in the terminal 100 can directly reject the file reading request, or can transmit the file reading request to the processor 120 in the server 100, and the processor 120 in the server 100 performs the file reading request. deal with.
  • Step 603 After acquiring the first data, the network card 140 in the server 100 may send a file read response to the client 200, where the file read response carries the first data of the file.
  • the format of the file read response may conform to the format of the file read response (read response) defined in RDAM.
  • the user can also add new data to the file stored in the server 100 through the client 200 or replace the data in the file stored in the server 100 through the client 200, that is, the file writing process.
  • a method for writing a file includes:
  • Step 801 The user initiates a file writing request to the server 100 through the client 200, the file writing request is used for requesting to write second data in the file, and the file writing request includes the file identifier and the second data.
  • the user can send the file writing request through the network card 210 of the client 200.
  • the manner of writing the second data in the file may be adding the second data to the data in the file, or replacing part or all of the data in the file with the second data.
  • Step 802 After receiving the file writing request, the network card 140 in the server 100 writes the second data into the file.
  • the network card 140 of the server 100 receives the file writing request.
  • the network card 140 of the server 100 can directly process the file writing request, that is, can parse the file writing request.
  • the network card 140 in the server 100 applies for a new storage area from the data storage area of the file, and can write the second data into the applied-for data storage area. in the storage area.
  • the network card 140 in the server 100 may replace part or all of the data in the storage area of the data in the file with the second data data (in this way, replace in the replacement write), the network card 140 in the server 100 can also apply for a storage area from the data storage area of the file, and can write the second data into the applied storage area (this The way is to replace the rewrite scene in the write).
  • Step 803 The network card 140 in the server 100 updates the file index, so that the file index can indicate the storage location of the data in the file after writing the second data.
  • the file index is updated so that the file index can indicate the storage location of the data in the file after the second data is written.
  • the network card of the server 100 140 may not update the file index, that is, step 803 does not need to be performed.
  • the file log may also be updated, and the file log may record the additionally written second data and the storage location of the second data.
  • the network card 140 in the server 100 may also update the file information, and record information such as the time of writing the second data in the file information.
  • the file log can also be updated, and the file log can record the written second data and the storage location of the second data.
  • the network card 140 in the server 100 may also update the file information, and record information such as the time of writing the second data in the file information.
  • the client 200 may also be validated for validity according to the file identifier.
  • the network card 140 in the server 100 performs validity verification on the client 200 according to the file identifier, reference may be made to the foregoing description, which will not be repeated here.
  • Step 804 The network card 140 in the server 100 sends a file write response to the network card 140 of the client 200, notifying the client 200 that the second data has been successfully written.
  • the format of the file write response may conform to the format of the file write response (write response) defined in RDAM.
  • the file writing process is divided into additional writing and replacement writing, and the two file writing processes are described below respectively.
  • the file write request in the additional write process is called the first file write request
  • the corresponding file write response is called the first file write response.
  • the file write request in the replacement write process is called the second file write request
  • the corresponding file write response is called the second file write response.
  • FIG. 9 a schematic flowchart of additional writing of a file provided by an embodiment of the present application, the method includes:
  • Step 901 the user initiates a first file writing request to the network card 140 in the server 100 through the network card 210 of the client 200 .
  • the first file writing request is used to request to additionally write second data in the file, where the second data may be second data that needs to be added in the file.
  • the user can input or select the second data that needs to be added in the file on the interface provided by the client 200, such as inputting or selecting the second data that needs to be added in the file; after detecting the user's operation, the client 200 determines the user
  • a first file write request is initiated to the server 100, and the first file write request carries the file identifier and the second data.
  • related information of the second data such as the data length of the second data, may also be carried.
  • the data message body of the first file writing request may carry the second data
  • the data message header of the first file writing request may carry relevant information of the second data (such as the data length) and can also indicate the type of write request for this file.
  • the second data that can be carried in the data message body of the second file write request the data
  • the message header includes the file operation type (operation code, OP) and the file identifier.
  • the OP may be append (append), which is used to indicate that the request is a file writing request, and is used to request to additionally write the second data in the file.
  • the data packet header may also include the data length (length) of the second data, the offset (offset) of the second data in all data of the file, and the page index (page index).
  • the offset may be equal to the data length of all data in the current file.
  • page index refer to the foregoing content, which will not be repeated here.
  • the embodiment of the present application does not limit the number of bytes occupied by each information in the data packet header and the position of each information in the data packet header, as shown in FIG. 10 is only an example.
  • Step 902 After the network card 140 in the server 100 receives the first file writing request, the network card 140 in the server 100 can apply for a storage area for the second data from the data storage area of the file, and the second data Write to the storage area to which the application is made.
  • the network card 140 of the server 100 applies for a storage area for the second data, and the size of the storage area is equal to the data length of the second data. That is to say, the starting position of the applied storage area (here the starting position indicates a virtual address) is the ending storage position of the data stored in the file, that is, the starting position of the applied storage area points to the end of the file. end.
  • Step 903 The network card 140 in the server 100 may update the file index, and the updated file index may indicate the storage location of the second data.
  • the network card 140 in the server 100 determines the file index corresponding to the file identifier according to the data table shown in FIG. 4 , and then determines the index storage area of the file according to the file index, reads the file index, and updates the file index.
  • the network card 140 in the server 100 can also update the file information and the file log.
  • update method of the file information and the file log reference may be made to the relevant descriptions in the embodiment shown in FIG. 7 , which will not be repeated here.
  • Step 904 After writing the second data, the network card 140 in the server 100 may send a first file write response to the network card 210 in the client 200, and the second data has been successfully written through the client 200.
  • a schematic flowchart of file replacement and writing provided by an embodiment of the present application, the method includes:
  • Step 1101 the user initiates a second file writing request to the network card 140 in the server 100 through the network card 210 of the client 200 .
  • the second file writing request is used to request to replace data in the file with second data, where the second data may be data that needs to be replaced and written in the file.
  • the user can input or select the second data that needs to be written in the file on the interface provided by the client 200, such as inputting or selecting the second data that needs to be replaced in the file; after detecting the user's operation, the client 200 determines the user If the second data needs to be written in the file, a second file writing request is initiated to the server 100, and the second file writing request carries the file identifier and the second data.
  • related information of the second data may also be carried, such as the data length of the second data, the offset of the data of the second data in the file, and the like.
  • the data message body of the second file writing request may carry the second data
  • the data message header of the second file writing request may carry relevant information of the second data (such as the data length, the data offset of the second data in the file), and may also indicate the type of the file write request.
  • the second data that can be carried in the data message body of the second file write request the The data packet header includes a file operation type (operation code, OP), a file identifier, a data length (length) of the second data, and an offset (offset) of the second data in all data of the file.
  • the OP may be a replacement (replace).
  • the network card 140 in the server 100 needs to overwrite the data in the file and write the second data.
  • the OP may be rewrite.
  • the network card 140 in the server 100 may retain the existing data in the file and apply for a new storage area to write the second data. If the OP is replace or rewrite, both can indicate that the request is a file write request, but the way of writing the second data is different.
  • the data packet header may further include a page index, and the description of the page index may refer to the foregoing content, which will not be repeated here.
  • the embodiment of the present application does not limit the number of bytes occupied by each information in the data packet header and the position of each information in the data packet header, as shown in FIG. 12 is only an example.
  • Step 1102 After receiving the second file writing request, the network card 140 in the server 100 determines a storage area for the second data, and writes the second data in the storage area for the second data.
  • the storage area of the second data may be the space of the stored data in the data storage area of the file (corresponding to the scenario where the OP is replace), or the network card 140 in the server 100 may be the second data in the data storage area of the file The storage area applied for (corresponding to the scenario where the OP is rewrite).
  • the network card 140 in the server 100 determines, according to the relevant information of the second data, such as the data length of the second data and the offset of the data of the second data in the file, the data required in the data storage area of the file.
  • the storage area of the replaced data, the storage area is the storage area of the second data, and the second data is written in the storage area.
  • the network card 140 in the server 100 may not overwrite the existing data in the file, that is, keep the existing data in the file, and the network card 140 in the server 100 can store data in the file in the data storage area (
  • the blank area) is the second data application storage area
  • the size of the storage area may be equal to the data size of the second data
  • the second data is written into the applied storage area.
  • the storage area applied here may not be adjacent to the area where the data has been stored in the data storage area of the file, that is, the starting position of the storage area applied for may not point to the end of the file.
  • the network card 140 of the server 100 may not update the file index, that is, step 1103 does not need to be performed.
  • the network card 140 of the server 100 needs to update the file index, that is, step 1103 needs to be performed.
  • Step 1103 The network card 140 in the server 100 determines the file index according to the file identifier, and updates the file index.
  • the network card 140 of the server 100 can determine the index storage area of the file according to the file identifier, read the file index from the index storage area of the file, and read the file index according to the data length of the second data and the offset of the second data in the file. Determine the part of the file index for indicating the storage location of the data that needs to be replaced, and modify the part so that the part can indicate the storage location of the second data.
  • the network card 140 in the server 100 can also update the file information and file log of the file.
  • file information of the file and the update method of the file log reference may be made to the relevant descriptions in the embodiment shown in FIG. 8 , which will not be repeated here.
  • Step 1104 After writing the second data, the network card 140 in the server 100 may send a second file write response to the network card 210 of the client 200 to notify the client 200 that the second data has been successfully written.
  • the file identifier is allocated to the network card 140, and the file identifier can be used to verify the validity of the client 200 as an example for description.
  • the file identifier may also be allocated by the processor 120 in the server 100 for the file, and the processor 120 in the server 100 may also notify the file identifier when registering the file with the network card 140 To the network card 140 in the server 100.
  • the remote key allocated to the file may not be used as the file identifier, but only used for the client 200 to verify the validity.
  • the file identifier and the remote key carry in the file open response.
  • the file identifier may also be allocated to the file by the network card 140 in the server 100, and the file identifier is not a remote key. That is, the network card 140 in the server 100 can assign a file identifier and a remote key to the file. Carry the file ID and remote key in the file open response.
  • the embodiments of the present application further provide a file access device, which can be deployed on a network card and used to execute the above-mentioned FIG. 3 , FIG. 5 , FIG. 8 , FIG. 9 , and FIG.
  • a file access device which can be deployed on a network card and used to execute the above-mentioned FIG. 3 , FIG. 5 , FIG. 8 , FIG. 9 , and FIG.
  • the apparatus 1300 includes a receiving unit 1301 , a processing unit 1302 and a sending unit 1303 .
  • the receiving unit 1301 is configured to receive a file access request from the second device, where the file access request is used to request access to a file, and the file access request includes a file identifier of the file.
  • the processing unit 1302 is configured to access a file in the memory of the first device according to the file identifier.
  • the sending unit 1303 is configured to send a file access response to the second device.
  • the processing unit 1302 may obtain the storage area of the file from the processor of the device, and perform a memory registration operation for the storage area of the file to The file is assigned a file identifier; then, the sending unit 1303 may send the file identifier to the second device.
  • the file storage area includes at least one of the following:
  • the data storage area of the file The data storage area of the file, the index storage area of the file, the information storage area of the file, and the log storage area of the file.
  • the file access request is used to request to read the first data of the file; when the processing unit 1302 accesses the file according to the file identifier and sends a file access response to the second device, the processing unit 1302 can determine the file according to the file identifier Then, according to the first data obtained from the file index, a file access response is sent to the second device, and the file access response includes the first data.
  • the processing unit 1302 may determine the index storage area of the file according to the file identifier, and read the file index from the index storage area of the file.
  • the file index is used to indicate the position of the data in the file in the data storage area of the file.
  • the processing unit 1302 obtains the first data according to the file index, it can store the data from the file according to the file index. Get the first data in the area.
  • the file access request is used to request to write the second data in the file
  • the file access request includes the second data
  • the processing unit 1302 can write the first data in the file when accessing the file according to the file identifier.
  • Two data; the sending order can send a file access response to the second device, and the file access response is used to indicate that the second data is successfully written.
  • the processing unit 1302 may also determine the file index according to the file identifier, update the file index, and the updated file index is used to indicate the storage location of the second data.
  • the first type the file access request appends the second data to the file. That is, the writing method is append writing (append).
  • the processing unit 1302 When the processing unit 1302 writes the second data in the file, it first applies for the first storage area, and writes the second data into the first storage area.
  • the first storage area is located in the data storage area of the file, and the first storage area is in the data storage area.
  • the starting position points to the end of the file, that is, the second data written should be stored next to the existing data in the file.
  • the second type The file access request is used to request to replace the data of the file with the second data, and keep the original data. That is, the writing method is rewrite.
  • the processing unit 1302 may apply for the second storage area first, and then write the second data into the second storage area, which is located in the data storage area of the file.
  • the third type the file access request is used to request to replace the data of the file with the second data. That is, the writing method is rewriting (replace).
  • the file access request also includes the data length of the second data and the offset of the second data in the file.
  • the processing unit 1302 writes the second data in the file, it can The offset of the data in the file determines the data to be replaced from the data storage area of the file, and overwrites the data to be replaced with the second data.
  • the processing unit 1302 when the processing unit 1302 determines the file index according to the file identifier and updates the file index, the processing unit 1302 can determine the index storage area of the file according to the file identifier, read the file index from the index storage area of the file, and update the file. index.
  • an embodiment of the present application further provides a file registration device, where the file registration device is deployed on a processor and is used to execute the processor 120 in the above method embodiment shown in FIG. 3 .
  • the apparatus 1400 includes a processing unit 1402 .
  • the processing unit 1402 is configured to determine the storage area of the file in the memory of the first device, and notify the network card of the first device of the storage area of the file.
  • the apparatus 1400 further includes a receiving unit 1401 and a sending unit 1403.
  • the receiving unit 1401 may receive the data from the second device.
  • File open request file open request is used to request to open a file.
  • the sending unit 1403 may send a file open response to the second device through the network card of the first device, where the file open response is used to indicate that the file is opened successfully.
  • the processing unit 1402 may allocate a storage area for the file when determining the storage area of the file.
  • the file storage area includes at least one of the following:
  • the data storage area of the file The data storage area of the file, the index storage area of the file, the information storage area of the file, and the log storage area of the file.
  • the embodiment of the present application further provides a storage node, where the storage node is used to execute the method embodiments shown in FIG. 3 , FIG. 5 , FIG. 8 , FIG. 9 , and FIG. 11 above.
  • the storage node is used to execute the method embodiments shown in FIG. 3 , FIG. 5 , FIG. 8 , FIG. 9 , and FIG. 11 above.
  • the relevant features can be found in the foregoing method embodiments, which will not be repeated here.
  • the division of units in the embodiments of the present application is illustrative, and is only a logical function division, and other division methods may be used in actual implementation.
  • Each functional unit in the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination.
  • the above-described embodiments may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that contains one or more sets of available media.
  • the usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media.
  • the semiconductor medium may be a solid state drive (SSD).
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种文件访问方法、存储节点以及网卡,其中,该方法包括:第二设备可以向第一设备发起文件访问请求,文件访问请求包括文件的文件标识,第一设备的网卡接收到该文件访问请求后,可以直接对该文件访问请求进行解析,获取该文件访问请中携带的文件标识,然后根据该文件标识访问第一设备的内存中的文件,并向第二设备发送文件访问响应。当第一设备在接收到文件访问请求时,第一设备的网卡可以对该文件访问请求进行处理,不需要第一设备的处理器介入,能够提高文件访问效率。

Description

一种文件访问方法、存储节点以及网卡 技术领域
本申请涉及存储技术领域,尤其涉及一种文件访问方法、存储节点以及网卡。
背景技术
目前,在客户端和服务端之间实现文件读写需要执行多次交互。例如,在客户端与服务端之间基于远程直接内存访问(remote direct memory access,RDMA)协议现实数据传输的情况下,若客户端需要读取服务端上存储的文件时,客户端需要先发起文件索引获取请求,从服务端读取文件索引。客户端在获取该文件索引后,利用文件索引向服务端发起文件读取请求,从服务端获取文件。
若客户端需要将文件存储至服务端时,需要存储文件的多种信息,如文件中的数据、文件日志、文件索引等。而RDMA单边操作只一次存储一种信息,故而需要客户端与服务端之间多次交互,分别将文件的多种信息存储到服务端。
由上可知,无论是文件读取还是文件写入,客户端和服务端之间均需执行多次交互,效率较低、耗时较长。
发明内容
本申请提供一种文件访问方法、存储节点以及网卡,用以提升文件访问效率。
第一方面,本申请实施例提供了一种文件访问方法,第二设备(可以作为客户端)可以向第一设备(可以作为服务端)发起文件访问请求,文件访问请求用于请求访问存储第一设备中的文件,该文件访问请求可以用于请求读取文件中的第一数据(也即为文件读取请求),该文件访问请求也可以用于请求在文件中写入第二数据(也即为文件写入请求)。文件访问请求包括文件的文件标识;文件标识可以是第一设备(如第一设备的网卡或处理器)为文件分配的,并预先通知给第二设备的。
当文件访问请求到达第一设备时,第一设备的网卡先接收该文件访问请求,第一设备的网卡可以直接对该文件访问请求进行解析,获取该文件访问请中携带的文件标识,根据该文件标识访问第一设备的内存中的文件,并向第二设备发送文件访问响应。
通过上述方法,当第二设备需要访问第一设备中的文件时,第一设备的网卡可以直接对文件访问请求处理,不需要第一设备的处理器介入,能够提高文件访问效率。另外,第一设备和第二设备之间仅需交互一次(也即第二设备发送文件访问请求,第一设备反馈文件访问响应)就可以实现文件访问,减少了设备间的交互次数,保证了文件访问的高效性。
在一种可能的实现方式中,第一设备的网卡接收来自第二设备的文件访问请求之前,可以先向第二设备发送文件标识。
例如,当第二设备向第一设备发起文件打开请求时,第一设备的处理器可以根据文件打开请求打开文件,还可以通知第一设备的网卡在第一设备的内存中该文件的存储区域。当第一设备的网卡从第一设备的处理器获取文件的存储区域后,针对文件的存储区域执行内存注册操作,为文件分配文件标识;之后,第一设备的网卡可以向第二设备发送文件标识。
上述说明中,是以在文件打开场景下,第一设备的网卡从第一设备的处理器获取文件的存储区域、以及文件标识是由网卡分配的为例进行说明的。实际上,本申请实施例并不 限定第一设备的网卡从第一设备的处理器获取文件的存储区域的场景。该文件标识也可以是由第一设备的处理器为文件分配的,并通知给第一设备的网卡的。
通过上述方法,第一设备的网卡可以从第一设备的处理器获取文件的存储区域,使得该第一设备的网卡能够确定文件的存储位置,在执行了内存注册之后,可以对该文件的存储区域进行操作,如读取数据或写入数据,为后续由第一设备的网卡访问文件提供可能性。
在一种可能的实现方式中,文件除了文件中的数据,还有文件的相关信息,如文件索引,文件信息以及文件日志。文件的存储区域可以用于存储文件中的数据、文件的相关信息,也就是说,文件的存储区域可以包括下列之一:文件的数据存储区域(用于存储文件中的数据)、文件的索引存储区域(用于存储文件索引)、文件的信息存储区域(用于存储文件信息)、文件的日志存储区域(用于存储文件日志)。
通过上述方法,该文件的存储区域中可以包括用于存储文件中的数据、以及文件相关信息的区域,使得第一设备的网卡可以对文件中的数据、以及文件相关信息进行访问。
在一种可能的实现方式中,当文件访问请求用于请求读取文件的第一数据时,第一设备的网卡可以根据文件标识确定文件的文件索引;之后,再根据文件索引获取第一数据,向第二设备发送文件访问响应,文件访问响应中包括第一数据。
通过上述方法,第一设备的网卡可以根据文件标识从文件中读取第一数据,不再需要处理器参与,也不需要与第二设备交互多次,保证了文件读取效率。
在一种可能的实现方式中,第一设备的网卡根据文件标识确定文件的文件索引时,可以先根据文件标识确定文件的索引存储区域,从文件的索引存储区域中读取文件索引。
通过上述方法,由于第一设备的网卡预先从第一设备的处理器获取了文件索引的存储区域,使得第一设备的网卡能够从该文件的索引存储区域中获取该文件索引,文件索引的获取方式更加简单、便捷。
在一种可能的实现方式中,文件索引可以指示文件中的数据在文件的数据存储区域中的位置。第一设备的网卡在获取该文件索引后,可以获知该文件中的数据在文件的数据存储区域中的位置,也就能够从文件的数据存储区域获取该文件中的第一数据。
通过上述方法,第一设备的网卡能够解析该文件索引,确定文件中的数据在文件中的数据存储区域中的位置。由于第一设备的网卡预先从第一设备的处理器获取了文件的存储区域,并执行了内存注册操作,使得第一设备的网卡能够从该文件的数据存储区域中读取第一数据。数据读取过程仅需第一设备的网卡执行,保证了文件读取效率。
在一种可能的实现方式中,当文件访问请求用于请求在文件中写入第二数据时,文件访问请求包括第二数据。第一设备的网卡可以根据该文件访问请求,在文件中写入第二数据。在写入该第二数据后,第一设备的网卡可以向第二设备发送文件访问响应,文件访问响应用于指示第二数据成功写入。
通过上述方法,第一设备的网卡可以根据文件访问请求在文件中写入第二数据,不再需要处理器参与,也不需要与第二设备交互多次,保证了文件写入效率。
在一种可能的实现方式中,若在文件中写入第二数据,导致文件中的数据在文件的数据存储区域的位置发生变化,第一设备的网卡可以根据文件标识确定文件索引,更新文件索引,更新后的文件索引可以指示文件中写入第二数据后,文件中的数据的存储位置,也即更新后的文件索引可以指示第二数据的存储位置。
通过上述方法,该更新后的文件索引能够准确的指示在第二数据写入后,文件中的数据在文件的数据存储区域的存储位置。
在一种可能的实现方式中,文件访问请求用于请求在文件中写入第二数据的方式有很多种,下面分别进行说明:
第一种:该文件访问请求在文件中追加写入第二数据。也即写入方式为追加写(append)。
第一设备的网卡在文件中写入第二数据时,先申请第一存储区域,将第二数据写入到第一存储区域中,第一存储区域位于文件的数据存储区域中,第一存储区域的起始位置(起始虚拟地址)指向文件的末尾,也即写入的第二数据应当紧挨着文件中已存储的数据。
第二种:文件访问请求用于请求将文件的数据替换为第二数据,保留原始数据。也即写入方式为替换(rewrite)。
第一设备的网卡在文件中写入第二数据时,可以先申请第二存储区域,之后,将第二数据写入到第二存储区域中,第二存储区域位于文件的数据存储区域中。
第三种:文件访问请求用于请求将文件的数据替换为第二数据。也即写入方式为重写(replace)。
文件访问请求还包括第二数据的数据长度以及第二数据在文件中的数据的偏移量,第一设备的网卡在文件中写入第二数据时,可以根据第二数据的数据长度以及第二数据在文件中的数据的偏移量从文件的数据存储区域确定需要替换的数据,利用第二数据覆盖需要替换的数据。
通过上述方法,第一设备的网卡可以实现不同方式的数据写入,适用于不同应用场景。
在一种可能的实现方式中,第一设备的网卡根据文件标识确定文件索引,更新文件索引时,可以根据文件标识确定文件的索引存储区域,之后从文件的索引存储区域中读取文件索引,更新文件索引。
通过上述方法,第一设备的网卡能够较为方便的获取文件索引,更新文件索引。保证了文件索引能够准确指示文件中的数据在文件的数据存储区域中的存储位置。
第二方面,本申请实施例提供了一种文件注册方法,方法应用于存储文件的第一设备,在该方法中,第二设备可以向第一设备发送文件打开请求,该文件打开请求文件用于请求打开文件;当文件打开请求到达第一设备时,先由第一设备的网卡接收,之后,第一设备的网卡将该文件打开请求发送给第一设备的处理器。
第一设备的处理器在接收该文件打开请求后,可以根据文件打开请求打开该文件,还可以确定第一设备的内存中文件的存储区域,并向第一设备的网卡通知文件的存储区域。之后,第一设备的网卡可以向第二设备发送文件打开响应,文件打开响应用于指示文件打开成功,文件打开响应包括文件的文件标识,文件标识为第一设备的网卡为文件分配的(如第一设备的处理器在通知第一设备的网卡该文件的存储区域后,第一设备的网卡可以针对该文件的存储区域执行内存注册操作,也可以为该文件分配文件标识),也可以是由第一设备的处理器为该文件分配的(第一设备的处理器可以将该文件标识通知给第一设备的网卡)。
通过上述方法,第一设备的处理器可以将文件的存储区域通知给第一设备的网卡,保证第一设备的网卡能够获知该文件的存储区域,便于后续第一设备的网卡能够访问文件。
在一种可能的实现方式中,若第一设备的内存中未存储该文件,第一设备的处理器在 确定文件的存储区域时,可以为文件分配存储区域。
通过上述方法,第一设备的处理器能够为文件分配存储区域,使得该文件可以存储在该第一设备的内存中,便于后续第一设备的网卡访问文件。
在一种可能的实现方式中,文件的存储区域包括下列的至少一个:文件的数据存储区域、文件的索引存储区域、文件的信息存储区域、文件的日志存储区域。
通过上述方法,该文件的存储区域中可以包括用于存储文件中的数据、以及文件相关信息的区域,保证第一设备的网卡可以对文件中的数据、以及文件相关信息进行访问。
第三方面,本申请实施例提供了一种存储节点,该存储节点能够实现文件注册的方式也可以实现文件访问的方式,具体有益效果可以参见第一方面和第二方面的说明,此处不再赘述。存储节点包括网卡以及内存。
网卡,用于接收来自客户端的文件访问请求,文件访问请求用于请求访问文件,文件访问请求包括文件的文件标识;根据文件标识访问该存储节点的内存中的文件,并向客户端发送文件访问响应;
内存,用于存储该文件。该内存可以为易失性存储器,例如随机存取存储器(random access memory,RAM)或动态随机存取存储器(dynamic random access memory,DRAM)或两者组合。也可以为非易失性存储器,例如,存储级内存(storage class memory,SCM)等,或者为易失性存储器与非易失性存储器的组合。
在一种可能的实现方式中,存储节点还包括处理器;
处理器可以确定该内存中文件的存储区域,向网卡通知文件的存储区域,例如当处理器节点到来自客户端的文件打开请求时,处理器可以根据该文件打开请求打开该文件,还可以确定该内存中文件的存储区域,并向网卡通知文件的存储区域;
网卡可以从处理器获取文件的存储区域,针对文件的存储区域执行内存注册操作,为文件分配文件标识;向客户端发送文件标识,例如,网卡可以向客户端发送文件打开响应,该文件打开响应中包括该文件标识。
在一种可能的实现方式中,若内存中未存储该文件,处理器在确定文件的存储区域时,可以为文件分配存储区域。
在一种可能的实现方式中,文件的存储区域包括下列的至少一个:文件的数据存储区域、文件的索引存储区域、文件的信息存储区域、文件的日志存储区域。
在一种可能的实现方式中,网卡还可以对文件进行访问,具体的,网卡可以接收来自客户端的文件访问请求,文件访问请求用于请求访问文件,文件读取请求包括文件标识;之后,网卡可以根据文件标识访问文件,并向客户端发送文件访问响应。
在一种可能的实现方式中,文件访问请求可以为文件读取请求,用于请求读取文件的第一数据。网卡在访问文件,并向客户端发送文件访问响应时,可以根据文件标识确定文件的文件索引;之后,根据文件索引中获取第一数据,向客户端发送文件访问响应,文件访问响应中包括第一数据。
在一种可能的实现方式中,网卡在根据文件标识确定文件的文件索引时,可以先根据文件标识确定文件的索引存储区域,之后再从文件的索引存储区域中读取文件索引。
在一种可能的实现方式中,文件索引用于指示文件中的数据在文件的数据存储区域中的位置,网卡可以根据文件索引确定文件中的数据在文件的数据存储区域中的位置,从文 件的数据存储区域中读取第一数据。
在一种可能的实现方式中,文件访问请求为文件写入请求,用于请求在文件中写入第二数据;文件访问请求包括第二数据。网卡根据文件标识访问文件,并向客户端发送文件访问响应时,可以在文件中写入第二数据,向客户端发送文件访问响应,文件访问响应用于指示第二数据成功写入。
在一种可能的实现方式中,网卡还可以根据文件标识确定文件的文件索引,更新文件索引,更新后的文件索引以指示第二数据的存储位置。
在一种可能的实现方式中,文件访问请求用于请求在文件中写入第二数据的方式有很多种,下面分别进行说明:
第一种、文件访问请求用于请求在文件中追加写入第二数据。
网卡在文件中写入第二数据时可以先申请第一存储区域,将第二数据写入到第一存储区域中,第一存储区域位于文件的数据存储区域中,第一存储区域的起始位置指向文件的末尾。
第二种、文件访问请求用于请求将文件中的数据替换为第二数据,保留原始数据。
网卡在文件中写入第二数据时,可以申请第二存储区域,将第二数据写入到第二存储区域中,第二存储区域位于文件的数据存储区域中。
第三种、文件访问请求用于请求将文件中的数据替换为第二数据。
文件访问请求还包括第二数据的数据长度以及第二数据在文件中的数据的偏移量,网卡在文件中写入第二数据时,可以根据第二数据的数据长度以及第二数据在文件中的数据的偏移量从文件的数据存储区域中确定需要替换的数据,利用第二数据覆盖需要替换的数据。
在一种可能的实现方式中,网卡在根据文件标识确定文件索引,更新文件索引时,可以根据文件标识确定文件的索引存储区域,从文件的索引存储区域中读取文件索引,更新文件索引。
第四方面,本申请实施例还提供了一种文件访问装置,该文件访问装置部署在网卡上,具有实现上述第一方面的方法实例中行为的功能,有益效果可以参见第一方面的描述此处不再赘述。所述功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个与上述功能相对应的单元。在一个可能的设计中,所述装置的结构中包括接收单元、处理单元、发送单元,这些单元可以执行上述第一方面方法示例中网卡的相应功能,具体参见方法示例中的详细描述,此处不做赘述。
第五方面,本申请实施例还提供了一种文件注册装置,该文件注册装置部署在处理器上,具有实现上述第二方面的方法实例中行为的功能,有益效果可以参见第二方面的描述此处不再赘述。所述功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个与上述功能相对应的单元。在一个可能的设计中,所述装置的结构中包括接收单元、处理单元、发送单元,这些单元可以执行上述第二方面方法示例中处理器的相应功能,具体参见方法示例中的详细描述,此处不做赘述。
第六方面,本申请实施例还提供了一种网卡,该网卡具有实现上述第一方面的方法实例中行为的功能,有益效果可以参见第一方面的描述此处不再赘述。
在一种可能的实现方式中,所述网卡的结构中可以包括处理器和存储器,所述处理器 被配置为支持所述网卡执行上述第一方面或第二方面方法中网卡相应的功能。所述存储器与所述处理器耦合,其保存所述网卡必要的程序指令和数据。该网卡中还可以包括接口,用于与其他设备通信,例如,接收文件访问请求、发送文件访问响应等。
在另一种可能的实现方式中,所述网卡的结构中也可以包括处理器和接口,所述处理器被配置为支持所述网卡执行上述第一方面方法中相应的功能。该处理器还可以通过所述接口传输数据,例如,接收文件访问请求、发送文件访问响应等。
第七方面,本申请还提供一种计算机可读存储介质,所述计算机可读存储介质中存储有指令,当其在计算机上运行时,使得计算机执行上述第一方面以及第一方面的各个可能的实施方式中所述的方法。或当其在计算机上运行时,使得计算机执行上述第二方面以及第二方面的各个可能的实施方式中所述的方法。
第八方面,本申请还提供一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述第一方面以及第一方面的各个可能的实施方式中所述的方法或使得计算机执行上述第二方面以及第二方面的各个可能的实施方式中所述的方法。
第九方面,本申请还提供一种计算机芯片,所述芯片与存储器相连,所述芯片用于读取并执行所述存储器中存储的软件程序,执行上述第一方面以及第一方面的各个可能的实施方式中所述的方法,或执行上述第二方面以及第二方面的各个可能的实施方式中所述的方法。
附图说明
图1为本申请提供的一种系统的架构示意图;
图2为本申请提供的一种服务端的结构示意图;
图3为本申请提供的一种文件打开和关闭的方法示意图;
图4为本申请提供的一种数据表的示意图;
图5为本申请提供的一种文件访问方法示意图;
图6为本申请提供的一种文件读取方法示意图;
图7为本申请提供的一种数据报文头的结构示意图;
图8为本申请提供的一种文件写入方法示意图;
图9为本申请提供的一种文件追加写入方法示意图;
图10为本申请提供的一种数据报文头的结构示意图;
图11为本申请提供的一种文件替换写入方法示意图;
图12为本申请提供的一种数据报文头的结构示意图;
图13为本申请提供的一种文件访问装置的结构示意图;
图14为本申请提供的一种文件注册装置的结构示意图。
具体实施方式
本申请实施例涉及文件读取,为了便于理解本申请实施例,首先,对本申请涉及的部分术语进行解释说明。
1、文件
文件是呈现数据的一种形式。本申请实施例并不限定文件的具体类型,例如该文件可以是视频文件、语音文件或文本文件。文件中可以包括相同类型的数据,如文件中的数据均为语音数据或图片。文件中的数据也可以包括不同类型的数据,如文件可以为计算设备 之间传输的通信文件,该文件中可以包括图片、文本等数据。
文件可以被存储在不同存储器中。存储文件时,除了需要存储文件中的数据,还需要存储文件的相关信息,文件的相关信息用于对文件进行描述。
文件的相关信息包括下列的部分或全部:文件索引(index)、文件信息(file information)、文件日志(file log)、文件标识。
2、文件标识
文件标识用于唯一地标识某一文件。在本申请实施例中,客户端(如客户端的网卡)向服务端(服务端的网卡)发起携带有文件标识的文件读取请求或文件写入请求,指示服务端从文件中读取数据或在文件中写入数据。在本申请实施例中该文件标识是由服务端提供给客户端的(如在客户端向服务端发起文件打开请求的场景下,服务端的处理器或网卡为该文件分配一个文件标识,然后网卡提供给客户端)。
在本申请实施例中并不限定文件标识的类型,例如该文件标识可以是服务端的网卡执行内存注册时,为该文件分配的远端键remote key,也可以服务端的网卡或处理器为文件分配的编号等信息。凡是能够唯一标识文件的信息均适应于本申请实施例。
3、文件索引(index)
文件索引用于指示文件中的数据在服务端内存中的存储位置,在本申请实施例中文件索引可以指示该文件中的数据在文件的数据存储区域中的位置。该文件的数据存储区域位于服务端的内存中。通过解析该文件索引可以确定该文件中的数据的存储位置,如可以确定文件中的数据存储的物理地址。
4、文件信息(file information)
文件信息为文件的属性信息,该文件信息可以包括文件大小以及描述文件被访问的相关信息,如文件被访问的时间、文件被访问的数据大小以及数据的偏移量等。
5、文件日志(file log)
文件日志为文件恢复(如文件中的数据恢复)时所需要的信息,文件日志能够在一定程度上保证文件存储的可靠性。在不同的应用场景中文件日志中记录的内容不同。
6、远程直接内存访问(remote direct memory access,RDMA)
RDMA是一种直接内存访问方法,基于RDMA,可以使一个计算设备直接对其他计算设备的内存进行数据读写,过程中不需要经过处理器的处理。
7、文件注册
在本申请实施例中服务端的处理器可以将文件注册给服务端的网卡,也即服务端的处理器可以将文件的存储区域通知给服务端的网卡,这样网卡可以获取文件的存储区域。文件的存储区域为用于存储文件中的数据以及文件的相关信息的存储区域。
在本申请实施例中,文件的存储区域可以包括下列的部分或全部:文件的索引存储区域、文件的信息存储区域、文件的日志存储区域、文件的数据存储区域。
文件的索引存储区域为服务端内存中存储文件索引的区域,文件的信息存储区域为服务端内存中存储文件信息的区域,文件的日志存储区域为服务端内存中存储文件日志的区域。文件的数据存储区域为服务端内存中存储文件中的数据的区域。
需要说明的是,在本申请实施例中文件的数据存储区域为服务端的处理器为文件分配的、用于存储数据的存储区域。该文件的数据存储区域中可以包括已存储了数据的区域和 空白区域,该空白区域可以用于后续在对该文件进行写入时,存储需要新写入的数据。文件索引指示的是当前文件中的数据的存储位置。每次在文件中写入数据之后,文件索引需要更新以使得该文件索引能指示在写入数据后,该文件中的数据的存储位置。
8、内存注册(memory registration)
RDMA可以直接对内存进行数据读写。而对内存进行数据读写的前提是需要实现内存注册,内存注册可以理解为获取该内存的控制权,以实现对内存的数据读写。
memory registration是RDMA中对内存的一种保护措施,只有将内存注册到RDMA内存域(memory region,MR)中,这块内存就可以由RDMA来操作了。在完成内存注册之后,可以对该内存进行数据读写。
在本申请实施例中,服务端的处理器可以将文件注册到服务端的网卡(该网卡具备RDMA引擎),即服务端的处理器可以将文件的存储区域通知给服务端的网卡,网卡在获知文件的存储区域后,可以针对该文件的存储区域执行内存注册操作,将文件的存储区域注册到RDMA内存域中。
网卡在执行内存注册操作后,能够对文件中的数据、文件索引、文件信息以及文件的日志进行操作,便于后续对文件进行读取或写入,例如,读取文件中的数据,或在文件中写入数据,对文件索引、文件信息以及文件日志进行更新等。
9、队列(queues)
RDMA一共支持三种队列,发送队列(send queues,SQ)和接收队列(receive queues,RQ),完成队列(complete queues,CQ)。其中,SQ和RQ通常成对创建,被称为队列对(queue pairs,QP)。
RDMA是基于消息的传输协议,数据传输为异步操作。RDMA的操作过程如下:
(1)、主机(host)的处理器提交工作请求(work request,WR)到网卡,网卡将该工作请求配置到工作队列(work queues,WQ),工作队列包括发送队列(SQ)和接收队列(RQ)。工作队列的每一个元素叫做(work queues element,WQE),一个元素也就对应一个WR。
(2)、host的处理器可以通过网卡从完成队列(CQ)中获取工作完成(work complete,WC),完成队列里的每一个元素叫做(complete queues element,CQE),一个元素也就对应一个WC。
其中,具有RDMA引擎的硬件(hardware),如网卡,可以看做为一个队列元素处理模块。该硬件不断地从工作队列(WQ)中去取工作请求(WR)来执行,执行完了就给完成队列(CQ)中放置工作完成(WC)。
10、RDMA数据传输
在本申请实施例中仅以RDMA发送/接收操作(send/receive)为例进行说明。RDMA是基于消息的数据传输协议的,所有数据包的组装都在具有RDMA引擎的硬件上完成的。
RDMA中发送/接收操作是双边操作,即必须要远端的应用感知参与才能完成收发。
以主机A向主机B(下面简称A、B)发送数据为例:
首先,A和B都要创建并初始化好各自的QP、CQ。
A和B分别向各自的WQ中添加WQE,对于A,WQ=SQ,WQE描述指向等待被发送的数据;对于B,WQ=RQ,WQE描述指向用于存储数据的缓存(buffer)。
A的网卡异步调度轮到A的WQE,解析到这是一个发送(send)消息,直接向B发出 WQE指向的数据。数据流到达B的网卡后,B的WQE被消耗,B的网卡把数据直接存储到WQE指向的缓存。
A与B通信完成后,A的CQ中会产生一个完成消息CQE表示发送完成。与此同时,B的CQ中也会产生一个完成消息CQE表示接收完成。每个WQ中WQE的处理完成都会产生一个CQE。
如图1所示为本申请实施例提供的一种系统架构示意图,如图1所示,该系统中包括服务端100和客户端200。
客户端200部署在用户侧,用于在用户的触发下向服务端100发起请求。本申请实施例并不限定客户端200的具体形态,例如客户端200可为用户的本地计算设备或专用客户端。
客户端200上可以部署有文件系统客户端(file system client,FS client),文件系统客户端可以是客户端200上运行的软件。用户可以通过设置在本地计算设备中的FS client访问服务端100中存储的文件,指示服务端100(如服务端100的网卡)对服务端100中存储的文件进行读取、写入(在本申请实施例中写包括追加写和替换写)等操作。
客户端200上还可以部署有网卡210,当用户通过FS client访问服务端100时,FS client可以触发网卡210生成文件访问请求(如文件读取请求、文件写入请求),并通过网卡将文件访问请求发送给服务端100。
在本申请实施例中服务端100能够存储有用户的文件。该服务端100实质上为存储节点,具备文件存储功能。本申请实施例并不限定服务端100的部署位置,例如服务端100可以部署在边缘数据中心,也可以部署在云数据中心中,也可以部署在终端计算设备上。服务端100也可以分布式的部署在边缘数据中心、云数据中心以及终端计算设备中的部分或全部环境中。
服务端100上运行有文件系统(file system,FS),该文件系统可以是运行在服务端100的处理器120上的软件。该文件系统对服务端100上存储的文件进行管理,能够在服务端100的内存130中为文件分配存储区域,还可以将文件的存储区域通知给网卡140,也即将文件注册给网卡140,触发网卡140执行内存注册操作。该网卡140在获知了该文件的存储区域后,可以对文件进行读取和写入,当网卡140需要文件读取或文件写入时,网卡140可以创建文件句柄(file handle),通过对该file handle的操作,实现文件的读取和写入。
在本申请实施例中,客户端200与服务端100交互可以实现对文件的读取、写入等操作。客户端200与服务端100之间进行交互时,为了减少客户端200与服务端100之间的交互次数,简化交互流程。在本申请实施例中,服务端100的网卡140能够从服务端100的处理器120处理器获取文件的存储区域,执行内存注册操作,还可以为文件分配文件标识。在此基础上,服务端100的网卡140能够根据文件标识直接确定文件的存储区域(如文件的索引存储区域)。当客户端200可以向服务端100发送携带有文件标识的文件访问请求后,服务端100(如服务端100的网卡)可以根据文件访问请求携带的文件标识确定文件的存储区域,根据文件访问请求进行文件读取或文件写入,还可以在文件读取和文件写入之后,向客户端200发送访问响应(如文件读取响应、文件写入响应)。
具体到服务端100内部,服务端100对文件的访问操作可以不需要服务端100的处理 器执行,而交由服务端100的网卡140执行。
如图2所示,为服务端100的结构示意图,服务端100包括总线110、处理器120、内存130以及网卡140。处理器120、内存130、网卡140之间通过总线110通信。
其中,处理器120可以为中央处理器(central processing unit,CPU)、专用集成电路(application specific integrated circuit,ASIC)、现场可编程门阵列(field programmable gate array,FPGA)、人工智能(artificial intelligence,AI)芯片、片上系统(system on chip,SoC)或复杂可编程逻辑器件(complex programmable logic device,CPLD),图形处理器(graphics processing unit,GPU)等。
内存130可以包括易失性存储器(volatile memory),例如RAM、DRAM等。也可以为非易失性存储器(non-volatile memory),例如SCM等,或者易失性存储器与非易失性存储器的组合等。
内存130中还可以包括操作系统等其他运行进程所需的软件模块。操作系统可以为LINUX TM,UNIX TM,WINDOWS TM等。
处理器120可以通过调用内存130中存储的计算机执行指令,使得服务端100可以执行如图3所示的实施例中服务端100的处理器120所执行的操作。
尽管未示出,该服务端100中还可以包括辅助存储器,也可以称为外存,该外存可以为非易失性存储器(non-volatile memory),例如只读存储器(read-only memory,ROM),硬盘驱动器(hard disk drive,HDD)或固态驱动器(solid state disk,SSD)等。这外存可以用于永久的存储文件。
在本申请实施例中,处理器120可以将文件从外存调入到内存130中,处理器120可以在内存130中为该文件分配存储区域,所分配的存储区域可以用于存储该文件。当需要打开该文件时,处理器120可以确定内存130中该文件的存储区域,并将该文件的存储区域通知给网卡140。若需要打开的文件未预先调入到该内存130中,处理器120在接收到文件打开请求时,可以在内存130中为该文件分配存储区域,以存储该文件,并为该文件分配的存储区域通知给网卡140。
当需要关闭文件时,处理器120可以注销该文件的存储区域,删除内存120中存储的文件。
在本申请实施例中网卡140除了能够进行数据传输(如接收文件打开/关闭请求、文件访问请求,发送文件打开/关闭响应、文件访问响应),还能够对文件进行读取和写入。
网卡140包括处理器141以及接口142,处理器141的具体形态可以参见处理器120以及的相关说明,此处不再赘述。处理器141使得服务端100中的网卡140可以执行如图3、图5、图8、图9、图11所示的实施例中服务端100的网卡140所执行的操作,处理器141还可以通过接口142发送或接收信息,如接收文件打开请求、文件访问请求、文件关闭请求等,或发送文件打开响应、文件访问响应、文件关闭响应。
传统的文件读写过程中,文件未注册到网卡上,网卡不能直接对文件进行读取和写入,因此在对文件进行读写时,客户端需要先从服务端获取文件索引,之后客户端根据文件索引确定文件中的数据的存储位置,服务端侧的网卡在客户端的指示下(如客户端发起RDMA单边操作)从客户端指示的存储位置中读取或写入数据,这种方式需要客户端和服务端之间进行多次交互。或者在对文件进行读写时,服务端侧的处理器需要在客户端的指示下中 获取文件数据,并通过网卡发送文件数据,在这种方式中需要处理器的参与,效率较差。
而在本申请实施例中,在客户端200与服务端100之间交互,对服务端100中存储的文件进行读取或写入前,服务端100中的处理器120可以先将文件注册到网卡140中,通知网卡140该文件的存储区域,触发网卡140执行内存注册操作。服务端100的网卡140为文件分配文件标识,并将文件标识反馈给客户端200。
服务端100的网卡140在从客户端200接收到携带有文件标识的文件访问请求之后,根据文件标识访问该文件,对文件进行读取或写入。采用本申请实施例的方式,在对文件进行访问时,不需要客户端200和服务端100多次交互,也不需要服务端100的处理器120参与,能够有效提高文件访问效率。
本申请实施例提供的文件访问方法可以分为二个流程,分别为文件注册和内存注册流程、文件访问流程,其中文件访问流程又可以分为文件读取流程和文件写入流程。下面分别进行说明。
一、文件注册和内存注册流程
在文件注册和内存注册流程中,服务端100的处理器120可以告知服务端100的网卡140中文件的存储区域,并触发网卡140执行内存注册操作,促使网卡140为文件分配文件标识。这样当客户端200想要对文件进行读或写等操作时,可以在请求中携带该文件标识,以标识需要读或写的文件。由于网卡140已经获取了文件的存储区域且执行了内存注册操作,网卡140就可以根据文件标识从文件的存储区域中读取数据或写入数据。
参见图3,以服务端100中的网卡140为文件分配文件标识为例,对本申请实施例提供的文件注册方法进行说明,该方法包括:
步骤301:用户通过客户端200向服务端100发起文件打开请求。
用户可以在客户端200提供的界面上,点击文件;客户端200在检测到用户的操作后,确定用户需要打开该文件,向服务端100发起文件打开请求,该文件打开请求中携带文件的名称,文件的名称可以是用户为该文件分配的,文件的名称在服务端100存储该文件时,存储在服务端100。
在步骤301中,客户端200中的网卡210可以基于RDMA向服务端100中的网卡140发送文件打开请求,也即将文件打开请求作为WQ中的WQE添加在客户端200侧的SQ中,该WQE指示需要打开该文件。
步骤302:服务端100中的网卡140在接收到该文件打开请求后请求,将该文件打开请求发送给服务端100的处理器120,服务端100中的处理器120打开文件。
服务端100中的网卡140可以从服务端100侧的RQ中获取该文件打开请求,将该文件打开请求发送给服务端100中的处理器120,服务端100中的处理器120可以根据该文件打开请求打开该文件,确定文件的存储区域。
需要说明的是,由于文件中的数据以及文件日志会随之文件中的数据的写入,数据量会变大,文件的数据存储区域可以是根据文件所能容纳的数据量按需分配的。随着数据的写入,服务端100中的处理器120也可以扩展该文件的数据存储区域。服务端100中的处理器120在扩展了文件的数据存储区域的情况下,还可以在将扩展的该文件的数据存储区域通知给网卡140。文件的日志存储区域也类似。
需要说明的是,若用户需要打开的文件是未在服务端100的内存130中存储的文件, 用户也可以通过客户端200向服务端100发起文件打开请求。在这种场景下,服务端100中的处理器120根据文件打开请求中携带的文件的名称确定内存130中当前未存储该文件,服务端100中的处理器120可以在内存130中为该文件分配存储区域。
步骤303:服务端100中的处理器120将文件注册到服务端100中的网卡140,通知网卡140该文件的存储区域。
服务端100中的处理器120在确定或分配了文件的存储区域后,可以将文件注册到该服务端100中的网卡140中,也即服务端100中的处理器120可以将该文件的存储区域通知给网卡140,使得网卡140获取该文件的存储区域。具体的,服务端100中的处理器120可以将用于描述文件的存储区域的信息(这里简称为文件的存储区域的信息)发送给网卡140,并触发网卡140执行内存注册操作,为文件分配该文件标识,网卡140可以保存文件的存储区域的信息和该文件标识。
由前述说明可知,文件的存储区域包括文件的索引存储区域、文件的数据存储区域、文件的信息存储区域、文件的日志存储区域。
文件的存储区域的信息包括文件的索引存储区域的信息、文件的数据存储区域的信息、文件的信息存储区域的信息、文件的日志存储区域的信息。
以文件的索引存储区域的信息为例进行说明,文件的索引存储区域的信息用于描述文件的索引存储区域,描述文件的索引存储区域的方式有很多种,相应的,文件的索引存储区域的信息也有很多种。例如,文件的索引存储区域的信息可以包括文件索引的起始存储地址和文件索引的终止存储地址。文件的索引存储区域的信息可以包括文件索引的起始存储地址和文件索引的大小。
步骤304:服务端100中的网卡140获取了文件的存储区域之后,服务端100中的网卡140针对该文件的存储空间执行内存注册操作,为文件分配文件标识。
服务端100中的网卡140针对该文件的存储空间执行内存注册操作时,可以将该文件的存储区域加入到RDMA的内存域中,保证该网卡140可以对该文件的存储区域进行操作,如读取该文件中存储区域中文件索引、文件中的数据,更新文件信息或文件日志等;又如,在该文件中存储区域中写入数据,更新文件索引、文件信息或文件日志等。
服务端100中的网卡140还可以记录文件的存储空间的信息和文件标识。
举例来说,服务端100中的网卡140保存的文件的存储区域的信息和该文件标识可以为如图4所示的数据表。该数据表中示例性的展示了文件的索引存储区域的信息、文件的日志存储区域的信息、文件的信息存储区域和文件标识。
其中,文件信息的存储区域的信息可以为文件信息地址。
该文件的索引存储区域的信息包括索引地址(index address)、索引长度(index size)。其中,索引地址用于指示文件索引的起始存储地址。索引长度用于指示文件索引的长度。
文件的日志存储区域的信息包括日志键(log mkey)、日志地址、日志长度(log size)、或日志偏移量中部分或全部指示。日志长度用以指示日志的长度。
根据文件索引的作用可知,文件索引能够指示文件的数据存储区域中文件中的数据的存储位置,服务端100中的处理器120在将文件注册给服务端100中的网卡140后,网卡140获知了文件中的数据存储区域、文件的日志存储区域、文件的信息存储区域以及文件的索引存储区域。网卡140可以根据如图4所示的数据表,确定文件标识所对应的文件索引, 根据文件索引确定文件中的数据。
步骤305:服务端100中的处理器120通过服务端100中的网卡140向客户端200发送文件打开响应,该文件打开响应中携带有文件标识。
服务端100中的网卡140可以基于RDMA向客户端200发送文件打开响应,也即将文件打开响应作为WQ中的WQE添加在服务端100侧的SQ中,该WQE指示文件打开。
客户端200中的网卡210可以从客户端200侧的RQ中获取该文件打开响应,从该文件打开响应中获取文件标识。
通过步骤301~步骤305,客户端200可以通过发送文件打开请求,触发服务端100侧将文件注册到网卡140中。
相应的,用户在通过客户端200关闭文件时,也可以通过客户端200触发服务端100在网卡140中注销该文件。具体可参见步骤306~步骤309。
步骤306:用户通过客户端200向服务端100发起文件关闭请求。
用户可以在客户端200提供的界面上,选择“文件关闭”选项;客户端200在检测到用户的操作后,确定用户需要关闭该文件,向服务端100发起文件关闭请求,该文件打开请求中携带文件标识。
客户端200的网卡210可以基于RDMA向服务端100发送文件关闭请求,客户端200基于RDMA向服务端100发送文件关闭请求与客户端200的网卡140基于RDMA向服务端100发送文打开请求的方式类似,具体可以参见前述说明,此处不再赘述。
步骤307:服务端100的处理器120在接收到该文件关闭请求后,服务端100中的处理器120关闭文件,如处理器120可以释放文件占用的资源。
步骤308:服务端100中的处理器120从该服务端100中的网卡140中注销该文件。
服务端100中的处理器120可以从服务端100中的网卡140中注销该文件的存储区域,如通知网卡140删除该文件的日志存储区域、文件的信息存储区域、文件的数据存储区域的信息和该文件标识。
步骤309:服务端100中的处理器120在关闭了文件之后,可以通过服务端100中的网卡140向客户端200发送文件关闭响应。
服务端100中的网卡140可以基于RDMA向客户端200端发送文件关闭响应,服务端100中的网卡140基于RDMA向客户端200端发送文件关闭响应的方式与服务端100中的网卡140基于RDMA向客户端200发送文件打开响应的方式类似,具体可以参见前述内容,此处不再赘述。
需要说明的是,在图3中仅是以在客户端200发起文件打开请求的场景下,服务端100中的处理器120将文件注册给服务端100中的网卡140为例进行说明的,在实际应用中本申请实施例并不限定服务端100的处理器120将文件注册给服务端100中的网卡140的场景,例如,可以是在文件存储到服务端100时,服务端100中的处理器120将文件注册给服务器的网卡140的。
二、文件访问流程
如图5所示,为本申请实施例提供的一种文件访问方法,该方法包括:
步骤501:用户通过客户端200向服务端100发起文件访问请求。该文件访问请求用于请求访问文件,该文件访问请求中包括文件标识。
用户可以通过客户端200部署的文件系统客户端触发客户端200的网卡210发起文件访问请求,文件系统客户端可以将携带文件标识的文件访问请求发送给客户端200的网卡210,指示网卡210向该服务端100发起该文件访问请求。采用这种方式客户端200的网卡210可以不感知该文件访问请求中的信息。
这里文件访问请求可以用于请求读取文件中的第一数据,也即该文件访问请求为文件读取请求。文件访问请求也可以用于请求在文件中写入第二数据,也即该文件访问请求为文件写入请求。
步骤502:服务端100的网卡140接收来自第二设备的文件访问请求,根据文件标识访问文件。
服务端100的网卡140在接收到该文件访问请求后,可以从该文件访问请求中解析出该文件标识,根据该文件标识确定文件索引,进而确定该文件中数据的存储位置。
若该文件访问请求为文件读取请求,服务端100的网卡140可以根据文件读取请求从文件中读取第一数据。
若该文件访问请求为文件写入请求,服务端100的网卡140可以根据文件写入请求在文件中写入第二数据。
步骤503:服务端100的网卡140向客户端200发送文件访问响应。
若该文件访问请求为文件读取请求,文件访问响应为文件读取响应,该文件读取响应中包括第一数据。
若该文件访问请求为文件写入请求,文件访问响应为文件写入响应,文件写入响应指示成功写入第二数据。
由上可知文件访问流程分为文件读取流程和文件写入流程,下面分别对文件读取流程和文件写入流程进行说明。
(一)文件读取流程
在将文件注册给服务端100中的网卡140后,用户可以通过客户端200从服务端100读取该文件中的数据。
如图6所示,为本申请实施例提供的一种文件读取方法,该方法包括:
步骤601:用户通过客户端200(如客户端200的网卡210)向服务端100发起文件读取请求。该文件读取请求用于请求读取文件中的第一数据。该第一数据可以为文件中的全部数据,也可以为文件中的部分数据。
用户可以在客户端200提供的界面上,从文件中选择需要读取的第一数据,如选择文件中的数据,如点击“读取”的选项;客户端200在检测到用户的操作后,确定用户需要读取文件的第一数据,向服务端100发起文件读取请求,该文件读取请求中携带文件标识和第一数据的相关信息。该第一数据的相关信息可以包括为第一数据的数据长度、该第一数据在该文件中的数据中的偏移量(如指示第一数据的第一个比特与文件中的数据中第一个比特的偏移量)等。
在本申请实施例中,文件读取请求的数据报文头中可以携带第一数据的相关信息,还可以指示该文件读取请求的类型。
举例来说,如图7所示,为本申请实施例提供的文件读取请求的数据报文头的格式,如图7所示,该数据报文头包括文件操作码(operation code,OP)、文件标识、第一数据的 数据长度(length)、该第一数据在该文件中的数据中的偏移量(offset)。在文件读取请求中OP可以为读(read),用于指示该请求为文件读取请求,用于请求读取文件中的第一数据。
可选的,该数据报文头中还可以包括页面索引(page index),该页面索引用于指示服务端100侧存储第一数据的页面(该页面是文件的数据存储区域中的页面)。该页面索引可以是客户端200根据offset与页面的大小计算确定的。通常页面大小是固定的,如均为4千字节(kb),利用offset与页面大小的比值,可以确定该第一数据的页面,进而确定页面索引。
在一些场景中,该数据报文头中也可以不携带page index,服务端100中的网卡140可以根据offset与页面的大小计算出该page index。图7中,OP和offset可以占用0~3字节,page index可以占用4~7字节,文件标识可以占用8~11字节,length可以占用12~14字节。本申请实施例中并不限定数据报文头中各个信息所占用的字节数以及各个信息在数据报文头中的位置,如图7所示仅为举例。
需要说明的是,文件读取请求中也可以不携带第一数据的相关信息,例如,当需要读取文件中所有的数据,文件读取请求可以携带指示符,该指示符可以为客户端200和服务端100预先预定的字符,该指示符用于指示需读取文件中的所有数据。
步骤602:服务端100中的网卡140在接收到该文件读取请求后,获取该第一数据。当文件读取请求达到服务端100时,首先是服务端100的网卡140接收到该文件读取请求。在步骤602中,服务端100的网卡140可以直接对该文件读取请求进行处理,解析该文件读取请求。
在获取第一数据时,服务端100中的网卡140可以根据文件标识确定文件的索引存储区域,从中获取文件索引。服务端100中的网卡140在获取该文件索引后,可以根据该文件索引确定文件中的数据的存储位置,然后根据第一数据的相关信息,如第一数据的长度以及第一数据在文件中所有数据的偏移,从该文件中的数据的存储地址中读取该第一数据。
需要说明的是,服务端100中的网卡140在获取第一数据之前,还可以根据文件标识对客户端200进行有效性验证。服务端100中的网卡140在针对该文件执行内存注册时,会针对该文件生成远端键(remote key)和本地键(local key),remote key可以作为文件标识发送给客户端200,local key可以保存在本地。
服务端100中的网卡140在接收到文件读取请求后,可以对文件标识和本地存储的local key进行比对,在一致的情况下,对客户端200有效性验证通过,否则验证失败,服务端100中的网卡140可以直接拒绝该文件读取请求,也可以将该文件读取请求传输给服务端100中的处理器120,由服务端100中的处理器120对该文件读取请求进行处理。
步骤603:服务端100中的网卡140在获取第一数据之后,可以向客户端200发送文件读取响应,该文件读取响应中携带有文件的第一数据。在本申请实施例中文件读取响应的格式可以符合RDAM中定义的文件读取响应(read response)的格式。
(二)文件写入流程
用户还可以通过客户端200在服务端100中存储的该文件中增加新的数据或通过客户端200替换服务端100中存储的该文件中的数据,也即文件写入流程。
如图8所示,为本申请实施例提供的一种文件写入方法,该方法包括:
步骤801:用户通过客户端200向服务端100发起文件写入请求,该文件写入请求用于请求在文件中写入第二数据,该文件写入请求中包括文件标识和第二数据。用户可以通过 客户端200的网卡210发送该文件写入请求。
在本申请实施例中在文件中写入第二数据的方式可以为在文件中的数据中追加写入第二数据,也可以为将文件中的部分或全部数据替换为第二数据。
步骤802:服务端100中的网卡140在接收到该文件写入请求后,将第二数据写入到文件中。当文件写入请求达到服务端100时,首先是服务端100的网卡140接收到该文件写入请求。在步骤802中,服务端100的网卡140可以直接对该文件写入请求进行处理,也即可以解析该文件写入请求。
当文件写入请求用于请求在文件中追加写入第二数据,服务端100中的网卡140从该文件的数据存储区域中申请新的存储区域,可以将第二数据写入到申请到的存储区域中。
当文件写入请求用于请求将文件中的部分或全部数据替换为第二数据,服务端100中的网卡140可以将文件中的数据的该存储区域中的部分或全部数据替换为该第二数据(这种方式为替换写中的replace),服务端100中的网卡140也可以从该文件的数据存储区域申请存储区域,可以为第二数据写入到申请到的存储区域中(这种方式为替换写中的rewrite场景)。
步骤803:服务端100中的网卡140更新文件索引,使得文件索引能够指示写入第二数据后的文件中的数据的存储位置。
当文件写入请求用于请求在文件中追加写入第二数据时,更新文件索引,使得该文件索引能够指示写入第二数据后文件中数据的存储位置。
当文件写入请求用于请求将文件中的部分或全部数据替换为第二数据时,在rewrite场景下,由于文件的数据存储区域中文件中的数据的存储位置发生改变,服务端100的网卡140需要更新该文件索引。而在replace场景下,由于文件的数据存储区域中文件中数据的存储位置并未改变,服务端100的网卡140可以不更新该文件索引,也即不需要执行步骤803。
当文件写入请求用于请求在文件中追加写入第二数据时,还可以更新该文件日志,该文件日志可以记录追加写入的第二数据、以及第二数据的存储位置。服务端100中的网卡140还可以更新文件信息,在文件信息中记录写入第二数据的时间等信息。
当文件写入请求用于请求将文件中的部分或全部数据替换为第二数据时,还可以更新该文件日志,该文件日志可以记录写入的第二数据,以及第二数据的存储位置。服务端100中的网卡140还可以更新文件信息,在文件信息中记录写入第二数据的时间等信息。
需要说明的是,服务端100中的网卡140将该第二数据写入到该存储区域中之前,还可以根据文件标识对客户端200进行有效性验证。服务端100中的网卡140根据文件标识对客户端200进行有效性验证的方式可以参见前述说明,此处不再赘述。
步骤804:服务端100中的网卡140向客户端200的网卡140发送文件写入响应,通知客户端200第二数据已成功写入。在本申请实施例中文件写入响应的格式可以符合RDAM中定义的文件写入响应(write response)的格式。
由上可知,在本申请实施例中,文件写入流程分为追加写以及替换写,下面分别对这两种文件写入流程进行说明。为了区别这两种流程中发送的文件写入请求,将追加写流程中的文件写入请求称为第一文件写入请求,相应的文件写入响应称为第一文件写入响应。 将替换写流程中的文件写入请求称为第二文件写入请求,相应的文件写入响应称为第二文件写入响应。
1)、追加写
如图9所示,为本申请实施例提供的一种文件追加写的流程示意图,该方法包括:
步骤901:用户通过客户端200的网卡210向服务端100中的网卡140发起第一文件写入请求。该第一文件写入请求用于请求在文件中追加写入第二数据,该第二数据可以为文件中需要新增的第二数据。
用户可以在客户端200提供的界面上,输入或选择文件中需要增加的第二数据,如输入或选择文件中需要新增的第二数据;客户端200在检测到用户的操作后,确定用户需要在文件中写入第二数据,向服务端100发起第一文件写入请求,该第一文件写入请求中携带文件标识和第二数据。可选的,还可以携带第二数据的相关信息,例如该第二数据的数据长度等。
在本申请实施例中,第一文件写入请求的数据报文体中可以携带第二数据,第一文件写入请求的数据报文头中可以携带第二数据的相关信息(如第二数据的数据长度),还可以指示该文件写入请求的类型。
举例来说,如图10所示,为本申请实施例提供的第一文件写入请求格式,如图10中,第二文件写入请求的数据报文体中可以携带的第二数据,该数据报文头包括文件操作类型(operation code,OP)、文件标识。在第一文件写入请求中OP可以为追加(append),用于指示该请求为文件写入请求,用于请求在文件中追加写入第二数据。
该数据报文头中还可以包括第二数据的数据长度(length)、该第二数据在该文件全部数据中的偏移量(offset)、页面索引(page index),由于为需要在文件中追加写入第二数据,offset可以等于当前文件中所有数据的数据长度,该页面索引的说明可以参见前述内容,此处不再赘述。本申请实施例中并不限定数据报文头中各个信息所占用的字节数以及各个信息在该数据报文头中的位置,如图10所示仅为举例。
步骤902:服务端100中的网卡140在接收到该第一文件写入请求后,服务端100中的网卡140可以从文件的数据存储区域中为第二数据申请存储区域,将该第二数据写入到申请到的存储区域中。
由于该第一文件写入请求请求在文件中追加写入第二数据,也即该第二数据需要位于该文件中已存储的数据之后,且紧邻该文件中已存储的数据。服务端100的网卡140为第二数据申请存储区域,该存储区域的大小等于该第二数据的数据长度。也就是说,申请的存储区域的起始位置(这里的起始位置指示的是虚拟地址)为该文件中已存储的数据的终止存储位置,也即申请的存储区域的起始位置指向文件的末尾。
步骤903:服务端100中的网卡140可以更新文件索引,更新后的文件索引可以指示该第二数据的存储位置。
服务端100中的网卡140根据如图4所示的数据表,确定文件标识所对应的文件索引,进而根据文件索引确定文件的索引存储区域,读取该文件索引,更新该文件索引。
除了追加写入该第二数据、更新文件索引,服务端100中的网卡140还可以更新文件信息以及文件日志。文件信息以及文件日志的更新方式可以参见图7所示的实施例中的相关说明此处不再赘述。
步骤904:服务端100中的网卡140在写入第二数据之后,可以向客户端200中的网卡210发送第一文件写入响应,通过客户端200第二数据已成功写入。
2)、替换写
如图11所示,为本申请实施例提供的一种文件替换写的流程示意图,该方法包括:
步骤1101:用户通过客户端200的网卡210向服务端100中的网卡140发起第二文件写入请求。该第二文件写入请求用于请求将文件中的数据替换为第二数据,该第二数据可以为文件中需要替换写入的数据。
用户可以在客户端200提供的界面上,输入或选择文件中需要写入的第二数据,如输入或选择文件中需要替换的第二数据;客户端200在检测到用户的操作后,确定用户需要在文件中写入第二数据,向服务端100发起第二文件写入请求,该第二文件写入请求中携带文件标识和第二数据。可选的,还可以携带第二数据的相关信息,例如该第二数据的数据长度、第二数据在文件中的数据的偏移量等。
在本申请实施例中,第二文件写入请求的数据报文体中可以携带第二数据,第二文件写入请求的数据报文头中可以携带第二数据的相关信息(如第二数据的数据长度、第二数据在文件中的数据的偏移量),还可以指示该文件写入请求的类型。
举例来说,如图12所示,为本申请实施例提供的第二文件写入请求格式,如图12所示,第二文件写入请求的数据报文体中可以携带的第二数据,该数据报文头包括文件操作类型(operation code,OP)、文件标识、第二数据的数据长度(length)、该第二数据在该文件全部数据中的偏移量(offset)。在第二文件写入请求中OP可以为替换(replace),这种场景下,需要服务端100中的网卡140将文件中的数据覆盖,写入第二数据。在第二文件写入请求中OP可以为重写(rewrite),这种场景下,服务端100中的网卡140可以保留文件中已有的数据,申请新的存储区域写入第二数据。OP为replace或rewrite,均可以指示该请求为文件写入请求,仅是第二数据写入的方式不同。
可选的,该数据报文头中还可以包括页面索引(page index),该页面索引的说明可以参见前述内容,此处不再赘述。本申请实施例中并不限定数据报文头中各个信息所占用的字节数以及各个信息在数据报文头中的位置,如图12所示仅为举例。
步骤1102:服务端100中的网卡140在接收到该第二文件写入请求后,确定第二数据的存储区域,在该第二数据的存储区域写入第二数据。
第二数据的存储区域可以为文件的数据存储区域中已存储的数据的空间(对应OP为replace的场景),也可以是服务端100中的网卡140在文件的数据存储区域中为第二数据申请的存储区域(对应OP为rewrite的场景)。
OP为replace时,服务端100中的网卡140根据第二数据的相关信息,如第二数据的数据长度、第二数据在文件中的数据的偏移量,确定该文件的数据存储区域中需要被替换的数据的存储区域,该存储区域即为第二数据的存储区域,在该存储区域上写入第二数据。
OP为rewrite时,服务端100中的网卡140也可以不覆盖该文件中已有的数据,也即保留该文件中已有的数据,服务端100中的网卡140可以在文件的数据存储区域(空白区域)中为第二数据申请存储区域,该存储区域的大小可以等于第二数据的数据大小,将第二数据写入到申请到的存储区域中。与追加写方式不同,这里申请的存储区域可以不紧邻该文件的数据存储区域中已存储数据的区域,也即该申请到的存储区域的起始位置可以不指向 该文件的末尾。
当OP为replace,服务端100的网卡140可以不更新文件索引,也即不需要执行步骤1103,当OP为rewrite,服务端100的网卡140需要更新文件索引,也即需要执行步骤1103。
步骤1103:服务端100中的网卡140根据文件标识确定文件索引,更新文件索引。
服务端100的网卡140可以根据文件标识确定文件的索引存储区域,从文件的索引存储区域中读取该文件索引,根据该第二数据的数据长度以及第二数据在文件中的数据的偏移量确定该文件索引中用于指示需要替换的数据的存储位置的部分,修改该部分,使得该部分能够指示第二数据的存储位置。
除了写入该第二数据、更新文件索引,服务端100中的网卡140还可以更新该文件的文件信息以及文件日志。该文件的文件信息以及文件日志的更新方式可以参见图8所示的实施例中的相关说明此处不再赘述。
步骤1104:服务端100中的网卡140在写入第二数据之后,可以向客户端200的网卡210发送第二文件写入响应,通知客户端200第二数据已成功写入。
需要说明的是,在上述实施例中是以文件标识为网卡140分配的,且该文件标识可以用于对客户端200进行有效性验证为例进行说明的。在实际应用中,该文件标识也可以是服务端100中的处理器120为文件分配的,在服务端100中的处理器120在将该文件注册给网卡140时,也可以将该文件标识通知给服务端100中的网卡140。这种情况下,服务端100中的网卡140在执行内存注册时,为文件分配的remote key可以不作为文件标识,而是仅用于客户端200进行有效性验证,该文件标识以及remote key携带在文件打开响应中。在另一种可能场景中,该文件标识也可以是服务端100中的网卡140为文件分配的,且该文件标识并非是remote key。也就是说,服务端100中的网卡140可以为文件分配文件标识以及remote key。将该文件标识以及remote key携带在文件打开响应中。
基于与方法实施例同一发明构思,本申请实施例还提供了一种文件访问装置,该文件访问装置可以部署在网卡上,用于执行上述如图3、图5、图8、图9、图11所示的方法实施例中所述网卡140执行的方法,相关特征可参见上述方法实施例,此处不再赘述。如图13所示,所述装置1300包括接收单元1301、处理单元1302以及发送单元1303。
接收单元1301,用于接收来自第二设备的文件访问请求,文件访问请求用于请求访问文件,文件访问请求包括文件的文件标识。
处理单元1302,用于根据文件标识访问第一设备的内存中的文件。
发送单元1303,用于向第二设备发送文件访问响应。
在一种可能的实施方式中,接收单元1301在接收来自第二设备的文件访问请求之前,处理单元1302可以从设备的处理器获取文件的存储区域,针对文件的存储区域执行内存注册操作,为文件分配文件标识;之后,发送单元1303可以向第二设备发送文件标识。
在一种可能的实施方式中,文件的存储区域包括下列的至少一个:
文件的数据存储区域、文件的索引存储区域、文件的信息存储区域和文件的日志存储区域。
在一种可能的实施方式中,文件访问请求用于请求读取文件的第一数据;处理单元1302在根据文件标识访问文件,并向第二设备发送文件访问响应时,可以根据文件标识确定文件的文件索引;之后,根据文件索引中获取第一数据,向第二设备发送文件访问响应,文 件访问响应中包括第一数据。
在一种可能的实施方式中,处理单元1302在根据文件标识确定文件的文件索引时,可以根据文件标识确定文件的索引存储区域,从文件的索引存储区域中读取文件索引。
在一种可能的实施方式中,文件索引用于指示文件中的数据在文件的数据存储区域中的位置,处理单元1302在根据文件索引获取第一数据时,可以根据文件索引从文件的数据存储区域中获取第一数据。
在一种可能的实施方式中,文件访问请求用于请求在文件中写入第二数据,文件访问请求包括第二数据,处理单元1302在根据文件标识访问文件时,可以在文件中写入第二数据;发送单可以向第二设备发送文件访问响应,文件访问响应用于指示第二数据成功写入。
在一种可能的实施方式中,处理单元1302除了写入第二数据,还可以根据文件标识确定文件索引,更新文件索引,更新后的文件索引用以指示第二数据的存储位置。
在一种可能的实施方式中,文件访问请求用于请求在文件中写入第二数据的方式有很多种,下面分别进行说明:
第一种:该文件访问请求在文件中追加写入第二数据。也即写入方式为追加写(append)。
处理单元1302在文件中写入第二数据时,先申请第一存储区域,将第二数据写入到第一存储区域中,第一存储区域位于文件的数据存储区域中,第一存储区域的起始位置指向文件的末尾,也即写入的第二数据应当紧挨着文件中已有的数据存储。
第二种:文件访问请求用于请求将文件的数据替换为第二数据,保留原始数据。也即写入方式为替换(rewrite)。
处理单元1302在文件中写入第二数据时,可以先申请第二存储区域,之后,将第二数据写入到第二存储区域中,第二存储区域位于文件的数据存储区域中。
第三种:文件访问请求用于请求将文件的数据替换为第二数据。也即写入方式为重写(replace)。
文件访问请求还包括第二数据的数据长度以及第二数据在文件中的数据的偏移量,处理单元1302在文件中写入第二数据时,可以根据第二数据的数据长度以及第二数据在文件中的数据的偏移量从文件的数据存储区域确定需要替换的数据,利用第二数据覆盖需要替换的数据。
在一种可能的实施方式中,处理单元1302在根据文件标识确定文件索引,更新文件索引时,可以根据文件标识确定文件的索引存储区域,从文件的索引存储区域中读取文件索引,更新文件索引。
基于与方法实施例同一发明构思,本申请实施例还提供了一种文件注册装置,该文件注册装置部署在处理器上,用于执行上述如图3所示的方法实施例中处理器120执行的方法,相关特征可参见上述方法实施例,此处不再赘述。如图14所示,装置1400包括处理单元1402。
处理单元1402,用于确定第一设备的内存中文件的存储区域,并向第一设备的网卡通知文件的存储区域。
在一种可能的实施方式中,装置1400还包括接收单元1401和发送单元1403,在处理单元1402确定文件在第一设备的内存中的存储区域之前,接收单元1401可以接收到来自第二设备的文件打开请求,文件打开请求用于请求打开文件。发送单元1403可以通过第一 设备的网卡向第二设备发送文件打开响应,文件打开响应用于指示文件打开成功。
在一种可能的实施方式中,若文件为第一设备的内存中未存储的文件,处理单元1402在确定文件的存储区域时,可以为文件分配存储区域。
在一种可能的实施方式中,文件的存储区域包括下列的至少一个:
文件的数据存储区域、文件的索引存储区域、文件的信息存储区域和文件的日志存储区域。
基于与方法实施例同一发明构思,本申请实施例还提供了一种存储节点,该存储节点用于执行上述如图3、图5、图8、图9、图11所示的方法实施例中服务端100中处理器120和网卡140执行的方法,相关特征可参见上述方法实施例,此处不再赘述。需要说明的是,本申请实施例中对单元的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。在本申请的实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
上述实施例,可以全部或部分地通过软件、硬件、固件或其他任意组合来实现。当使用软件实现时,上述实施例可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载或执行所述计算机程序指令时,全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以为通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集合的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质。半导体介质可以是固态硬盘(solid state drive,SSD)。
本领域内的技术人员应明白,本申请的实施例可提供为方法、系统、或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方 框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然,本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请范围。这样,倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内,则本申请也意图包含这些改动和变型在内。

Claims (36)

  1. 一种文件访问方法,其特征在于,所述方法包括:
    第一设备的网卡接收来自第二设备的文件访问请求,所述文件访问请求用于请求访问文件,所述文件访问请求包括所述文件的文件标识;
    所述网卡根据所述文件标识访问所述第一设备的内存中的所述文件,并向所述第二设备发送文件访问响应。
  2. 如权利要求1所述的方法,其特征在于,所述第一设备的网卡接收来自第二设备的文件访问请求之前,还包括:
    所述第一设备的处理器接收到来自第二设备的文件打开请求,所述文件打开请求用于请求打开所述文件;
    所述处理器根据所述文件打开请求打开所述文件,确定所述第一设备的内存中所述文件的存储区域,向所述网卡通知所述文件的存储区域;
    所述网卡针对所述文件的存储区域执行内存注册操作,为所述文件分配所述文件标识;
    所述网卡向所述第二设备发送所述文件标识。
  3. 如权利要求2所述的方法,其特征在于,所述文件的存储区域包括下列区域中的至少一个:
    所述文件的数据存储区域、所述文件的索引存储区域、所述文件的信息存储区域和所述文件的日志存储区域。
  4. 如权利要求2或3所述的方法,其特征在于,所述文件访问请求用于请求读取文件的第一数据;所述网卡根据所述文件标识访问所述第一设备的内存中的所述文件,并向所述第二设备发送文件访问响应,包括:
    所述网卡根据所述文件标识确定所述文件的文件索引;
    所述网卡根据所述文件索引获取所述第一数据,向所述第二设备发送文件访问响应,所述文件访问响应中包括所述第一数据。
  5. 如权利要求4所述的方法,其特征在于,所述网卡根据所述文件标识确定所述文件的文件索引,包括:
    所述网卡根据所述文件标识确定所述文件的索引存储区域,从所述文件的索引存储区域中读取所述文件索引。
  6. 如权利要求4或5所述的方法,其特征在于,所述文件索引用于指示所述文件中的数据在所述文件的数据存储区域中的位置,所述网卡根据所述文件索引获取所述第一数据,包括:
    所述网卡根据所述文件索引从所述文件的数据存储区域中获取所述第一数据。
  7. 如权利要求2或3所述的方法,其特征在于,所述文件访问请求用于请求在文件中写入第二数据,所述文件访问请求包括所述第二数据,所述网卡根据所述文件标识访问所述第一设备的内存中的所述文件,并向所述第二设备发送文件访问响应,包括:
    所述网卡在所述文件中写入所述第二数据;
    所述网卡向所述第二设备发送所述文件访问响应,所述文件访问响应用于指示所述第二数据成功写入。
  8. 如权利要求7所述的方法,其特征在于,所述方法还包括:
    所述网卡根据所述文件标识确定所述文件索引,更新所述文件索引,更新后的所述文件索引用以指示所述第二数据的存储位置。
  9. 如权利要求7或8所述的方法,其特征在于,所述网卡在所述文件中写入所述第二数据,包括:
    所述网卡申请第一存储区域,将所述第二数据写入到所述第一存储区域中,所述第一存储区域位于所述文件的数据存储区域中,所述第一存储区域的起始位置指向所述文件的末尾。
  10. 如权利要求7或8所述的方法,其特征在于,所述网卡在所述文件中写入所述第二数据,包括:
    所述网卡申请第二存储区域,将所述第二数据写入到所述第二存储区域中,所述第二存储区域位于所述文件的数据存储区域中。
  11. 如权利要求7所述的方法,其特征在于,所述文件访问请求还包括所述第二数据的数据长度以及所述第二数据在所述文件中的数据的偏移量,所述网卡在所述文件中写入所述第二数据,包括:
    所述网卡根据所述第二数据的数据长度以及所述第二数据在所述文件中的数据的偏移量从所述文件的数据存储区域中确定需要替换的数据,利用所述第二数据覆盖所述需要替换的数据。
  12. 如权利要求8~10任一所述的方法,所述网卡根据所述文件标识确定所述文件索引,更新所述文件索引,包括:
    所述网卡根据所述文件标识确定所述文件的索引存储区域,从所述文件的索引存储区域中读取所述文件索引,更新所述文件索引。
  13. 一种存储节点,其特征在于,所述存储节点包括网卡和内存;
    所述网卡,用于接收来自客户端的文件访问请求,所述文件访问请求用于请求访问文件,所述文件访问请求包括所述文件的文件标识;根据所述文件标识访问所述内存中的文件,并向所述客户端发送文件访问响应;
    所述内存,用于存储所述文件。
  14. 如权利要求13所述的存储节点,其特征在于,所述存储节点还包括处理器;
    所述处理器,用于接收到来自客户端的文件打开请求,所述文件打开请求用于请求打开所述文件;根据所述文件打开请求打开所述文件,确定所述内存中所述文件的存储区域,向所述网卡通知所述文件的存储区域;
    所述网卡,还用于针对所述文件的存储区域执行内存注册操作,为所述文件分配所述文件标识;向所述客户端发送文件打开响应,所述文件打开响应中包括所述文件标识。
  15. 如权利要求14所述的存储节点,其特征在于,所述文件的存储区域包括下列区域中的至少一个:
    所述文件的数据存储区域、所述文件的索引存储区域、所述文件的信息存储区域和所述文件的日志存储区域。
  16. 如权利要求14或15所述的存储节点,其特征在于,所述文件访问请求用于请求读取文件的第一数据;所述网卡根据所述文件标识访问所述文件,并向所述客户端发送 文件访问响应时,具体用于:
    根据所述文件标识确定所述文件的文件索引;
    根据所述文件索引获取所述第一数据,向所述客户端反馈文件访问响应,所述文件访问响应中包括所述第一数据。
  17. 如权利要求16所述的存储节点,其特征在于,所述网卡在根据所述文件标识确定所述文件的文件索引时,具体用于:
    根据所述文件标识确定所述文件的索引存储区域,从所述文件的索引存储区域中读取所述文件索引。
  18. 如权利要求16或17所述的存储节点,其特征在于,所述文件索引用于指示所述文件中的数据在所述文件的数据存储区域中的位置,所述网卡根据所述文件索引获取所述第一数据时,具体用于:
    根据所述文件索引从所述文件的数据存储区域中获取所述第一数据。
  19. 如权利要求14或15所述的存储节点,其特征在于,所述文件访问请求用于请求在文件中写入第二数据;所述文件访问请求还包括所述第二数据,所述网卡根据所述文件标识访问所述文件,并向所述客户端发送文件访问响应时,具体用于:
    在所述文件中写入所述第二数据;
    向所述客户端反馈所述文件访问响应,所述文件访问响应用于指示所述第二数据成功写入。
  20. 如权利要求19所述的存储节点,其特征在于,所述网卡还用于:
    根据所述文件标识确定所述文件索引,更新所述文件索引,更新后的所述文件索引用以指示所述第二数据的存储位置。
  21. 如权利要求19或20所述的存储节点,其特征在于,所述网卡在所述文件中写入所述第二数据时,具体用于:
    申请第一存储区域,将所述第二数据写入到所述第一存储区域中,所述第一存储区域位于所述文件的数据存储区域中,所述第一存储区域的起始位置指向所述文件的末尾。
  22. 如权利要求19或20所述的存储节点,其特征在于,所述网卡在所述文件中写入所述第二数据时,具体用于:
    申请第二存储区域,将所述第二数据写入到所述第二存储区域中,所述第二存储区域位于所述文件的数据存储区域中。
  23. 如权利要求19所述的存储节点,其特征在于,所述文件访问请求还包括所述第二数据的数据长度以及所述第二数据在所述文件中的数据的偏移量,所述网卡在所述文件中写入所述第二数据时,具体用于:
    根据所述第二数据的数据长度以及所述第二数据在所述文件中的数据的偏移量从所述文件的数据存储区域中确定需要替换的数据,利用所述第二数据覆盖所述需要替换的数据。
  24. 如权利要求20~22任一所述的存储节点,其特征在于,所述网卡根据所述文件标识确定所述文件索引,更新所述文件索引时,具体用于:
    根据所述文件标识确定所述文件的索引存储区域,从所述文件的索引存储区域中读 取所述文件索引,更新所述文件索引。
  25. 一种网卡,其特征在于,所述网卡能够用于第一设备中,所述网卡包括处理器以及接口;
    所述处理器用于:通过所述接口接收来自第二设备的文件访问请求,所述文件访问请求用于请求访问文件,所述文件访问请求包括所述文件的文件标识;根据所述文件标识访问所述第一设备的内存中的所述文件,并通过所述接口向所述第二设备发送文件访问响应。
  26. 如权利要求25所述的网卡,其特征在于,所述处理器在通过所述接口接收来自第二设备的文件访问请求之前,还用于:
    获取所述文件的存储区域,针对所述文件的存储区域执行内存注册操作,为所述文件分配所述文件标识;
    通过所述接口向所述第二设备发送所述文件标识。
  27. 如权利要求26所述的网卡,其特征在于,所述文件的存储区域包括下列的至少一个:
    所述文件的数据存储区域、所述文件的索引存储区域、所述文件的信息存储区域和所述文件的日志存储区域。
  28. 如权利要求26或27所述的网卡,其特征在于,所述文件访问请求用于请求读取文件的第一数据;所述处理器在根据所述文件标识访问所述文件,并向所述第二设备发送文件访问响应时,具体用于:
    根据所述文件标识确定所述文件的文件索引;
    根据所述文件索引中获取所述第一数据,向所述第二设备发送文件访问响应,所述文件访问响应中包括所述第一数据。
  29. 如权利要求28所述的网卡,其特征在于,所述处理器在根据所述文件标识确定所述文件的文件索引时,具体用于:
    根据所述文件标识确定所述文件的索引存储区域,从所述文件的索引存储区域中读取所述文件索引。
  30. 如权利要求28或29所述的网卡,其特征在于,所述文件索引用于指示所述文件中的数据在所述文件的数据存储区域中的位置,所述处理器在根据所述文件索引获取所述第一数据时,具体用于:
    根据所述文件索引从所述文件的数据存储区域中获取所述第一数据。
  31. 如权利要求26或27所述的网卡,其特征在于,所述文件访问请求用于请求在文件中写入第二数据,所述文件访问请求包括所述第二数据,所述处理器在根据所述文件标识访问所述文件,并向所述第二设备发送文件访问响应时,具体用于:
    在所述文件中写入所述第二数据;
    向所述第二设备发送所述文件访问响应,所述文件访问响应用于指示所述第二数据成功写入。
  32. 如权利要求31所述的网卡,其特征在于,所述处理器还用于:
    根据所述文件标识确定所述文件索引,更新所述文件索引,更新后的所述文件索引用以指示所述第二数据的存储位置。
  33. 如权利要求31或32所述的网卡,其特征在于,所述处理器在所述文件中写入所述第二数据时,具体用于:
    申请第一存储区域,将所述第二数据写入到所述第一存储区域中,所述第一存储区域位于所述文件的数据存储区域中,所述第一存储区域的起始位置指向所述文件的末尾。
  34. 如权利要求31或32所述的网卡,其特征在于,所述处理器在所述文件中写入所述第二数据时,具体用于:
    申请第二存储区域,将所述第二数据写入到所述第二存储区域中,所述第二存储区域位于所述文件的数据存储区域中。
  35. 如权利要求31所述的网卡,其特征在于,所述文件访问请求还包括所述第二数据的数据长度以及所述第二数据在所述文件中的数据的偏移量,所述处理模块在所述文件中写入所述第二数据时,具体用于:
    根据所述第二数据的数据长度以及所述第二数据在所述文件中的数据的偏移量从所述文件的数据存储区域中确定需要替换的数据,利用所述第二数据覆盖所述需要替换的数据。
  36. 如权利要求32~34任一所述的网卡,其特征在于,所述处理器在根据所述文件标识确定所述文件索引,更新所述文件索引时,具体用于:
    根据所述文件标识确定所述文件的索引存储区域,从所述文件的索引存储区域中读取所述文件索引,更新所述文件索引。
PCT/CN2021/115931 2020-12-08 2021-09-01 一种文件访问方法、存储节点以及网卡 WO2022121385A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21902093.0A EP4258122A1 (en) 2020-12-08 2021-09-01 File access method, storage node, and network card
US18/331,274 US20230315683A1 (en) 2020-12-08 2023-06-08 File Access Method, Storage Node, and Network Interface Card

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202011444954.3 2020-12-08
CN202011444954 2020-12-08
CN202110221660.2A CN114610678A (zh) 2020-12-08 2021-02-27 一种文件访问方法、存储节点以及网卡
CN202110221660.2 2021-02-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/331,274 Continuation US20230315683A1 (en) 2020-12-08 2023-06-08 File Access Method, Storage Node, and Network Interface Card

Publications (1)

Publication Number Publication Date
WO2022121385A1 true WO2022121385A1 (zh) 2022-06-16

Family

ID=81857578

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/115931 WO2022121385A1 (zh) 2020-12-08 2021-09-01 一种文件访问方法、存储节点以及网卡

Country Status (4)

Country Link
US (1) US20230315683A1 (zh)
EP (1) EP4258122A1 (zh)
CN (1) CN114610678A (zh)
WO (1) WO2022121385A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105450588A (zh) * 2014-07-31 2016-03-30 华为技术有限公司 一种基于rdma的数据传输方法及rdma网卡
US20170034268A1 (en) * 2015-07-31 2017-02-02 Netapp, Inc. Systems, methods and devices for rdma read/write operations
CN109634876A (zh) * 2018-12-11 2019-04-16 广东省新代通信与网络创新研究院 文件访问方法、装置及计算机可读存储介质
CN110647480A (zh) * 2018-06-26 2020-01-03 华为技术有限公司 数据处理方法、远程直接访存网卡和设备
CN110764688A (zh) * 2018-07-27 2020-02-07 杭州海康威视数字技术股份有限公司 对数据进行处理的方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105450588A (zh) * 2014-07-31 2016-03-30 华为技术有限公司 一种基于rdma的数据传输方法及rdma网卡
US20170034268A1 (en) * 2015-07-31 2017-02-02 Netapp, Inc. Systems, methods and devices for rdma read/write operations
CN110647480A (zh) * 2018-06-26 2020-01-03 华为技术有限公司 数据处理方法、远程直接访存网卡和设备
CN110764688A (zh) * 2018-07-27 2020-02-07 杭州海康威视数字技术股份有限公司 对数据进行处理的方法和装置
CN109634876A (zh) * 2018-12-11 2019-04-16 广东省新代通信与网络创新研究院 文件访问方法、装置及计算机可读存储介质

Also Published As

Publication number Publication date
EP4258122A1 (en) 2023-10-11
US20230315683A1 (en) 2023-10-05
CN114610678A (zh) 2022-06-10

Similar Documents

Publication Publication Date Title
KR102427561B1 (ko) Fpga 기반 가속화를 위한 새로운 ssd 구조
US9342448B2 (en) Local direct storage class memory access
US10848468B1 (en) In-flight data encryption/decryption for a distributed storage platform
US9973394B2 (en) Eventual consistency among many clusters including entities in a master member regime
WO2019127018A1 (zh) 存储系统访问方法及装置
US20090019223A1 (en) Method and systems for providing remote strage via a removable memory device
JP6757808B2 (ja) インフィニバンド(IB)上で仮想ホストバスアダプタ(vHBA)を管理およびサポートするためのシステムおよび方法、ならびに単一の外部メモリインターフェイスを用いてバッファの効率的な使用をサポートするためのシステムおよび方法
WO2017084348A1 (zh) Ip硬盘的管理方法和装置
CN112130748A (zh) 一种数据访问方法、网卡及服务器
WO2019127021A1 (zh) 存储系统中存储设备的管理方法及装置
CN104317716B (zh) 分布式节点间的数据传输方法及分布式节点设备
WO2015180650A1 (zh) 在存储网络中的键值生成方法及装置
TW201621699A (zh) 透過主動訊息支持遠端記憶體存取(rma)應用程式介面(api)
US11226778B2 (en) Method, apparatus and computer program product for managing metadata migration
WO2019127017A1 (zh) 存储系统中存储设备的管理方法及装置
WO2016101662A1 (zh) 一种数据处理方法及相关服务器
CN115470156A (zh) 基于rdma的内存使用方法、系统、电子设备和存储介质
US11157456B2 (en) Replication of data in a distributed file system using an arbiter
US20120005442A1 (en) Storage device, access control program recording medium, and control method of storage device
CN112148206A (zh) 一种数据读写方法、装置、电子设备及介质
CN111435286A (zh) 一种数据存储方法、装置和系统
US20230342087A1 (en) Data Access Method and Related Device
WO2022223003A1 (zh) 一种文件访问方法、网卡以及计算设备
WO2022121385A1 (zh) 一种文件访问方法、存储节点以及网卡
WO2022073399A1 (zh) 存储节点、存储设备及网络芯片

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21902093

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021902093

Country of ref document: EP

Effective date: 20230704