WO2023109864A1

WO2023109864A1 - Data system, and data access method and device

Info

Publication number: WO2023109864A1
Application number: PCT/CN2022/139000
Authority: WO
Inventors: 任仁; 马超; 申勇; 刘宏伟; 赵博南
Original assignee: 华为技术有限公司
Priority date: 2021-12-16
Filing date: 2022-12-14
Publication date: 2023-06-22
Also published as: CN116266842A

Abstract

Provided are a data system, and a data access method and a device. In the present application, when accessing data in a server, a client sends a data request to the server, wherein the data request comprises a sequence, and the sequence comprises an address of a metadata state, an address of metadata, and an address of a metadata state. The server acquires a plurality of pieces of data according to the sequence, wherein the plurality of pieces of data comprise metadata, a first state before the metadata is acquired, and a second state after the metadata is acquired. After acquiring the plurality of pieces of data, the server sends a data response to a client, wherein the data response comprises the metadata, the first state and the second state. When the client requests the metadata from the server, same acquires the plurality of pieces of data from the server by means of the data request, such that the efficiency of interaction between the server and the client can be effectively improved. After acquiring the first state and the second state, the client determines the integrity of the metadata by means of the first state and the second state, such that it is ensured that the client can accurately access the data in the server.

Description

A data system, data access method and device

technical field

The present application relates to the field of storage technologies, and in particular to a data system, a data access method and equipment.

Background technique

Remote direct memory access (remote direct memory access, RDMA), especially unilateral RDMA, can bypass the processor in the remote host and directly access the data in the remote host through the network card of the remote host. Because RDMA bypasses the processor, it can save a lot of processor resources and effectively improve the efficiency of data interaction.

However, for a storage system using a distributed architecture, the storage system includes multiple nodes, and changes in node status (such as node expansion, node failure, data migration between nodes, etc.) will affect the storage location of data in the storage system. When the client accesses the data stored by the nodes in the distributed storage system through unilateral RDMA, since the node's processor is bypassed, it cannot perceive the change of the node status in the distributed storage system, which will lead to data access failure.

Contents of the invention

The application provides a data system, a data access system and equipment to ensure accurate reading of data in a server.

In a first aspect, an embodiment of the present application provides a data system, and the data system includes a client and a server. When the client needs to read data from the server, it can send a data request to the server. The data request includes a sequence, and the sequence includes: the address of the first state, the address of the metadata, and the address of the second state, where the first state and the second state is the state of the metadata, the first state and the second state are used to describe whether the metadata changes, and the address of the second state is the same as the address of the first state.

After receiving the data request, the server may acquire a plurality of data according to a sequence, the plurality of data including metadata, a first state before obtaining the metadata, and a second state after obtaining the metadata. After acquiring the plurality of data, the server may send a data response to the client, where the data response includes metadata, the first state, and the second state.

Through the above system, when the client needs to read the data in the server, when the client requests metadata from the server, it can request metadata from the server through a data request and obtain the metadata status before the metadata (section state) and the state of the metadata after the metadata is acquired (the second state). That is to say, the client can obtain multiple data from the server through a data request, which can effectively improve the interaction efficiency between the server and the client. In addition, after the client obtains the first state and the second state, the client can determine the change of the metadata when obtaining the metadata through the first state and the second state, so as to determine the integrity and validity of the metadata. To ensure that the client can accurately access the data in the server.

In a possible implementation manner, the sequence includes the address of the first state, the address of the metadata, the address of the second state, and the address of the home state of the target data. That is to say, after the address of the first state, the address of metadata, and the address of the second state, the sequence further includes the address of the belonging state of the target data. The target data is the data described by the metadata. The data response fed back from the server to the client also includes the attribution status of the target data. After the server obtains the metadata, the first status, and the second status, it can also The address gets the attribution status of the target data.

Through the above system, the server can obtain the first state, the metadata, the second state and the attribution state of the target data sequentially according to the order of each address in the sequence. The server can carry multiple acquired data in the data response, and feed the data response back to the client, so that the client can obtain the first state, metadata, second state, and the attribution of the target data from the data response state, the client can further determine the integrity of the metadata to ensure that the target data can be accessed accurately in the future.

In a possible implementation manner, after receiving the data response, the client may acquire metadata, the first state, and the second state therefrom. The client may compare the first state with the second state to determine whether the first state is consistent with the second state.

Through the above system, the client can determine the integrity of the metadata by using the comparison result between the first state and the second state, so as to ensure that the client can accurately obtain the target data based on the complete metadata.

In a possible implementation manner, the attribution status of the target data is recorded in the metadata, and after receiving the data response, the client can obtain the first status, the metadata, the second status, and the attribution status of the target data. In addition to comparing the first state with the second state, the client can also compare the attribution state of the target data with the attribution state of the target data recorded in the metadata, and determine that the attribution state of the target data is consistent with the attribution state recorded in the metadata. Whether the attribution status of the target data is consistent.

Through the above system, the client can determine the integrity of the metadata by using the comparison result between the first state and the second state, and the comparison result between the attribution state of the target data and the attribution state of the target data recorded in the metadata, so as to ensure The client can accurately obtain target data based on complete metadata.

In a possible implementation, if the data response carries the first state, the second state, and metadata, the client obtains the target data from the server according to the metadata after determining that the first state is consistent with the second state .

If the data response carries the first state, the second state, metadata, and the attribution state of the target data, the client determines that the first state is consistent with the second state, and that the attribution state of the target data is consistent with the target data recorded in the metadata. When the attribution status is consistent, the target data is obtained from the server according to the metadata.

If the data response carries the first state, the second state, metadata, and the attribution state of the target data, the client determines that the first state is inconsistent with the second state, or that the attribution state of the target data is different from the target data recorded in the metadata. If the attribution status is inconsistent, the client can determine that the metadata is invalid, and the client can request the metadata from the server again, or interact with the server to determine the server to which the target data belongs.

Through the above system, the client can determine the integrity of the metadata according to the data carried in the data response, so that the target data can be accurately read from the server later.

In a possible implementation manner, the sequence includes the address of the first state, the address of the metadata, the address of the second state, the address of the state to which the target data belongs, and the address of the target data. That is to say, in this sequence, the address of the first state, the address of the metadata, and the address of the second state also include the address of the belonging state and the address of the target data, and the target data is the data described by the metadata; the data response Including the attribution status of the target data and the target data; after the server obtains the metadata, the first state, and the second state, it can also obtain the attribution status of the target data according to the address of the attribution status of the target data, and obtain the attribution status of the target data according to the address of the target data target data. The address of the target data is pre-saved by the client. For example, when the client uses the address of the target data to access the target data, the metadata of the target data can be saved. The metadata of the target data includes the address of the target data. Of course, the client may also only save the address of the target data.

Through the above system, when the client needs to read the data in the server, when the client requests metadata from the server, it can request metadata, the first state, the second state, and the target from the server through a data request data. The client can request multiple data through one data request, which can effectively reduce the number of interactions between the client and the server, and improve the interaction efficiency between the server and the client.

In a possible implementation, the client may store metadata locally, and if the client determines that the locally stored metadata is inconsistent with the metadata obtained from the server, the terminal to obtain target data.

Through the above system, the client can directly compare the consistency of the metadata to determine whether the metadata has been changed. If the inconsistency is determined, the target data can be read by using the metadata obtained from the server to ensure the final target data accuracy.

In a possible implementation manner, when sending the data request, the client may send the data request to the network card of the server based on RDMA.

Through the above system, the client sends a data request to the network card of the server based on RDMA, which can bypass the processor of the server and reduce the occupation of the processor of the server.

In a possible implementation manner, when the server internally processes the data request, the data request can be processed by a network card in the server, and the network card can respectively obtain the metadata, the first state before obtaining the metadata, the And the second state after obtaining the metadata; sending a data response to the client.

Through the above system, the network card in the server can process the data request more efficiently, and can also reduce the resource occupation of the processor in the server.

In the second aspect, the embodiment of the present application provides a data access method, which can be executed by the server, and for the beneficial effect, please refer to the related description of the first aspect. In this method, the server can receive a data request from the client, and the data request includes a sequence, and the sequence includes: the address of the first state, the address of the metadata, and the address of the second state, wherein the first state and the second state use To describe whether the metadata has changed, the address of the second state is the same as the address of the first state; after receiving the data request, the server can obtain the metadata, the first state before obtaining the metadata, and obtain The second state after the metadata; and sending a data response to the client, the data response includes the metadata, the first state, and the second state.

In a possible implementation, the sequence further includes the address of the attribution state of the target data, the address of the attribution state of the target data is located after the address of the second state, the target data is the data described by the metadata, and the data response also includes the target For the attribution status of the data, after the server obtains the metadata, the first status, and the second status, the attribution status can be obtained according to the address of the attribution status of the target data.

In a possible implementation, the sequence further includes the address of the home state of the target data and the address of the target data, the address of the home state of the target data is located after the address of the second state, and the address of the target data is located in the home state of the target data After the address, the target data is the data described by the metadata; the data response also includes the attribution status of the target data and the target data. After obtaining the metadata, the first state, and the second state, the server can also obtain the attribution state according to the address of the attribution state, and obtain the target data according to the address of the target data.

In a possible implementation manner, when the server internally processes the data request, the network card of the server may process the data request and feed back a data response. That is to say, the network card at the server end can obtain the metadata, the first state before obtaining the metadata, and the second state after obtaining the metadata respectively according to the sequence; and send a data response to the client.

In the third aspect, the embodiment of the present application provides a data access method, which can be executed by the client. For the beneficial effects, please refer to the relevant description of the first aspect, and details will not be repeated here. In this method, the client can send a data request to the server, and the data request includes a sequence, and the sequence includes: the address of the first state, the address of the metadata, and the address of the second state, wherein the first state and the second state are used for Describes whether the metadata has changed, and the address of the second state is the same as the address of the first state. Afterwards, the client may receive the data response sent by the server, where the data response includes metadata, the first state, and the second state.

In a possible implementation, the sequence further includes the address of the attribution state of the target data, the address of the attribution state of the target data is located after the address of the second state, the target data is the data described by the metadata, and the data response also includes the target The attribution status of the data.

In a possible implementation manner, after receiving the data access response, the client may compare the first state with the second state to determine whether the first state is consistent with the second state.

In a possible implementation, the attribution status of the target data is recorded in the metadata. After the client receives the data access response, the client can check the attribution status of the target data and the attribution of the target data recorded in the metadata. The status is compared to determine whether the attribution status of the target data is consistent with the attribution status of the target data recorded in the metadata.

In a possible implementation, when the client determines the integrity of the metadata, if the data response carries the first state, the second state, and the metadata, the client determines that the first state is consistent with the second state Next, the target data is obtained from the server according to the metadata.

In a possible implementation, the sequence further includes the address of the home state of the target data and the address of the target data, the address of the home state of the target data is located after the address of the second state, and the address of the target data is located in the home state of the target data After the address of , the target data is the data described by the metadata; the data response includes the attribution status of the target data and the target data.

In a possible implementation manner, when the client sends a data request to the client, the data request may be sent to the network card of the server based on RDMA.

In the fourth aspect, the embodiment of the present application also provides a data access device, the data access device has the function of realizing the behavior in the method example of the second aspect above, and the beneficial effects can be referred to the description of the first aspect, which will not be repeated here. The functions may be implemented by hardware, or may be implemented by executing corresponding software through hardware. Hardware or software includes one or more units corresponding to the functions described above. In a possible design, the structure of the device includes a receiving unit, an acquiring unit, and a sending unit, and these units can perform the corresponding functions in the method example of the second aspect above. For details, refer to the detailed description in the method example, which will not be repeated here. .

In the fifth aspect, the embodiment of the present application also provides a data access device, the data access device has the function of implementing the behavior in the method example of the third aspect above, and the beneficial effects can be referred to the description of the first aspect, which will not be repeated here. The functions may be implemented by hardware, or may be implemented by executing corresponding software through hardware. Hardware or software includes one or more units corresponding to the functions described above. In a possible design, the structure of the device includes a receiving unit, a sending unit, and optionally a determining unit. These units can perform the corresponding functions in the method example of the third aspect above. For details, refer to the detailed description in the method example. description, and will not be repeated here.

In a sixth aspect, the embodiment of the present application further provides a computing device, where the computing device includes a processor, a network card, and a memory.

In a possible implementation manner, the network card has the function of realizing the above-mentioned second aspect and the behaviors in the method examples in each possible implementation manner of the second aspect. For the beneficial effects, please refer to the description of the first aspect and will not repeat them here. .

In another possible implementation manner, computer program instructions are stored in the memory, and the processor is coupled to the memory, and the processor can call the computer-executed instructions stored in the memory to execute the above third aspect and various possible implementations of the third aspect methods in methods.

In the seventh aspect, the present application also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when it is run on a computer, the computer executes the above-mentioned second aspect and various possible implementations of the second aspect methods in methods. When it runs on a computer, the computer is made to execute the third aspect and the method in each possible implementation manner of the third aspect.

In an eighth aspect, the present application further provides a computer program product including instructions, which, when run on a computer, cause the computer to execute the above second aspect and the method in each possible implementation manner of the second aspect. When it runs on a computer, the computer is made to execute the third aspect and the method in each possible implementation manner of the third aspect.

In the ninth aspect, the present application also provides a computer chip, the chip is connected to the memory, and the chip is used to read and execute the software program stored in the memory, and execute the method in the above-mentioned second aspect and each possible implementation manner of the second aspect Or execute the third aspect and the method in each possible implementation manner of the third aspect.

Description of drawings

FIG. 1 is a schematic structural diagram of a system provided by the present application;

FIG. 2 is a schematic diagram of a data access method provided by the present application;

3-4 are schematic structural diagrams of a data access device provided in the present application.

Detailed ways

Before explaining the data processing method provided by this application, first explain the concepts involved in this application:

1. Metadata.

Also known as intermediary data and relay data. Metadata is data describing data (data about data). Metadata can indicate the attributes of data. For example, metadata can record the storage address of data, the attribution status of data, etc.

Wherein, the storage address is usually the physical address of the data in the storage system, and the attribution status of the data describes the node in the storage system where the data described by the metadata is located.

The embodiment of the present application does not limit the manner in which the metadata records the attribution status of the data, and a manner for recording the attribution status of the data is introduced below.

In the storage system, data is stored in units of shards, and data migration between nodes in the storage system can also be performed in units of shards. A shard includes multiple pieces of data. For any shard, the version number of the shard will be updated every time it is migrated between nodes in the storage system. For example, if the shard is migrated from node A to node B, the shard version number of the shard recorded in node A will remain unchanged, and the version number of the shard on node B will be increased by one. In addition, the metadata of the data in the slice may include a version number of the slice. Still take the migration of a fragment from node A to node B as an example. Before the migration of the fragment, the version number of the fragment in the metadata of the data in the fragment is consistent with the version number of the fragment recorded by node A. After the migration of the fragment , when new data is written to the shard, the version number of the shard in the metadata of the new data is consistent with the version number of the shard recorded by Node B. It should be noted that the embodiment of the present application does not limit the type of the storage system, which may be a distributed storage system or a centralized storage system.

2. Metadata status.

Metadata itself can also be regarded as a kind of data. For any metadata, there is a metadata description header, which can record the status of the metadata. When metadata is updated, the metadata status is updated accordingly.

The embodiment of the present application does not limit the way of recording the metadata state. For example, the metadata status can be represented by a variable value, which is incremented by one every time the metadata changes.

3. Remote direct memory access (RDMA).

RDMA is a technology that bypasses the operating system kernel of a remote device (such as a node in a storage system) to access data in its memory. Because it does not go through the operating system, it not only saves a lot of processor resources, but also improves system throughput and reduces It reduces the network communication delay of the system, and is especially suitable for wide application in large-scale parallel computer clusters.

RDMA has several major characteristics, (1) data is transmitted between the network and the remote device; (2) without the participation of the operating system kernel, all content related to sending and transmitting is offloaded to the network card; (3) virtual memory in user space The direct data transmission between the NIC does not involve the operating system kernel, and there is no additional data movement and copying.

4. Unilateral RDMA and bilateral RDMA.

Here, the two ends that need to exchange information are respectively referred to as a client device (client for short) and a server. The client is deployed on the user side, and the user can initiate a request to the server through the client. The server can be deployed at the remote end. The server generally refers to the storage system, and can be specifically understood as a node in the storage system.

Unilateral RDMA includes RDMA read (READ) operations and RDMA write (WRITE) operations.

Taking the RDMA read operation in unilateral RDMA as an example, the RDMA read operation is used for the client to read data in the storage space of the server.

Before the client performs the RDMA read operation, the server first allows the client to obtain the address of the data to be read in advance, that is, the address and length of the data to be read by the RDMA read operation are sent to the client. In this embodiment of the present application, the address of the data includes the address of some or all of the following metadata, the address of the metadata state, the address of the data attribution state, the address of the target data, and the like. After the client obtains the address of the data to be read, it can directly read the data from the storage space of the server through the network card of the server, and the whole process does not require the participation of the processor.

In the embodiment of the present application, the client device may send a data request in a unilateral RDMA manner, and read the data requested by the data request from the server.

Bilateral RDMA includes an RDMA send (SEND) operation and an RDMA receive (RECEIVE) operation, which requires the participation of the server processor.

As shown in Figure 1, it is a schematic structural diagram of a data system provided by the embodiment of the present application. The data system includes a client 120 and a server 110. The server 110 may be a node in the storage system. The embodiment of the present application does not The type of the storage system is not limited, and may be a distributed storage system or a centralized storage system.

Users access data through applications. The computers running these applications are called “client devices” and may also be referred to as clients 120 for short. Client 120 may be a physical machine or a virtual machine. Physical machines include, but are not limited to, desktop computers, servers, laptops, and mobile devices.

Users can access data in the server 110 through the client 120 , such as writing data in the server 110 or reading data from the server 110 .

When the client 120 detects the user's operation on the application program and determines that the user needs to access the data in the server 110 , the client 120 can interact with the server 110 to access the data in the server 110 .

When the client 120 interacts with the server 110 to access the data in the server 110, the client 120 may first obtain the metadata of the data, and obtain the data according to the metadata of the data.

When acquiring the metadata of the data, the client 120 may request the metadata of the data from the server 110 through a data request, as well as the state of the server 110 before and after reading the metadata. After the client 120 obtains the metadata of the data and the status of the server 110 before and after reading the metadata, it can determine the metadata of the data by comparing the status of the server 110 before and after reading the metadata. The integrity of the data further determines the validity of the data obtained using this metadata. In this way, it can avoid the situation that the metadata obtained according to the metadata is invalid due to the change of the metadata during the process of reading the metadata. In addition, multiple data can be requested from the server 110 through one data request, which can effectively reduce the number of interactions between the server 110 and the client 120 and improve data access efficiency.

When acquiring the metadata of the data, the client 120 may also request the metadata of the data, the status of the server 110 before and after reading the metadata, and the attribution status of the data from the server 110 through a data request. After the client 120 obtains the metadata of the data, the status of the server 110 before and after reading the metadata, and the attribution status of the data, it can compare the status of the server 110 before and after reading the metadata , or compare the attribution status of the acquired data with the attribution status of the data recorded in the metadata, determine the integrity of the metadata of the data, and further determine the validity of the data acquired by using the metadata. In this way, it is possible to avoid the situation that the metadata obtained according to the metadata is invalid due to the change of the metadata or the change of the attribution status of the data during the process of reading the metadata. In addition, multiple data can be requested from the server 110 through one data request, which can effectively reduce the number of interactions between the server 110 and the client 120 and improve data access efficiency.

When the client 120 interacts with the server 110 to access the data in the server 110 , the client 120 can acquire the data by using the previously saved data address of the data while acquiring the metadata of the data. When the client 120 obtains the metadata of the data, the client 120 may request the server 110 for the metadata of the data, the state of the server 110 before and after reading the metadata, and the data (the The data request carries the data address). After the client 120 obtains the metadata of the data and the status of the server 110 before and after reading the metadata, it can determine the metadata of the data by comparing the status of the server 110 before and after reading the metadata. The integrity of the data further determines the validity of obtaining the data by using the previously saved data address of the data. In this way, it can avoid the situation that the metadata obtained according to the metadata is invalid due to the change of the metadata during the process of reading the metadata. When it is determined that the metadata is complete, the obtained data is also valid, and there is no need to use the metadata to obtain the data, which can reduce the number of interactions between the server 110 and the client 120 . In addition, multiple data can be requested from the server 110 through one data request, which can effectively reduce the number of interactions between the server 110 and the client 120 and improve data access efficiency.

Specifically, referring to FIG. 3 for the structure of the server 110 , the server 110 includes a bus 111 , a processor 112 , a memory 113 , a network card 114 and a hard disk 115 .

It should be noted that, in the embodiment of the present application, a hard disk is used as the persistent storage of the server 110 as an example for illustration, but mechanical hard disks or other types of hard disks are also applicable to the embodiments of the present application.

The processor 112 may be a central processing unit (central processing unit, CPU), and the processor 112 may also be other general-purpose processors or modules with computing functions.

The memory 113 may include a volatile memory (volatile memory), such as a random access memory (random access memory, RAM), a dynamic random access memory (dynamic random access memory, DRAM), and the like. It may also be a non-volatile memory (non-volatile memory), such as a storage-class memory (storage-class memory, SCM), or a combination of a volatile memory and a non-volatile memory.

The server 110 may also include one or more hard disks 115 . The hard disk 115 can be used to permanently store data. Specifically, inside the hard disk 115, the hard disk 115 may also include a hard disk cache and a persistent storage medium.

The network card 114 is used for communicating with the client 120 and receiving a data request sent by the client 120 .

When the client 120 sends a data request to the server 110 (for example, the client 120 uses unilateral RDMA to interact with the server 110), inside the server 110, the network card can process the data request, and feed back to the client 120 according to the data request The data request requests the data. For example, the network card 114 may process the data access request by invoking the computer-executed instructions stored in the memory 113, that is, execute the data access method provided in the embodiment of the present application. For another example, the network card 114 may also invoke computer-executed instructions stored inside the network card 114 to execute the data access method provided in the embodiment of the present application. For another example, computer storage instructions may also be programmed on the network card 114, and the network card 114 may execute the data access method provided in the embodiment of the present application.

Specifically, referring to FIG. 3 for the structure of the client 120 , the client 120 includes a bus 121 , a processor 122 , a memory 123 , and a network card 124 . The functions and functions of the bus 121, processor 122, memory 123, and network card 124 are similar to those of the bus 111, processor 112, memory 113, network card 114, and hard disk 115. For details, please refer to the foregoing description, and will not repeat them here.

When the client 120 interacts with the server 110, inside the client 120, the processor 122 can call the computer execution instructions stored in the memory 123 to execute the data access method provided by the embodiment of the present application.

The following describes the data access method provided by the embodiment of the present application with reference to FIG. 2 . For the convenience of description, the data described by the metadata is called target data. The method includes:

Step 201: the client 120 sends a data request to the server 110 when it needs to access target data. The data request is used to request data from the server 110, and the data request carries an address sequence (the address sequence may also be referred to as a sequence for short), and the address sequence includes multiple addresses. The multiple addresses are the addresses of the data that needs to be requested from the server 110 .

The user can trigger the client 120 to access the target data stored in the server by operating the application program running on the client 120 .

When the client 120 accesses the target data, it needs to obtain the metadata of the target data first, and determine the data address of the target data according to the metadata, and then the client 120 can request the target from the server 110 according to the metadata of the target data. data.

That is to say, the client 120 needs to obtain metadata first, and the data requested by the data request includes metadata. In addition, in order to ensure the integrity of the metadata, the data requested by the data request may also include other data. The address types included in the address sequence in the request and the data requested by the data request are explained:

In the first type, the address sequence includes the address of the metadata state, the address of the metadata, and the address of the metadata state in order. The data requested in the data request is metadata, a state of the metadata before reading the metadata, and a state of the metadata after reading the state of the metadata. In order to facilitate distinction, the metadata state before reading the metadata and the metadata state after reading the metadata state, in this embodiment of the application, the metadata state before reading the metadata is referred to as the first state . The state of the metadata after reading the metadata is referred to as the second state. The address of the metadata here refers to the address of the metadata of the target data. The address of the metadata state refers to the address of the metadata state of the target data. For example, the address of the metadata state may be the address of the description header of the metadata of the target data.

In the second type, the address sequence includes in order the address of the first state, the address of the metadata, the address of the second state, and the address of the attribution state of the target data. The data requested in the data request are metadata, the first state and the second state. Here, the address of the data attribution status refers to the address of the attribution status of the target data. For example, the data attribution status may be the version number of the segment to which the target data belongs in the server 110 , and the address of the data attribution status is the address in the server 110 that records the allocated version number.

In the third type, the address sequence includes the address of the first state, the address of the metadata, the address of the second state, the address of the belonging state of the target data, and the address of the target data in order. The data requested in the data request are metadata, the first state, the second state, the attribution state of the target data, and the target data. The address of the target data is the address used by the client 120 to read the target data before. If the client 120 has also read the target data before, the client 120 can save the target locally when reading the target data. The metadata of the data or the address of that target data.

Fourth, the address sequence includes the address of the first state, the address of the metadata, the address of the second state, and the address of the target data in sequence. The data requested by the data request are metadata, first state, second state, and target data.

It should be noted that the addresses that may be included in the above address sequence can be pre-acquired by the client 120, and this embodiment of this application does not limit the way that the client 120 pre-acquires the addresses included in the address sequence. For example, for the metadata address, the server 110 can centrally store each metadata in the memory, the server 110 can provide the client 120 with the starting address of the metadata in the memory in advance, and the client 120 device calculates based on a preset algorithm The starting address of the metadata is processed to obtain the address of the metadata of the target data. For another example, for metadata state addresses, when storing each metadata state, the server 110 may establish an association relationship between the address of the metadata state and the address of the corresponding metadata. The association relationship may be notified to the client 120 by the server 110 in advance, or may be pre-scheduled between the server 110 and the client 120 . After determining the address of the metadata, the client 120 may determine the address of the metadata state based on the association relationship and the address of the metadata. For another example, for the address of the attribution status of the target data, the server 110 may notify the client 120 of the address for recording the version number of each fragment in advance, and when the client 120 needs to determine the attribution status of the target data, it may first determine The segment to which the target data belongs is further determined to record the address of the version of the segment.

Step 202: the server 110 receives the data request, and obtains the data requested by the data according to the data request.

After receiving the data request, the server 110 can obtain the address sequence in the data request; the server 110 obtains the data stored at the address according to the order of multiple addresses in the address sequence. The manner in which the server 110 processes different address sequences is described below:

In the first type, the address sequence includes the address of the metadata state, the address of the metadata, and the address of the metadata state in order.

The server 110 first reads the first state according to the address of the metadata state, then reads the metadata according to the address of the metadata, and then reads the second state according to the address of the metadata state after obtaining the metadata.

It can be seen that when processing the address sequence, the server 110 reads the metadata status once before and after reading the metadata, and the read metadata status is the first status and the second status respectively.

For the server 110, the processing of the data by the server 110 itself or the interaction between the servers 110 will cause the metadata of the data to change. For example, the server 110 performs data migration (the data address of the metadata record will change), modification, and deletion (the metadata of the data may be deleted) within the server 110 . The processing of the data by the server 110 may lead to changes in the metadata of the data, that is, the server 110 may update the metadata of the data, resulting in a change in the status of the metadata. If the server 110 updates the metadata while the client 120 sends a data request to obtain the metadata, the metadata obtained by the client 120 from the server may be incomplete. In this embodiment of the present application, in order to prevent the client 120 from updating the metadata while the client 120 is obtaining the metadata from the server 110 (it can also be understood that the server 110 reads the metadata at the request of the client 120 ). In the address sequence carried in the data request, the address of the metadata state is also placed before and after the address of the metadata. In this way, in addition to obtaining the metadata from the server 110, the client 120 may also obtain the state of the metadata before and after reading the metadata from the server.

In the second type, the address sequence includes in order the address of the metadata state, the address of the metadata, the address of the metadata state, and the address of the attribution state of the target data.

The server 110 first reads the first state according to the address of the metadata state, then reads the metadata according to the address of the metadata, and after obtaining the metadata, reads the second state according to the address of the metadata state, and then reads the second state according to the address of the metadata state. The address of the attribution status of the target data reads the attribution status of the target data.

It can be seen that, when processing the address sequence, the server 110 not only reads the metadata, the first status, and the second status, but also reads the attribution status of the target data.

For the reasons for obtaining the first state and the second state, reference may be made to the foregoing description, which will not be repeated here.

For the server 110, the processing of the data by the server 110 itself or the interaction between the servers 110 will cause the server 110 where the data is located to change, for example, the shard to which the target data belongs is migrated from the server 110 to another The server 110, that is, the data ownership status changes. For example, data migration can be performed between the server 110 and other server 110, which will cause the ownership of the data to change from the server 110 to another server 110, and the server 110 itself can delete data, which will cause the data to no longer exist. Attributed server 110 . If the attribution status of the target data changes when the client 120 sends a data request to obtain the data request, the metadata obtained by the client 120 from the server may be incomplete. In this embodiment of the application, in order to prevent the client 120 from obtaining metadata from the server 110 (it can also be understood that the server 110 reads the metadata under the request of the client 120), the attribution status of the target data is in the Variety. The address sequence carried in the data request also includes the address of the data attribution status. In this way, in addition to acquiring metadata from the server 110, the client 120 can also acquire the attribution status of the read target data from the server.

In the third type, the address sequence includes the address of the first state, the address of the metadata, the address of the second state, the address of the belonging state of the target data, and the address of the target data in sequence. The data requested in the data request are metadata, the first state, the second state, the attribution state of the target data, and the target data.

The server 110 first reads the first state according to the address of the metadata state, then reads the metadata according to the address of the metadata, and after obtaining the metadata, reads the second state according to the address of the metadata state, and then reads the second state according to the address of the metadata state. The data attribution status reads the data attribution status, and then reads the target data according to the address of the target data.

It can be seen that, when processing the address sequence, the server 110 will not only read the metadata, the first state, the second state, and the data attribution state, but also read the target data.

The reasons for reading the first state, the second state, and the data attribution state can be referred to the foregoing description, and will not be repeated here.

The address sequence also carries the address of the target data, which is the address used by the client 120 to access the target data from the server 110 before. The client 120 may save the address of the target data when accessing the target data before. Since the address sequence carries the address of the target data, that is to say, the server 110 will also read the target data and feed it back to the client 120 device. After the client 120 determines the integrity of the metadata, It also determines the validity of the target data fed back by the server 110. If the target data is valid, the client 120 does not need to obtain the target data from the server 110 according to the metadata, but directly obtains the target data from the server 110 this time. Feedback target data is enough.

Fourth, the address sequence includes the address of the first state, the address of the metadata, the address of the second state, and the address of the target data in order. The data requested by the data request are metadata, first state, second state, and target data.

The server 110 first reads the first state according to the address of the metadata state, then reads the metadata according to the address of the metadata, and after obtaining the metadata, reads the second state according to the address of the metadata state, and then reads the second state according to the address of the metadata state. The address of the target data reads the target data.

It can be seen that, when processing the address sequence, the server 110 will not only read the metadata, the first state, and the second state, but also read the target data.

For the reasons for reading the first state, the second state, and the target data, reference may be made to the foregoing description, which will not be repeated here.

Step 203: The server 110 feeds back a data response to the client 120, and the data response carries the data read by the server 110.

After the server 110 reads the data according to the address sequence, it can feed back the read data to the client 120 .

When the address sequence includes the address of the first state, the address of the metadata, and the address of the second state in order. The data response includes the first state, metadata, and the second state.

When the address sequence includes the address of the first state, the address of the metadata, the address of the second state, and the address of the belonging state of the target data in order. The data response includes the first status, metadata and the second status, the attribution status of the target data.

When the address sequence includes the address of the first state, the address of the metadata, the address of the second state, the address of the belonging state of the target data, and the address of the target data in order. The data response includes a first status, metadata, a second status, an attribution status of the target data, and the target data.

When the address sequence includes the address of the first state, the address of the metadata, the address of the second state, and the address of the target data in order. The data response includes a first state, metadata, a second state, and target data.

Step 204: the client 120 receives the data response, and checks the integrity of the metadata according to the data response.

After receiving the data response, the client 120 may parse the data carried in the data response to determine the integrity of the metadata.

When the data response includes the first state, metadata, and the second state.

The client 120 can compare the first state with the second state. When it is determined that the first state and the second state are consistent, it means that the metadata has not changed before and after reading, and the metadata carried in the data response is complete. , the client 120 may request the server 110 to read the target data based on the metadata. When it is determined that the first state and the second state are inconsistent, it means that the metadata has changed before and after reading, and the metadata carried in the data response is incomplete, the client 120 can request the metadata from the server 110 again, and the client The terminal 120 may also use bilateral RDMA to interact with the server 110 to obtain the target data.

When the data response includes the first status, metadata and the second status, the attribution status of the target data.

The client 120 may compare the first status with the second status, and compare the attribution status of the target data recorded in the metadata with the attribution status of the target data carried in the data response. When the results of the above two comparisons are consistent, it means that the metadata obtained this time is complete and the attribution status of the target data has not changed. The client 120 can request the server 110 to read the target data based on the metadata. When at least one result of the above two comparisons indicates inconsistency, it means that the metadata obtained this time is incomplete. Afterwards, the client 120 may request the metadata from the server 110 again, and the client 120 may also use bilateral RDMA to interact with the server 110 to obtain the target data. When the attribution status of the target data is inconsistent, the client 120 can interact with the server 110 to determine the server 110 to which the target data belongs, and then the client 120 can interact with the server 110 to which the target data belongs to obtain the target data.

When the data response includes the first status, the metadata, the second status, the attribution status of the target data, and the target data.

The client 120 may compare the first status with the second status, and compare the data attribution status recorded in the metadata with the data attribution status carried in the data response. When the results of the above two comparisons are consistent, it means that the metadata obtained this time is complete, and the attribution status of the data has not changed, and the target data carried in the data response is valid, and the client 120 can directly obtain the data from the data response. Read target data. When at least one result of the above two comparisons indicates inconsistency, it means that the metadata obtained this time is incomplete. Afterwards, the client 120 may request the metadata from the server 110 again, and the client 120 may also use bilateral RDMA to interact with the server 110 to obtain the target data. When the attribution status of the target data is inconsistent, the client 120 can interact with the server 110 to determine the server 110 to which the target data belongs, and then the client 120 can interact with the server 110 to which the target data belongs to obtain the target data.

It should be noted that, if the client 120 has saved the metadata of the target data locally before, the client 120 will not only compare the first state with the second state, but also compare the data attribution state recorded in the metadata and the data carried in the data response. The data attribution status comparison can also compare the metadata of the target data stored locally with the metadata obtained this time. That is to say, three kinds of comparisons need to be performed in this case, and when the results of these three kinds of comparisons all indicate the same, it means that the metadata obtained this time is complete, and the client 120 can directly read the target data from the data response. When at least one of the three comparisons indicates an inconsistency, it means that the target data read this time is invalid, and the client 120 cannot directly use the target data in the data response. 110 reacquires the target data (for example, among the three types of comparisons, only the locally stored metadata of the target data is inconsistent with the metadata obtained this time), and the client 120 can also acquire the target data in the aforementioned manner.

When the data response includes the first state, metadata, second state, and target data.

The client 120 can compare the first state with the second state. When the comparison results indicate that they are consistent, it means that the metadata acquired this time is complete, and the target data carried in the data response is valid, and the client 120 can directly read the target data from the data response. When the result of the comparison indicates inconsistency, it means that the metadata obtained this time is incomplete. After that, the client 120 can request the metadata from the server 110 again. The client 120 may also use bilateral RDMA to interact with the server 110 to acquire the target data.

It should be noted that, if the client 120 has stored the metadata of the target data locally before, the client 120 can compare the metadata of the target data stored locally and the obtained metadata for comparison. That is to say, in this case, two comparisons need to be performed. When the results of the two comparisons indicate the same, it means that the metadata acquired this time is complete, and the client 120 can directly read the target data from the data response. When at least one of the results of these two comparisons indicates an inconsistency, it means that the target data read this time is invalid, and the client 120 cannot directly use the target data in the data response. 110 reacquires the target data (for example, only the locally stored metadata of the target data in the two comparisons is inconsistent with the metadata obtained this time), and the client 120 can also acquire the target data in the aforementioned manner.

It should be noted that, in the embodiment of the present application, the interaction between the client 120 and the server 110 in step 201 and step 203 may be based on unilateral RDMA. In practical applications, the embodiment of the present application does not limit the way in which the server 110 interacts with the client 120, any method that can bypass the processor of the server 110, and directly interacts between the client 120 and the network card of the server 110 is applicable In the embodiment of this application.

Based on the same inventive concept as the method embodiment, the embodiment of the present application also provides a data access device, which is used to execute the method executed by the server in the method embodiment shown in Figure 2 above, and related features can be found in The foregoing method embodiments will not be repeated here. As shown in FIG. 3 , the data access device 300 includes a receiving unit 301, an acquiring unit 302, and a sending unit 303;

The receiving unit 301 is configured to receive a data request from a client, the data request includes a sequence, and the sequence includes: an address of a first state, an address of metadata, and an address of a second state, wherein the first state and the second state are used to describe Whether the metadata changes, the address of the second state is the same as the address of the first state; the receiving unit 301 can execute the method executed by the server in step 201 as shown in FIG. 2 .

The obtaining unit 302 is configured to respectively obtain metadata, a first state before obtaining the metadata, and a second state after obtaining the metadata according to the sequence. The acquiring unit 302 may execute step 202 as shown in FIG. 2 .

The sending unit 303 is configured to send a data response to the client, where the data response includes metadata, a first state, and a second state. The sending unit 303 may execute step 203 shown in FIG. 2 .

In a possible implementation, the sequence further includes the address of the attribution state of the target data, the address of the attribution state of the target data is located after the address of the second state, the target data is the data described by the metadata, and the data response also includes the target For the attribution status of the data, after acquiring the metadata, the first status, and the second status, the acquisition unit 302 may also acquire the attribution status of the target data according to the address of the attribution status of the target data.

In a possible implementation, the sequence further includes the address of the attribution state of the target data and the address of the target data, the address of the attribution state of the target data is located after the address of the second state, and the address of the target data is at the attribution state of the target data After the address of the target data, the target data is the data described by the metadata; after acquiring the metadata, the first state, and the second state, the acquisition unit 302 can also acquire the attribution status of the target data according to the address of the attribution status of the target data, according to The address of the target data acquires the target data.

In a possible implementation manner, the acquiring unit 302 is deployed on a network card of the server.

Based on the same inventive concept as the method embodiment, the embodiment of the present application also provides a data access device, which is used to execute the method executed by the client in the method embodiment shown in Figure 2 above, and related features can be found in The foregoing method embodiments will not be repeated here. As shown in FIG. 4 , the data access device 400 includes a receiving unit 401 and a sending unit 403 . Optionally, a determining unit 402 may also be included.

The sending unit 403 is configured to send a data request to the server, the data request includes a sequence, and the sequence includes: an address of a first state, an address of metadata, and an address of a second state, wherein the first state and the second state are used to describe Whether the metadata changes, the address of the second state is the same as the address of the first state. The sending unit 403 may execute step 201 as shown in FIG. 2 .

The receiving unit 401 is configured to receive a data response sent by the server, where the data response includes metadata, a first state, and a second state. The receiving unit 401 may execute step 203 shown in FIG. 2 .

In a possible implementation manner, the determining unit 402 may receive the data access response, and determine whether the obtained first state is consistent with the second state.

In a possible implementation manner, the attribution status of the target data is recorded in the metadata, and the determining unit 402 may also determine whether the attribution status of the target data is consistent with the attribution status of the target data recorded in the metadata.

In a possible implementation manner, if the data response carries the first state, the second state, and metadata, the determining unit 402 obtains the target from the server according to the metadata when it is determined that the first state is consistent with the second state. data.

If the data response carries the first status, the second status, metadata and the attribution status of the target data, the determination unit 402 determines that the first status is consistent with the second status, and the attribution status of the target data is the same as the target data recorded in the metadata. In the case of consistent attribution status, the target data is obtained from the server according to the metadata.

If the data response carries the first status, the second status, metadata and the attribution status of the target data, the determining unit 402 determines that the first status is inconsistent with the second status, or the attribution status of the target data is different from the target data recorded in the metadata. In the case of inconsistent attribution status, the client can determine that the metadata is invalid, and the client can request the metadata from the server again, or interact with the server to determine the server to which the target data belongs.

In a possible implementation, the sequence further includes the address of the attribution state of the target data and the address of the target data, the address of the attribution state of the target data is located after the address of the second state, and the address of the target data is at the attribution state of the target data After the address of , the target data is the data described by the metadata; the data response includes the attribution status of the target data and the target data.

In a possible implementation, the determining unit 402 obtains the target data from the server according to the metadata when determining that the metadata stored locally by the client is inconsistent with the metadata.

In a possible implementation manner, when sending the data request to the client, the sending unit 403 may send the data request to the network card of the server based on RDMA.

It should be noted that the division of units in the embodiment of the present application is schematic, and is only a logical function division, and there may be another division manner in actual implementation. Each functional unit in the embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

The above-mentioned embodiments may be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, the above-described embodiments may be implemented in whole or in part in the form of computer program products. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on the computer, the processes or functions according to the embodiments of the present invention will be generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center that includes one or more sets of available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media. The semiconductor medium may be a solid state drive (SSD).

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the scope of the application. In this way, if these modifications and variations of the application fall within the scope of the claims of the application and their equivalent technologies, the application also intends to include these modifications and variations.

Claims

A data system, characterized in that the data system includes a client and a server;

The client is configured to send a data request to the server, the data request includes a sequence, and the sequence includes: an address of the first state, an address of metadata, and an address of the second state, wherein the The first state and the second state are used to describe whether the metadata changes, and the address of the second state is the same as the address of the first state;

The server is configured to receive the data request, respectively obtain the metadata, the first state before obtaining the metadata, and the second state after obtaining the metadata according to the sequence ; sending a data response to the client, where the data response includes the metadata, the first state, and the second state.
The system of claim 1, wherein the sequence further includes an address of a home state of target data, the address of the home state following the address of the second state, the target data being the element The data described by the data, the data response also includes the attribution status, and after the server obtains the metadata, the first status, and the second status, it is further used to:

The attribution status is acquired according to the address of the attribution status.
The system according to claim 1 or 2, wherein the client is configured to receive the data access response and determine whether the first state is consistent with the second state.
The system according to claim 3, wherein the attribution status of the target data is recorded in the metadata, and the client is further used for:

Determine whether the attribution status is consistent with the attribution status of the target data recorded in the metadata.
The system according to claim 4, wherein the client is configured to: record the target in the first state consistent with the second state, and in the attribution state and the metadata If the attribution status of the data is consistent, the target data is acquired from the server according to the metadata.
The system of claim 1, wherein the sequence further includes an address of a home state of the target data and an address of the target data, the address of the home state is located after the address of the second state, the The address of the target data is located after the address of the home state, and the target data is the data described by the metadata; the data response also includes the address of the home state and the target data; the server After obtaining the metadata, the first state, and the second state, it is further used to:

The attribution status is acquired according to the address of the attribution status, and the target data is acquired according to the address of the target data.
The system according to claim 6, wherein the client is further configured to: when it is determined that the metadata stored locally is inconsistent with the metadata, from the server according to the metadata Get the target data.
The system according to any one of claims 1-7, wherein the client is specifically used for:

Sending the data request to the network card of the server based on remote direct memory access (RDMA).
The system according to claim 8, wherein the server is specifically used for:

The network card respectively acquires the metadata, the first state before acquiring the metadata, and the second state after acquiring the metadata according to the sequence; and sends the data to the client response.
A data access method, characterized in that the method comprises:

The server receives a data request from the client, the data request includes a sequence, and the sequence includes: an address of a first state, an address of metadata, and an address of the second state, wherein the first state and the The second state is used to describe whether the metadata has changed, and the address of the second state is the same as the address of the first state;

The server respectively acquires the metadata, the first state before acquiring the metadata, and the second state after acquiring the metadata according to the sequence;

The server sends a data response to the client, and the data response includes the metadata, the first state, and the second state.
The method of claim 10, wherein the sequence further includes the address of the home state of the target data, the address of the home state is located after the address of the second state, the target data is the element The data described by the data, the data response also includes the attribution status, and after the server obtains the metadata, the first status, and the second status, it further includes:

The server obtains the attribution status according to the address of the attribution status.
The method of claim 10, wherein the sequence further includes an address of a home state of the target data and an address of the target data, the address of the home state is located after the address of the second state, the address of the target The address of the data is located after the address of the attribution status, and the target data is the data described by the metadata; the data response also includes the address of the attribution status and the target data; the server obtains After the metadata, the first state, and the second state, further include:

The attribution status is acquired according to the address of the attribution status, and the target data is acquired according to the address of the target data.
The method according to any one of claims 10-12, wherein the server respectively acquires the metadata and the first state before acquiring the metadata according to the sequence, including:

The network card of the server respectively obtains the metadata, the first state before obtaining the metadata, and the second state after obtaining the metadata according to the sequence.
A data access device, characterized in that the device includes a receiving unit, an acquiring unit, and a sending unit:

The receiving unit is configured to receive a data request from a client, the data request includes a sequence, and the sequence includes: an address of the first state, an address of metadata, and an address of the second state, wherein the second A state and the second state are used to describe whether the metadata changes, and the address of the second state is the same as the address of the first state;

The acquiring unit is configured to respectively acquire the metadata, the first state before acquiring the metadata, and the second state after acquiring the metadata according to the sequence;

The sending unit is configured to send a data response to the client, where the data response includes the metadata, the first state, and the second state.
The apparatus of claim 14, wherein the sequence further includes the address of the home state of the target data, the address of the home state is located after the address of the second state, the target data is the The data described by the data, the data response also includes the attribution status, and after the acquisition unit acquires the metadata, the first status, and the second status, it is further configured to:

The attribution status is acquired according to the address of the attribution status.
The apparatus according to claim 14, wherein the sequence further comprises an address of a home state of the target data and an address of the target data, the address of the home state is located after the address of the second state, the The address of the target data is located after the address of the attribution status, and the target data is the data described by the metadata; the data response also includes the attribution status of the target data and the target data; the acquiring After obtaining the metadata, the first state, and the second state, the unit is further configured to:

The attribution status is acquired according to the address of the attribution status, and the target data is acquired according to the address of the target data.
The device according to any one of claims 14-16, wherein the acquiring unit is deployed on a network card.
A computer storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to make a computer execute the method according to any one of claims 10 to 13.
A computing device, characterized in that the computing device includes a network card, and the network card executes computing instructions for causing the computer to execute the method according to any one of claims 10 to 13.