CN114625762A

CN114625762A - Metadata acquisition method, network equipment and system

Info

Publication number: CN114625762A
Application number: CN202011360768.1A
Authority: CN
Inventors: 余博伟; 蒋凡璐; 罗四维; 游俊
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-11-27
Filing date: 2020-11-27
Publication date: 2022-06-14

Abstract

The application provides a metadata acquisition method, which is applied to network equipment, wherein the network equipment is used for connecting a client and a server, and metadata information of partial data of application is cached in a memory of the server, and the method comprises the following steps: receiving an access request of a client for sending target metadata; acquiring metadata information at a position specified by the access request in a memory; comparing the obtained metadata information with the information of the target metadata to determine whether the target metadata is stored in the memory; when the target metadata is stored in the memory, the target metadata is obtained and transmitted to the client, when the target metadata is not found in the memory, the network card can continuously request the processor of the server to inquire the target metadata without informing the client, and the client sends an inquiry request to the processor of the server once again, so that the load of the client is reduced, the network bandwidth is saved, and the inquiry delay of the metadata is reduced.

Description

Metadata acquisition method, network equipment and system

Technical Field

The present application relates to the field of communications, and in particular, to a metadata obtaining method, a network device, and a system.

Background

With the continuous development of science and technology, mass data generated in the information explosion era has penetrated into every industry and business function field, and a distributed storage system is widely used due to high availability, high reliability and high access efficiency. In the distributed storage system, data is stored in a plurality of servers in a scattered manner, and address information of the data is written into metadata of the data, so that each time a user uses a client to read certain data from a server, the client needs to acquire the metadata of the data from the server, then determine the address information of the data according to the metadata, and acquire the data from the address information.

However, the metadata stored in the server generally has a two-layer storage structure, the first layer is a cache (cache) layer, and the client can send a single-side operation request to the server to read the metadata stored in the cache layer; the second layer is a full index layer, and the client can initiate a bilateral operation request to the server to read the metadata stored in the full index layer. Based on this, in the process of reading metadata from the server by the current client, the client usually first initiates a single-side operation request for accessing the target metadata to the server, and after receiving the metadata acquired by the server in response to the single-side operation request, the client determines whether the received metadata is the target metadata, and then initiates a double-side operation request for accessing the target metadata to the server under the condition that the metadata is not the target metadata, so as to obtain the target metadata.

Disclosure of Invention

The application provides a metadata acquisition method, network equipment and a system, which can solve the problems of high processing pressure of a client, high communication overhead among client amplitudes and low metadata reading efficiency in a metadata reading process.

In a first aspect, a metadata obtaining method is provided, where the method may be applied to a network device, the network device is used to connect a client and a server, the client runs an application, and a memory of the server caches metadata information of partial data of the application, and the method may include the following steps: the network equipment receives an access request of target metadata sent by a client, wherein the access request carries information of the target metadata and a position designated by the access request, the network equipment acquires the metadata information from the position designated by the access request in a memory, then compares the acquired metadata information with the information of the target metadata to determine whether the target metadata is stored in the memory, and when the target metadata is stored in the memory, acquires the target metadata and transmits the target metadata to the client.

In the method described in the first aspect, the client first sends an access request for the target metadata to the network device of the server, the network device of the server queries the target metadata in the memory of the server, and when the target metadata is stored in the memory, the network device may read the target metadata and then return the target metadata to the client, so that the client does not need to determine whether the received data is the target metadata, thereby reducing the processing pressure of the client and improving the metadata reading efficiency.

In a possible implementation, the method may further include the steps of: when the target metadata is not stored in the memory, the network equipment sends the information of the target metadata to the processor of the server, so that the processor of the server acquires the target metadata according to the information of the target metadata, and the network equipment receives the target metadata sent by the processor of the server and transmits the target metadata to the client.

By implementing the implementation manner, when the target metadata is not stored in the memory, the server does not return non-target metadata to the client, and the client does not need to confirm whether the received metadata is the target metadata, but notifies the processor of the server of an access request of the target metadata through the network device, and the processor of the server queries the target metadata in the memory and the storage medium globally, so that the processing pressure of the client is reduced, meanwhile, the communication times between the client and the server are reduced, the network bandwidth is saved, and the query delay of the metadata is reduced.

In a possible implementation manner, the metadata information cached in the memory includes an identifier of the metadata and the metadata or an identifier of the metadata and metadata acquisition information, and the metadata acquisition information is used for acquiring the metadata. Optionally, the metadata information is stored in the memory in the form of a key-value pair, where the identifier of the metadata is a key (key) in the key-value pair, and the metadata or the metadata acquisition information is a value (value) in the key-value pair.

Specifically, the value in the metadata information may be the metadata itself, or may also be information for obtaining the metadata, such as key of metadata a is 1 and value is metadata a, or key of metadata a is 1 and value is a storage address of metadata a, and the meaning of value in the key-value pair may be determined according to actual circumstances, which is not limited in this application.

By implementing the implementation mode, the metadata information cached in the memory is stored in a key value pair mode, and the storage mode can be used for quickly inquiring in a single key mode, a combination key mode, a range key mode and the like, so that the metadata information can be read more simply and quickly.

In a possible implementation manner, metadata information of part of the applied data is cached in a memory of the server by using a hash table, and a hash value obtained by performing hash operation on a data identifier of the metadata by the server is a storage location of the metadata information in the hash table. The position appointed by the access request is a hash value obtained by the client side performing hash operation on the target metadata, wherein the hash function used by the client side for performing hash operation is the same as the hash function used by the server side. Before the client sends the access request of the target metadata to the server, the server can send the hash function to the client, for example, after the client and the server are connected, the server sends the hash function used for storing the metadata information to the client, and if the hash function used by the server is changed, the server can send the changed hash function to the client.

In a specific implementation, the step of comparing the obtained metadata information with the information of the target metadata to determine whether the target metadata is stored in the memory may specifically include the following steps: the network equipment compares the mark of the target metadata with the key in the key value pair, determines whether the target metadata is stored in the memory, acquires the target metadata and transmits the target metadata to the client when the target metadata is stored in the memory, wherein when the mark of the target metadata is consistent with the key, the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata, and the network equipment acquires the target metadata according to the acquisition information of the target metadata and transmits the target metadata to the client.

By implementing the implementation mode, the server can store the metadata information at the storage position corresponding to the hash value after performing hash operation on the key of the metadata to obtain the hash value, and send the hash function used by the hash operation to the client in advance, so that the client can predict the storage position of the target metadata in the server through the hash function, when the client initiates an access request of the target metadata, the predicted storage position can be carried in the access request, the server reads the metadata information from the position specified by the access request, if the key in the metadata information is consistent with the key of the target metadata, the metadata information is the information of the target metadata, the target metadata can be obtained according to the read metadata information, otherwise, the processor can be informed to perform bilateral reading operation, so that the client does not need to confirm the received metadata any more, the processing pressure of the client is reduced, meanwhile, the communication times between the client and the server are reduced, the network bandwidth is saved, and the query time delay of the metadata is reduced.

In a possible implementation manner, the access request of the target metadata may also carry version information of the target metadata. The server side can also comprise a version number of the metadata in the metadata information acquired from the position specified by the access request, when the information of the target metadata is compared with the metadata information to judge whether the information of the target metadata is consistent or not, the network equipment of the server side can firstly compare the key in the metadata information acquired from the position specified by the access request with the key in the target metadata information, if the key and the metadata information are consistent, the version information in the metadata information acquired from the position specified by the access request is compared with the version information in the target metadata information, and if the key and the version information are also consistent, the target metadata is read from the memory according to the metadata information, otherwise, the network equipment informs the server side processor to carry out bilateral operation and reads the target metadata according to the key of the target metadata.

By implementing the implementation mode, the access of the metadata of the specific version can be realized, the occurrence of one condition of returning the target metadata of the wrong version is avoided, and the accuracy of reading the metadata is improved.

Optionally, the access request may further include a data length L, where the read length is used for the server to read a piece of data with the data length L from the location specified by the access request.

Optionally, the access request may further include a write address, where the write address is an address of the client where the target metadata is written, so that after the network device at the server acquires the target metadata, the target metadata may be written in the write address through RDMA single-sided operation, so that the target metadata is written without participation of the CPU at the server, and the read efficiency of the target metadata is improved.

Optionally, the access request may further include a descriptor of the access request, where the descriptor of the read request is used for the server to determine a meaning of each field in the read request, such as which field is a key, which field is a location specified by the access request, and which field is a data length, for example, the read request may be a segment of a message, the descriptor may be located in a header of the message, and after the server reads the descriptor from the header of the message, the server determines that the 1 st to 4KB fields are descriptors, the 5 th to 8KB fields are keys, the 9 th to 16 th fields are locations specified by the access request, and the 17 th to 24 th fields are data lengths. It should be understood that the above examples are only used for illustration, the access request may also include other information, such as a key or a check code for improving data transmission security performance, and the present application is not limited in particular, and the descriptor may also be in other expressions, such as using a line feed, a space, a colon, etc. as separators to split multiple fields in the access request, and the present application is not limited in this respect.

In a specific implementation, the network device may be a network card (nic) having a Remote Direct Memory Access (RDMA) function, such as an RDMA enabled nic (RNIC), or a network card or a switch supporting an InfiniBand function (IB), so that the network device may implement the above scheme provided by the present application through a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), and other programmable chips.

In a second aspect, a computer system is provided, where the computer system may include a client and a server, and the server includes a network device, the network device is configured to connect the client and the server, where the client is configured to run an application, and the server is configured to cache metadata information of partial data of the application in a memory, and the network device is configured to obtain the metadata information from a location specified by an access request of the client to access target metadata, determine whether the target metadata is stored in the memory by comparing the metadata information with information of the target metadata, and when the target metadata is stored in the memory, obtain the target metadata and transmit the target metadata to the client, where the access request of the target metadata includes information of the target metadata.

In a possible implementation manner, the network device is further configured to send information of the target metadata to the processor of the server when the target metadata is not stored in the memory; the processor of the server is used for acquiring the target metadata according to the information of the target metadata; the network device is also used for transmitting the target metadata acquired by the processor to the client.

In a possible implementation manner, the metadata information cached in the memory includes an identifier of the metadata and the metadata or an identifier of the metadata and metadata acquisition information, and the metadata acquisition information is used for acquiring the metadata.

In a possible implementation manner, the metadata information is stored in the memory in the form of Key Value pairs, where the identifier of the metadata is a Key in a Key Value pair, and the metadata or the metadata acquisition information is a Value in the Key Value pair.

In a possible implementation manner, metadata information of part of the applied data is cached in a memory of the server by using a hash table, and a hash value obtained by performing hash operation on a data identifier of the metadata by the server is a storage location of the metadata information in the hash table.

In a possible implementation manner, the location specified by the access request is a hash value obtained by performing hash operation on the target metadata by the client, where a hash function used by the client for performing the hash operation is the same as a hash function used by the server.

In a possible implementation manner, the information of the target metadata includes an identifier of the target metadata, the network device is configured to compare the identifier of the target metadata with a key in the key value pair, when the identifier of the target metadata is consistent with the key, the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is acquisition information of the target metadata, and the network device acquires the target metadata according to the acquisition information of the target metadata.

In a third aspect, a network device is provided, which is applied to a computer system, where the computer system includes a client and a server, the network device is used to connect the client and the server, the client runs an application, and a memory of the server caches metadata information of partial data of the application, and the network device includes: the receiving unit is used for receiving an access request of target metadata sent by a client, wherein the access request of the target metadata carries information of the target metadata and a position specified by the access request; an information acquisition unit for acquiring metadata information at a position specified by the access request in the memory; a determining unit for comparing the acquired metadata information with information of target metadata to determine whether the target metadata is stored in the memory; and the metadata acquisition unit is used for acquiring the target metadata and transmitting the target metadata to the client when the target metadata is stored in the memory.

In a possible implementation, the network device further comprises a sending unit,

and the sending unit is used for sending the information of the target metadata to a processor of the server side when the target metadata is not stored in the memory, so that the processor can obtain the target metadata according to the information of the target metadata and transmit the target metadata to the client side.

In a possible implementation manner, the metadata information is stored in the memory in the form of a Key Value pair, where the identifier of the metadata is a Key in the Key Value pair, and the metadata or the acquisition information of the metadata is a Value in the Key Value pair.

In a possible implementation manner, the information of the target metadata includes an identifier of the target metadata, and the determining unit is configured to compare the identifier of the target metadata with a key in a key value pair, and determine whether the target metadata is stored in the memory; the metadata acquisition unit is used for acquiring the target metadata according to the acquisition information of the target metadata and transmitting the target metadata to the client when the identifier of the target metadata is consistent with the key and the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata; and the sending unit is used for sending the information of the target metadata to a processor of the server side when the identifier of the target metadata is inconsistent with the key, so that the processor can acquire the target metadata according to the information of the target metadata and transmit the target metadata to the client side.

In a fourth aspect, a network device is provided, which includes a processor and a communication interface, the communication interface is configured to receive an access request from a client to access target metadata, and the processor is configured to execute the method described in the first aspect to process the access request.

In a fifth aspect, a computer program product is provided, comprising a computer program which, when read and executed by a computing device, implements the method as described in the first aspect.

A sixth aspect provides a computer-readable storage medium comprising instructions which, when executed on a computing device, cause the computing device to carry out the method as described in the first aspect.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.

FIG. 1 is a schematic diagram of an architecture for a computer system;

FIG. 2 is a schematic diagram of a hardware configuration of a computer system;

FIG. 3 is a diagram illustrating a data structure of metadata;

FIG. 4 is a flow chart illustrating steps of a metadata obtaining method provided herein;

FIG. 5 is a schematic flow chart diagram of another data acquisition method provided herein;

fig. 6 is a schematic structural diagram of a network device provided in the present application;

fig. 7 is a schematic hardware structure diagram of a network device provided in the present application.

Detailed Description

In order to facilitate understanding of the technical solutions of the present invention, first, some terms related to the present invention are explained. It is to be understood that the terminology used in the description of the embodiments section of the present application is for the purpose of describing particular embodiments of the present application only and is not intended to be limiting of the present application.

Unilateral operation: the unilateral operation generally means that a CPU of a receiving party does not participate in data reading operation, and a network card of the receiving party reads data from a memory, specifically, during the unilateral operation, after a source address and a destination address of the data are determined by a sending side, the CPU of the receiving side does not need to sense the communication, and data reading and writing are completed through the network cards of the sending side and the receiving side, which is generally applicable to batch data transmission. Specifically, for example, after the client a and the server B are connected, the server B stores the data X in a segment of memory space, and sends address information VB of the memory space to the client a, when the client a reads the data X of the server B, the client a can encapsulate the address VA (destination address) locally used for storing the data X and the address information VB (source address) in a single-side read request, and send the request to the network card of the server B, and the network card of the server B writes the data X in the VB into the VA in response to the single-side read request, and the whole process does not need participation of CPUs of both sides, so that the single-side operation can reduce the CPU occupancy rates of the client and the server, improve the communication efficiency, and achieve the purpose of high-bandwidth and low-delay communication. Illustratively, the unilateral operation may be an RDMA unilateral operation.

Bilateral operation: a bilateral operation generally refers to a work that is performed by the CPU of the receiving party to read and write data, such as an RDMA bilateral operation. Specifically, during bilateral operation, the receiving-side CPU needs to sense this bilateral operation to complete transmission and reception, which is generally applicable to transmission and reception of control messages, and after executing corresponding operation steps, the receiving-side CPU returns a completion message to the transmitting side in response to the received control messages. Although the CPUs of the two parties participate in the whole process, the zero copy of the data transmitted by the cross-node data is realized without traversing a multi-layer network protocol stack.

Key-value pair (key, value): the key-value pair is a character string, in the storage field, metadata may be stored in a key-value pair (K, V), where K represents a key (key), the key may be a name or an identifier of the metadata, V represents a value (value) corresponding to the key, the value may be defined according to service requirements, for example, the value may be metadata itself, or location information of the metadata, or a storage location of the location information of the metadata in a hash table, and each key corresponds to at least one value. For example, K is the name of the metadata x, and V is the storage address of the metadata x. The storage mode can simply and quickly obtain the data required by the service through single key query, combined key query and range key query.

To facilitate understanding of the embodiments of the present application, first, a brief description is given of an application scenario "metadata reading" related to the present application.

Modern internet has entered the big data era, a large amount of data is generated every day, and a traditional network computer system adopts a centralized storage server to store all data, so that the performance of the storage server becomes the bottleneck of the system performance, and the requirement of large-scale storage application cannot be met. Therefore, the distributed storage system has come up, the distributed storage system means that data is stored on a plurality of independent devices in a scattered manner, a plurality of storage servers are used for sharing storage load, and the metadata server is used for positioning the position information of the stored data.

Fig. 1 is a schematic diagram of an architecture of a computer system 100, and as shown in fig. 1, the computer system 100 may include a client (client)100 and a server 200, wherein a network 300 is connected between the client 100 and the server 200, the network 300 is a network suitable for RDMA technology, such as Ethernet (Ethernet), and the number of the clients 100 and the number of the servers 200 may be one or more. It should be understood that fig. 1 illustrates 2 clients (client 100a and client 100b, respectively) and 2 servers (server 200a and server 200b, respectively), but the number of clients 100 and servers 200 is not limited in this application.

The client 100 is configured to provide an interactive interface for a user, and send a data reading request to the server 200 through the RDMA network 300 according to various operations of the user in the interactive interface, where the specific form of the client 100 may be a web browser, or may be a locally running application (application), and the application is not limited in particular. Moreover, the client 100 is flexible to deploy, and can be deployed on a single physical server, such as an X86 server, a Bare Metal Server (BMS), or the like; the method can also be deployed on terminals such as smart phones, tablet computers and the like; the virtual machine can also be deployed in a Virtual Machine (VM) implemented based on a general physical server in combination with a Network Function Virtualization (NFV) technology, where the VM refers to a complete computer system that has a complete hardware system function and runs in a completely isolated environment, such as a virtual machine in a cloud data center, and is simulated by software; the client and the server may also be deployed on the same physical server, and the present application is not limited specifically.

The server 200 is a cluster formed by a plurality of storage devices, and the storage devices may be used for storing data and may also be used for storing metadata (metadata) of the data, and the present application does not limit the content of the data stored in the server 200. The metadata here refers to data (data about data) describing the data, and mainly describes attribute information (property) of the data, and is used for supporting functions such as indicating a storage location, history data, resource search, file record, and the like. Specifically, the metadata of the data may include: the access right of the data, the file owner, the location information of the data, and the like, wherein the location information may include the address of the server 200 where the data is located, the partition of the storage medium where the data is located, the start position on the storage medium, the length of the data, the index information of the data, and the like. In short, a user needs to read data in one server 200, and then the user must obtain metadata of the data to locate the data and then obtain the data from the location.

In a specific implementation, the server 200 may be an entity device or a virtual device. When the server 200 is a physical device, the server 200 may have a processor, a hard disk, a memory, a network card, a system bus, and the like. When the server 200 is a virtual device, the server 200 may simulate, through software, a computing device having a complete hardware system function and running in a completely isolated environment, for example, the server 200 may be a virtual machine in a cloud data center, where the virtual machine runs an application on the computing device, and may simulate a standard PC environment based on x86, and this environment, as with a real computing device, includes one or more devices of a chipset, a CPU, a memory, a graphics card, a sound card, a network card, a floppy drive, a hard disk, an optical drive, a serial port, a parallel port, a USB controller, a Small Computer System Interface (SCSI) controller, and the like. In some application scenarios, the server 200 may also exist in other forms, for example, in a data storage scenario suitable for an enterprise, and the server 200 may exist in a form of a storage array or a hard disk frame, which is not specifically limited in this application.

It should be noted that each server may store only data, only metadata, or both data and metadata, and the data and the metadata of the data may be located on the same server or on different servers, which is not limited in this application. For example, as shown in fig. 1, the server 200a stores data b1, data b2, and data b3, and also stores metadata a, and the server 200b stores data a, data b4, and data c, and also stores metadata b, where metadata a is metadata of data a and describes a storage location of data a, metadata b is metadata of data b1 to data b5 and describes storage locations of data b1 to data b5 (assuming that data b is fragmented into data b1 to data b 5).

In the computer system shown in fig. 1, the flow of steps for the user to read the data a through the client 100a can be divided into the following two steps, step 1: the client 100a initiates a reading request of target metadata a to the server to obtain the metadata a; step 2: the client 100a determines that the server where the data a is located is 200c according to the metadata a, and initiates a read request for reading the data a to the server 200c to obtain the data a.

It should be understood that, in step 1, since the server 200 includes at least one storage device, when the client 100 sends a read request of the target metadata to the server 200, one storage device in the server 200 may receive the read request, and then determine whether the target metadata exists in the local device according to a key of the target metadata in the read request, if the target metadata exists in the local device, process the read request, and if the target metadata does not exist in the local device, forward the read request to the target storage device where the target metadata exists for processing. The storage device that first receives the read request may be a default storage device, may also be a storage device closest to the geographical location of the client 100, and may also be a random storage device, which is not specifically limited in this application.

The storage device may identify the target storage device storing the target metadata in various ways, and optionally, the storage device may determine the target storage device storing the target metadata by taking a modulus between the key and the total number of the storage devices, where the modulus may be a remainder obtained by dividing two numbers, for example, there are 3 storage devices, the instance IDs are 0, 1, and 2, and the key of the metadata a0 is 90, so that when the storage device receives a write request of the metadata, the metadata a0 may be stored on the storage device with the instance ID of 0, if the key of the metadata a1 is 88, the data a1 may be stored on the storage device with the instance ID of 1, if the key of the metadata a1 is 86, the data a1 may be stored on the storage device with the instance ID of 2, and so on, the metadata is stored. Similarly, when the storage device receives a read request of the target metadata, the storage device may obtain an instance ID of the storage device after modulo the key of the target metadata in the read request, and each storage device may store a network address corresponding to each instance ID, thereby determining the network address of the target storage device storing the target metadata, and forwarding the read request to the network address for the storage device to read the metadata.

Optionally, the storage device may also determine a target storage device storing the target metadata by reading an index, and the index between the storage devices is periodically synchronized, where each storage device maintains an index, and the index may contain multiple pieces of routing information, and each piece of routing information includes a key and an instance ID of the storage device where the key is located. Specifically, when metadata a (key k0) is written into storage device Y, storage device Y may create a route a ' in the index maintained by itself, where the route a ' records that the instance ID corresponding to "key k 0" is "0", and then storage device Y synchronizes the route a ' to the index maintained by each storage device. In this way, when the key carried by the access request received by any storage device is k0, the storage device may obtain, according to the information recorded by the route a' in the index, that the instance ID corresponding to "key ═ k 0" is "0", that is, the storage device storing the metadata a (the key is k0), whose instance ID is 0, and then confirm the network address of the target storage device storing the target metadata according to the network address corresponding to each instance ID, and forward the read request to the network address for reading the metadata.

It should be noted that the above two methods for determining a storage device storing target metadata are only used for illustration, and the application does not limit the specific manner in which the storage device determines the target storage device storing the target metadata.

In summary, each time a user uses a client to read a certain data a from a server, the client needs to first obtain metadata of the data a from the server, then determine a storage location of the data a according to the metadata, and finally obtain the data a from the storage location. The application scenario applicable to the scheme provided by the application is the application scenario when the client reads the metadata (step 1), and under the scenario, the scheme provided by the application can improve the reading speed of the metadata, so that the efficiency of reading the data of the distributed storage system is improved, and the use experience of a user is improved.

Fig. 2 is a schematic diagram of the hardware structure of the computer system 100. Illustratively, the client 100 may include a processor 120, a storage 110 and a network device 130, and the server 200 may include a processor 220, a memory 230, a storage medium 240 and a network device 210, it should be understood that the unit modules of the client 100 and the server 200 may have various partitions, for example, the client 100 and the server 200 may further include a communication module, an expansion interface, and the like, and fig. 2 is only an exemplary partition, and the present application is not limited in particular. Furthermore, the position relationship between the devices and the modules shown in fig. 2 is not limited in any way, for example, in the server 200a shown in fig. 2, the storage medium is disposed outside the server 200a, in other cases, the storage medium may also be an internal memory of the server, and the present application is not limited in particular.

The processor 120 of the client 100 and the processor 220 of the server 200 may be implemented by an application-specific integrated circuit (ASIC), or a Programmable Logic Device (PLD), which may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.

The Memory 110 of the client 100 may include a Volatile Memory (Volatile Memory), such as a Random Access Memory (RAM); the Memory 1030 may also include a Non-Volatile Memory (Non-Volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, HDD), a Solid-State Drive (SSD), and the like, and may also include a combination of the above types, which is not limited in this application.

The memory 230 of the server 200 may be a volatile memory (volatile memory), a Random Access Memory (RAM), a Dynamic Random Access Memory (DRAM), a Static Random Access Memory (SRAM), a Synchronous Dynamic Random Access Memory (SDRAM), a Double Data Rate (DDR), a read-only memory (ROM), a cache, or the like, and may further include a combination of the above categories, which is not limited in this application.

The storage medium 240 of the server 200 may be all carriers for storing data, such as a floppy disk, an optical disk, a Solid State Disk (SSD), a flash memory, a storage medium, a memory stick, and the like, and the present application is not particularly limited. The storage medium 240 is used for persistent storage of data in the memory 230 of the server 200. It should be noted that the number of storage media of the server 200 may be one or more, and the present application is not limited in particular.

The network device 130 of the client 100 and the network device 210 of the server 200 are network cards (nic), such as RNIC, which may support RDMA function, and may also be network cards or switches of IB, which is not specifically limited in this application. The network device is used for connecting the client 100 and the server 200 to realize communication therebetween. Specifically, when the client 100 reads the metadata a stored in the server 200, the network device 130 of the client 100 may send an access request of the metadata a to the network device 210 of the server 200, and the network device 210 of the server 200 may read the metadata a according to the access request and return the metadata a to the network device 130 of the client 100.

The data structure of the metadata stored in the server 200 is explained below.

As shown in fig. 3, the metadata in the server 200 is generally split into two layers of data structures, the first layer is a Cache layer, the client can quickly read the metadata of the data structure of the Cache layer through a single-side operation, the Cache layer generally forms a hash table by a hash (hash) method in a manner of key value pairs for storing in the memory, the hash method can store the metadata in the memory according to a set hash function and a method for handling conflicts, specifically, when the server stores the metadata of the data structure of the Cache layer, the server can first use the hash function to perform hash operation on the key of the metadata to obtain a hash value V, and then store the metadata according to the hash value V, for example, the hash value V is used as a storage address of the metadata to store the metadata and the key value pairs, or the hash value V is the metadata itself, or maps the hash value V and the memory address of the server, the metadata and key-value pairs are stored in the memory address ADD0 corresponding to the hash value V. If other data is written at the hash value V of the metadata, the other data and the metadata may be processed by using a method for handling a conflict, for example, the other data is written to a storage medium for persistent storage, and the metadata and the key value pair are written at the hash value V.

For example, as shown in fig. 3, when storing metadata 1 with a key K1, the server may first use a hash function f (x) to determine that a value V1 corresponding to the key K1 is f (K1), obtain a key-value pair (K1, f (K1)) of the metadata 1, assume that mapping relationships exist between multiple hash addresses and a continuous memory address set or interval, for example, ADD corresponds to the memory address 1 for hash value 1, ADD corresponds to the memory address ADD2 for hash value 2, and so on, so as to obtain a memory address ADDf (K1) corresponding to hash value f (K1), and the server may store the key-value pair of the metadata 1 and the metadata 1 at the memory address ADDf (K1). Meanwhile, if metadata 2 is already stored at the memory address ADDf (K1), if the key of metadata 2 and the key of metadata 1 are both K1, it indicates that metadata 1 is a modified version of metadata 2, and then metadata 1 may be overwritten by metadata 2; if the key of metadata 2 is K2, which is different from the key K1 of metadata 1, metadata 2 written to the memory address ADDf (K1) may be written to the storage medium 240 for persistent storage, and the key-value pairs of metadata 1 and metadata 1 may be written to the memory address ADDf (K1). It should be understood that the above examples are used for illustrating that the method for handling the conflict may also be a chain address method, an open address method, etc., and the method for handling the conflict is not limited in this application.

Similarly, when the server reads the target metadata of the cache layer data structure, the network card of the server 200 may first obtain the key value pair of the target metadata according to the key K of the target metadata, then obtain the metadata from the server memory according to the hash value V in the key value pair, and return the metadata to the client. Therefore, when the metadata of the data structure of the cache layer is queried, the CPU of the server does not need to participate in the query process, and the client side can read the metadata of the data structure of the cache layer rapidly through unilateral operation, so that the reading speed is high.

The second layer is a full index layer, and the client can read the metadata of the data structure of the full index layer through bilateral operation. The metadata is stored in the full index layer by a data structure of a search tree (search tree), such as a log-structured merge tree (LSM-tree). Specifically, in the full index layer, the metadata is written into the tree in the memory first, when the metadata stored in the tree in the memory reaches a certain threshold, the tree in the memory is refreshed into the storage medium for persistent storage, the released tree in the memory is continuously written with new metadata, and meanwhile, the tree in the storage medium may also perform a merge (merge) operation periodically to generate a new tree, as shown in fig. 3, the metadata may be written into the C0 tree in the memory first, then refreshed into the C1 tree in the storage medium, the C1 tree is merged into the C2 tree, and so on. Therefore, when reading the metadata a of the full index layer data structure, the CPU of the server may search the trees in the memory first, and then sequentially search the trees in the storage medium, for example, sequentially search from the C0 tree to the C2 tree, until the metadata a is obtained, it should be understood that when querying the metadata of the full index layer data structure, the CPU of the server needs to participate in the search of multiple trees, so that the client 100 reads the metadata of the full index layer data structure through bilateral operation. In general, metadata with a higher access frequency (also referred to as hot data) will be stored using a cache layer data structure, and metadata with a lower access frequency (also referred to as cold data) will be stored using a full index layer data structure, and in particular, the server may distinguish metadata with a higher access frequency from metadata with a lower access frequency through some memory management algorithms, such as a Least Recently Used (LRU) algorithm, which is not limited in this application.

Therefore, when the client 100 sends an access request of the target metadata a to the server 200, because the metadata reading speed of the cache layer is fast, the client 100 usually defaults that the target metadata a is stored using the cache layer data structure, and sends a single-side reading request to the server 200, the server 200 responds to the single-side reading request, executes the metadata reading flow of the cache layer data structure through the network card of the server 200, and returns the read data to the client 100, as can be seen from the embodiment shown in fig. 3, the target metadata may be stored in the memory or the storage medium, if the target metadata is stored in the storage medium, the network card of the server 200 responds to the data read by the single-side reading request, and is not the target metadata, that is, the single-side reading will fail, therefore, the client 100 will determine whether the received data is the target metadata a, if not, the client 100 will send the bilateral reading request to the server 200, and the server 200 executes the bilateral reading process, that is, the CPU sequentially traverses and searches the trees in the memory and the storage medium until the metadata is obtained.

To sum up, in the current distributed storage system, when the client 100 queries the metadata from the server 200, it first initiates a unilateral operation to search the metadata in the metadata of the hot data cached in the memory of the server, and when there is no hit in the memory, the client needs to initiate a bilateral operation, and the CPU of the server searches the metadata from the global metadata, that is, when there is no hit metadata in the memory, the client needs to initiate a query request to the server again, thereby increasing the load of the client, increasing the network bandwidth, and increasing the query delay of the metadata.

In order to solve the above-mentioned problems of the current distributed storage system, the present application provides a metadata obtaining method, which is applied in a computer system as shown in fig. 2, in which a client first sends an access request of target metadata to a network device at a server, the network device at the server queries the target metadata in a memory at the server, when the target metadata is not hit in the memory, the network device can send the access request of the target metadata to a processor at the server, the processor at the server searches the target metadata in global metadata, so that, when the target metadata is not found in the memory, the network device can continue to request the processor at the server to query the target metadata without notifying the client, the client sends a query request to the processor at the server again, thereby, the load of the client is reduced, the network bandwidth is saved, and the query delay of the metadata is reduced.

As shown in fig. 4, the method may include the steps of:

step S210: and the network equipment of the client sends an access request of the target metadata to the network equipment of the server. Wherein, the target metadata access request carries the information of the target metadata.

In particular implementations, the information of the target metadata may include an identification of the target metadata and a location specified by the access request. The identifier of the target metadata may be a key of the target metadata, and the location specified by the access request is a location of a key value pair of the target metadata calculated and obtained by the client in the memory of the server.

In an embodiment, the location specified by the access request may be obtained after the client performs hash operation on the key of the target metadata according to the hash function f (x), and is a storage location of the key value pair of the target metadata at the server predicted by the client, as can be known from the embodiment of fig. 3, if the server has used the hash function f (x) to store the target metadata in the memory, the location specified by the access request is the storage location of the key value pair of the target metadata, and if the target metadata is not stored in the memory, the location specified by the access request is not the storage location of the key value pair of the target metadata. Wherein the hash function is sent by the server to the client before step S210.

Specifically, the location specified by the access request may be a hash value obtained by performing hash operation on a key of the target metadata according to a hash function by the client, or may be a memory address determined according to the hash value. For example, if the identification of the target metadata is: if the server sends the mapping relationship between the memory address and the hash value to the client in advance, the client may determine the memory address ADD7 corresponding to the hash value V ═ 7 according to the mapping relationship, and then the location specified by the access request may also be the memory address ADD7 where the key value pair of the target metadata is located.

In an embodiment, the hash function may be sent by the server to the client after the client and the server establish a connection, it should be understood that, when the server caches metadata information of part of the data of the application in a memory, if a key value in the metadata information changes a hash function used by the key value, the server may send the changed hash function to the client, so that the client calculates a location specified by the access request according to the changed hash function.

In an embodiment, the key may be determined according to target metadata accessed by the client, and in a specific implementation, the client may store a correspondence between the metadata and the key in a database, for example, a key corresponding to metadata a is 11100, a key corresponding to metadata b is 11101, and so on, so that when the client initiates an access request requesting to access the metadata a to the server, the identifier of the target metadata carried in the access request may be determined according to the correspondence: key 11100, the access request specifies the location: the hash value V corresponding to the key 11100 obtained according to the hash function f (k) described above. It is to be understood that the above description is illustrative, and that the present application is not limited to this description.

In an embodiment, the access request for the target metadata may further include other information, and optionally, the access request may further include a data length L, where the read length is used for the server to read the metadata information with the data length L from the location specified by the access request. Optionally, the access request may further include a write address, where the write address is an address of the client where the target metadata is written, so that after the network device at the server acquires the target metadata, the target metadata may be written into the write address through an RDMA unilateral operation, so that the target metadata is written without participation of a CPU at the server, and the efficiency of reading the target metadata is improved. Optionally, the access request may further include a descriptor of the access request, where the descriptor of the read request is used for the server to determine a meaning of each field in the read request, such as which field is a key, which field is a location specified by the access request, and which field is a data length, for example, the read request may be a segment of a message, the descriptor may be located in a header of the message, and after the server reads the descriptor from the header of the message, the server determines that the 1 st to 4KB fields are descriptors, the 5 th to 8KB fields are keys, the 9 th to 16 th fields are locations specified by the access request, and the 17 th to 24 th fields are data lengths. It should be understood that the above examples are merely illustrative, the access request may further include other information, such as a key or a check code for improving the security performance of data transmission, and the present application is not limited thereto, and the descriptor may also be in other expressions, such as using a line feed, a space, a colon, etc. as a separator to split a plurality of fields in the access request, and the present application is not limited thereto.

S220: the network device of the server obtains the metadata information at the location specified by the access request.

The metadata information includes an identifier of the metadata and the metadata, or the identifier of the metadata and metadata acquisition information, where the identifier of the metadata refers to a key, and the metadata acquisition information is used by the server to acquire the metadata, and may specifically be a value V in a key value pair of the metadata. It should be understood that, since the location specified by the access request is the storage location of the target metadata predicted by the client according to the hash function, if the target metadata is stored in the memory of the server, the metadata information read by the server from the location is information of the target metadata, and if the target metadata is not stored in the memory of the server, the metadata information read by the server from the location is information of non-target metadata.

As can be seen from the foregoing, when the server stores the metadata in the memory, the server performs hash operation on the key of the metadata by using a hash function to obtain a hash value V corresponding to the key, and then stores the metadata according to the hash value V, where for example, the hash value is a memory address where the metadata is written into the server, or a mapping relationship exists between the hash value and the memory address where the metadata is written into the server. Therefore, if the target metadata is already stored in the memory by the server, the client performs hash operation on the target metadata key K0 using the same hash function to obtain a hash value V0, and then encapsulates the hash value V0 into an access request of the target metadata and sends the access request to the server, and the server can obtain the target metadata according to the hash value V0, that is, the metadata information obtained in step S220 is information of the target metadata. However, as can be seen from the embodiment shown in fig. 3, no matter whether the target metadata is a cache layer data structure or a full index layer data structure, the target metadata may be stored in the memory of the server or in the storage medium of the server. Therefore, if the target metadata is not stored in the memory by the server but stored in the storage medium of the server, the metadata information acquired in step S220 is not the information of the target metadata, and at this time, the network device of the server cannot read the target metadata through the one-sided operation, and the network device of the server can notify the CPU of the server to read the target metadata through the two-sided operation.

S230: and the network equipment of the server compares the acquired metadata information with the target metadata information to determine whether the target metadata is stored in the memory.

As can be known from the embodiment of fig. 3, the metadata of the cache layer data structure is metadata of hot data, and when the server writes the metadata into the memory, if old data of the same key is stored at a memory address corresponding to a hash value obtained by computing the key of the metadata by using a hash function, the old data is written into the storage medium and stored by using a full index layer data structure, so that the client uses the hash function to hash the key K of the target metadata to obtain location information, and a situation that the server does not store the target metadata at the location information may occur. Therefore, after the network device at the server acquires the metadata information from the position indicated by the position information, the network device at the server may compare the information of the target metadata with the acquired metadata information, and if the metadata information is consistent, the metadata information read by the network device is the information of the target metadata requested to be acquired by the client, the metadata is stored in the memory and can be read through one-sided operation, and if the metadata information is inconsistent, the network device at the server indicates that the target metadata is stored in the storage medium and cannot be read through one-sided operation, and the network device at the server may notify the CPU at the server to perform two-sided operation to acquire the target metadata.

In a specific implementation, the network device at the server may compare the key K in the information of the metadata obtained from the location specified by the access request with the key K in the information of the target metadata, determine whether the metadata can be read through a single-sided operation by comparing whether the two are consistent, if the two are consistent, indicate that the target metadata is stored in the memory, may perform step S240 of reading the target metadata through the single-sided operation, if the two are not consistent, indicate that the target metadata is stored in the storage medium, and may perform step S250 of notifying the CPU of performing the double-sided operation to read the target metadata. Therefore, the client only needs to send an access request to the server once, the server automatically determines whether the metadata reading needs to be converted from the unilateral operation into the bilateral operation, and does not need to inform the client, so that the client does not need to send the access request to the server for multiple times, the communication efficiency is improved, the network bandwidth is saved, the processing pressure of the client is reduced, and the query delay of the metadata is reduced.

Still taking the example in step S210 as an example, assuming that the identifier of the target metadata carried in the access request is K equal to 3, the location specified by the access request obtained by the client is the memory address ADD7 corresponding to the hash value V equal to 7, the communication device at the service end can read and obtain one key value pair (K equal to 3 and V equal to f (3)) from the memory address ADD7, and by comparing the identifier of the target metadata with the key in the key value pair obtained according to the specified location, both of them are 3, so that the obtained key value pair is read from ADD7 as the key value pair of the target metadata, and the service end can obtain the target metadata according to the value V equal to f (3) in the key value pair. Similarly, if the key value pair read by ADD7 is (K is 5, V is f (5)), the identifier of the target metadata is 3, and the key of the metadata read at the specified position is K is 5, which are not consistent, the information indicating that the metadata read at the specified position is not the information of the target metadata, the single-side operation fails to be read, and the network device may notify the CPU at the service end to perform the double-side reading operation according to the identifier of the target metadata to obtain the target metadata. It should be understood that the above examples are for illustration purposes and the present application is not intended to be limited to the details shown.

S240: when the target metadata is stored in the memory, the network device of the server side acquires the target metadata and returns the target metadata to the client side.

As can be known from reference to step S220, the metadata information may specifically be a (K, V) key value pair of the metadata, where a value V corresponding to the key K may be the metadata or acquisition information of the metadata, and therefore, in the key value pair of the metadata stored in the server, if the value definition is the metadata itself, the network device of the server may directly return the value V in the key value pair to the client as the target metadata; in the key value pair of the metadata stored in the server, the network device may return the target metadata to the client after acquiring the target metadata according to the acquisition information if the value definition is the acquisition information of the metadata. For a specific process of obtaining metadata according to values in the key value pair, reference may be made to the embodiment in fig. 3, and details are not repeated here.

S250: and when the target metadata is not stored in the memory, the network equipment of the server side sends the information of the target metadata to the CPU of the server side.

In an embodiment, when the target metadata is not stored in the memory, the network device may notify the CPU of the server, and the CPU initiates a bilateral operation to obtain the metadata. In a specific implementation, the information of the target metadata may be an identifier of the target metadata carried in an access request sent by the client to the server, such as a key of the target metadata.

S260: and the CPU of the server side acquires the target metadata according to the information of the target metadata.

Referring to the embodiment of fig. 3, the CPU may first search whether the tree in the memory stores the target metadata according to the key of the target metadata, then search the C1 tree and the C2 tree in the storage medium, and so on until the target metadata is searched. Specifically, reference may be made to the description of how to read the metadata through the bilateral operation in the embodiment of fig. 3, which is not repeated herein.

S270: and the CPU of the server returns the target metadata to the network equipment, and the network equipment transmits the target metadata to the client.

In an embodiment, in step S240 and step S270, after the server obtains the target metadata, the server may return the target metadata to the client through the network device, and then the client determines whether to initiate an access request for the target data according to the target metadata; target data can be obtained according to the target metadata and then returned to the client; the target metadata can be acquired from the local computer and returned to the client under the condition that the target metadata is determined to be stored in the local computer after being acquired, so that the processing pressure of the client is further reduced, and the data reading efficiency of the distributed system is improved. In an embodiment, if the metadata version returned by the server is not the target metadata version requested to be accessed by the client due to the fact that the metadata version stored in the server is continuously updated, for example, the version of the metadata a at time T1 is V1, the client requests to access the latest version of the metadata a at time T1, the version of the metadata a at time T2 becomes V2, and the server returns the data a at the version V1 to the client at time T2, or the client requests to access the metadata a at the version V1, but the server returns the metadata a at the latest version V2, which results in the client obtaining an incorrect version of the target metadata. In order to solve the above problem, in step S210, the access request of the target metadata may further include version information of the target metadata, in step S220, the server may further include a version number of the metadata from the metadata information obtained from the location specified by the access request, in step S230, when comparing whether the information of the target metadata and the metadata information are consistent, the network device of the server may compare key in the information of the metadata obtained from the location specified by the access request with key in the information of the target metadata, in case that the key and the metadata information are consistent, the version information in the information of the metadata obtained from the location specified by the access request is compared with version information in the information of the target metadata, and in case that the key and the key are consistent, the network device further executes step S240, the network device reads the target metadata through one-sided operation, otherwise, step S250 to step S270 are executed, and the CPU is notified to perform the bilateral operation to read the target metadata.

For example, as shown in fig. 5, the server 200 receives an access request of the target metadata sent by the client 100, where the access request may include a key of the target metadata, version information of the target metadata, and a location specified by the access request, where the server 200 stores a key value pair of the metadata using a hash function, the location specified by the access request is obtained by the client 100 according to the same hash function and the key of the target metadata, and a flow of steps of the server 200 processing the read request may be as follows:

1. the network device 210 receives an access request for target metadata sent by the client 100.

2. The network device 210 obtains information of the metadata, which may include key-value pairs of the metadata and versions of the metadata, from a location specified by the access request. It should be understood that, as can be seen from the foregoing, if the target metadata is stored in the memory 230, the key value pair obtained by the network device from the location specified by the location request is the key value pair of the target metadata, then the key in the key value pair is the same as the key of the target metadata, and if the target metadata is not stored in the memory 230, then the key in the key value pair is not the same as the key of the target metadata.

3. The network device 210 determines whether the key in the key value pair of the metadata obtained from the position specified by the position request coincides with the key in the information of the target metadata.

4. The network device 210 determines whether the version information of the target metadata is consistent with the version information obtained from the location specified by the location request, in a case where the key in the key value pair is consistent with the key in the information of the target metadata. It should be noted that, the present application does not limit the sequence of the above step 3 and step 4.

5. The network device 210 may acquire the target metadata according to a value in a key value pair in metadata information obtained at a location specified by the location request when the version information of the target metadata is consistent with the version information of the metadata, where the value of the metadata in the server represents the metadata itself, the value may be directly returned to the client as the target metadata, and the value of the metadata represents the acquisition information of the metadata, the target metadata may be obtained according to the value, and the target metadata is returned to the client.

6. The network appliance 210 returns the target metadata to the client.

7. In the case where the key in the key value pair is inconsistent with the key in the information of the target metadata, or the version information of the target metadata is inconsistent with the version information of the metadata, the network device 210 transmits the information of the target metadata to the processor 220 to request query data. Specifically, the CPU may sequentially query the target metadata in the trees of the memory and the storage medium according to the key value of the target metadata, which may specifically refer to the foregoing embodiment of fig. 3, and details are not repeated here.

In an embodiment, if the target metadata does not carry version information and the server determines that the target metadata has multiple versions, the server may further feed back the condition that the target metadata has multiple versions to the client, the client re-initiates an access request of the target metadata carrying the version information, and the server may perform the steps 1 to 7 or the target metadata, so as to improve the user experience of the user.

In an embodiment, as can be seen from the foregoing, the network device 130 at the client and the network device 210 at the server may be network cards with RDMA functions, or network cards or switches supporting IB, so that the network device 130 and the network device 210 may implement the above-mentioned scheme provided in this application through programmable chips such as a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), and the like.

In summary, the present application provides a data reading method, in which a client first sends an access request of target metadata to a network device of a server, the network device of the server queries the target metadata in a memory of the server, and when the target metadata is not hit in the memory, the network device may send the access request of the target metadata to a processor of the server, and the processor of the server searches the target metadata in global metadata.

The method of the embodiments of the present application is described above in detail, and in order to better implement the above-mentioned aspects of the embodiments of the present application, the following also provides related apparatuses for implementing the above-mentioned aspects.

Fig. 6 is a schematic structural diagram of a network device 210 according to the present application, where the network device 210 is applied to the distributed storage system shown in fig. 2, where the network device 210 is connected to a server 200 and a client 100, the client runs an application, and a memory of the server caches metadata information of partial data of the application, and as shown in fig. 6, the network device 210 may include a receiving unit 610, an information obtaining unit 620, a determining unit 630, a data obtaining unit 640, and a sending unit 650.

A receiving unit 610, configured to receive an access request of target metadata sent by a client, where the access request of the target metadata carries information of the target metadata and a location specified by the access request, and please refer to detailed description of step S210 in the embodiment shown in fig. 4 and step 1 in the embodiment shown in fig. 5, which is not repeated here.

The information obtaining unit 620 is configured to obtain metadata information at a location specified by the access request in the memory, and for a specific implementation, please refer to detailed description of step S220 in the embodiment shown in fig. 4 and step 2 in the embodiment shown in fig. 5, which is not repeated herein.

A determining unit 630, configured to compare the obtained metadata information with information of the target metadata to determine whether the target metadata is stored in the memory, please refer to step S230 in the embodiment shown in fig. 4 and detailed descriptions of step 3 and step 4 in the embodiment shown in fig. 5, which is not repeated herein.

For a specific implementation, please refer to step S240 in the embodiment shown in fig. 4 and detailed descriptions of step 5 and step 6 in the embodiment shown in fig. 5, which are not repeated herein.

In an embodiment, the network device 210 further includes a sending unit 650, where the sending unit 650 is configured to send information of the target metadata to a processor of the server when the target metadata is not stored in the memory, so that the processor obtains the target metadata according to the information of the target metadata and transmits the target metadata to the client. For a specific implementation manner, please refer to step S250 in the embodiment shown in fig. 4 and detailed description of step 7 in the embodiment shown in fig. 5, which is not repeated herein.

In an embodiment, the metadata information cached in the memory includes an identifier of the metadata and the metadata or an identifier of the metadata and metadata obtaining information, and the metadata obtaining information is used for obtaining the metadata.

In an embodiment, the metadata information is stored in the memory in the form of a Key-Value pair, where the identifier of the metadata is a Key in the Key-Value pair, and the metadata or the metadata acquisition information is a Value in the Key-Value pair.

In an embodiment, metadata information of part of the applied data is cached in a memory of the server by using a hash table, and a hash value obtained by performing hash operation on a data identifier of the metadata by the server is a storage location of the metadata information in the hash table.

In an embodiment, the location specified by the access request is a hash value obtained by performing a hash operation on the target metadata by the client, where a hash function used by the client for the hash operation is the same as a hash function used by the server.

In an embodiment, the information of the target metadata includes an identifier of the target metadata, and the determining unit 630 is configured to compare the identifier of the target metadata with a key in a key value pair, and determine whether the target metadata is stored in the memory. The metadata obtaining unit 620 is configured to, when the identifier of the target metadata is consistent with the key, obtain the target metadata according to the obtaining information of the target metadata if the metadata corresponding to the key is the target metadata or the key-corresponding metadata obtaining information is the obtaining information of the target metadata, and transmit the target metadata to the client. And a sending unit 650, configured to send the information of the target metadata to the processor of the server when the identifier of the target metadata is inconsistent with the key, so that the processor obtains the target metadata according to the information of the target metadata and transmits the target metadata to the client.

In summary, the present application provides a network device, the network device is connected to a client and a server, the network device can receive an access request of target metadata sent by the client, and query the target metadata in a memory of the server, when the target metadata is not hit in the memory, the network device can send the access request of the target metadata to a processor of the server, and the processor of the server searches the target metadata in global metadata, so that when the target metadata is not found in the memory, a network card can continue to request the processor of the server to query the target metadata without notifying the client, and the client sends a query request to the processor of the server again, thereby reducing the load of the client, saving network bandwidth, and reducing query delay of the metadata.

Fig. 7 is a schematic hardware structure diagram of a network device 210 provided in the present application, and as shown in fig. 7, the network device 210 includes: a processor 710, a communication interface 720, and a memory 730. The processor 710, the communication interface 720 and the memory 730 may be connected to each other via an internal bus 740, or may communicate via other means such as wireless transmission. In the embodiment of the present application, the bus 740 is exemplified by being connected by a bus 740, and the bus 740 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 740 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 7, but that does not indicate only one bus or one type of bus.

Processor 710 may be comprised of at least one general purpose processor, such as a Central Processing Unit (CPU), or a combination of a CPU and hardware chips. The hardware chip may be an Application-Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a Field-Programmable Gate Array (FPGA), General Array Logic (GAL), or any combination thereof. Processor 710 executes various types of digitally stored instructions, such as software or firmware programs stored in memory 730, which enable network device 210 to provide a wide variety of services.

Memory 730 is used to store program code and is controlled for execution by processor 710 to perform the processing steps of network device 210 in any of the embodiments of fig. 1-7 described above. One or more software modules may be included in the program code. The one or more software modules may be software modules provided in the embodiment shown in fig. 6, such as a receiving unit, an information obtaining unit, a determining unit, a metadata obtaining unit, and a sending unit, where the receiving unit is configured to receive an access request for sending target metadata by a client; the information acquisition unit is used for acquiring metadata information at a position specified by the access request in the memory; the determining unit is configured to compare the acquired metadata information with information of the target metadata to determine whether the target metadata is stored in the memory, the metadata acquiring unit is configured to, when the target metadata is stored in the memory, the sending unit is used for sending the information of the target metadata to a processor of the server side when the target metadata is not stored in the memory, the processor may specifically be configured to execute steps S210 to S270 in the embodiment of fig. 4, steps 1 to step 7 in the embodiment of fig. 5, and optional steps of the steps, and may also be configured to execute other steps executed by the network device 210 in the embodiments of fig. 1 to fig. 7, which are not described herein again.

Memory 730 may include Volatile Memory (Volatile Memory), such as Random Access Memory (RAM); the Memory 730 may also include a Non-Volatile Memory (Non-Volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, HDD), or a Solid-State Drive (SSD); memory 730 may also include combinations of the above. The memory 730 may store program codes, and particularly, may include program codes for the processor 710 to execute other steps described in the embodiments of fig. 1 to fig. 6, which are not described herein again.

In one implementation, communication interface 720 may be a wired interface (e.g., an ethernet interface), an internal interface (e.g., a Peripheral Component Interconnect express (PCIe) bus interface), a wired interface (e.g., an ethernet interface), or a wireless interface (e.g., a cellular network interface or a wireless lan interface), and is configured to communicate with other servers or modules, and in another implementation, communication interface 720 may be configured to receive a message for processing by processor 710.

It should be noted that fig. 7 is only one possible implementation manner of the embodiment of the present application, and in practical applications, the network device may further include more or less components, which is not limited herein. For contents that are not shown or described in the embodiments of the present application, reference may be made to the related explanations in the embodiments of fig. 1 to fig. 6, and details are not repeated here.

Embodiments of the present application further provide a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is executed on a processor, the method flows shown in fig. 1 to fig. 6 are implemented.

Embodiments of the present application also provide a computer program product, and when the computer program product is run on a processor, the method flows shown in fig. 1-6 are implemented.

The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes at least one computer instruction. The procedures or functions according to the embodiments of the invention are wholly or partly generated when the computer program instructions are loaded or executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that contains at least one collection of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., Digital Video Disc (DVD), or a semiconductor medium.

While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A computer system comprises a client and a server, wherein the server comprises a network device, the network device is used for connecting the client and the server, and the computer system is characterized in that,

the client is used for running an application;

the server is used for caching metadata information of partial data of the application in a memory;

the network device is used for acquiring metadata information from a position specified by an access request of the client to access target metadata from the memory according to the access request, determining whether the target metadata is stored in the memory by comparing the metadata information with the information of the target metadata, and acquiring the target metadata and transmitting the target metadata to the client when the target metadata is stored in the memory.

2. The system according to claim 1, wherein the network device is further configured to send information of the target metadata to the processor of the server when the target metadata is not stored in the memory;

the processor of the server is used for acquiring the target metadata according to the information of the target metadata;

the network device is further configured to transmit the target metadata acquired by the processor to the client.

3. The system according to claim 1 or 2, wherein the metadata information cached in the memory comprises an identifier of metadata and metadata or an identifier of metadata and metadata acquisition information, and the metadata acquisition information is used for acquiring the metadata.

4. The system of claim 3, wherein the metadata information is stored in the memory in the form of Key-Value pairs, wherein the identifier of the metadata is a Key in the Key-Value pair, and the metadata or the metadata acquisition information is a Value in the Key-Value pair.

5. The system according to claim 3, wherein metadata information of the applied partial data is cached in a memory of the server by using a hash table, and a hash value obtained by performing a hash operation on the data identifier of the metadata by the server is a storage location of the metadata information in the hash table.

6. The system according to claim 5, wherein the location specified by the access request is a hash value obtained by the client performing a hash operation on the target metadata information, and wherein a hash function used by the client performing the hash operation is the same as a hash function used by the server when determining the storage location of the metadata information.

7. The system according to claim 4, wherein the information about the target metadata includes an identifier of the target metadata, the network device is configured to compare the identifier of the target metadata with a key in the key value pair, and when the identifier of the target metadata is consistent with the key, the metadata corresponding to the key is the target metadata or the key-corresponding metadata acquisition information is the acquisition information about the target metadata, and the network device acquires the target metadata according to the acquisition information about the target metadata.

8. A metadata acquisition method is applied to network equipment, the network equipment is used for connecting a client and a server, the client runs an application, and metadata information of partial data of the application is cached in a memory of the server; characterized in that the method comprises:

receiving an access request of target metadata sent by a client, wherein the access request of the target metadata carries information of the target metadata and a position specified by the access request;

acquiring metadata information at a position specified by the access request in the memory;

comparing the obtained metadata information with the information of the target metadata to determine whether the target metadata is stored in the memory;

and when the target metadata is stored in the memory, acquiring the target metadata and transmitting the target metadata to the client.

9. The method of claim 8, further comprising:

when the target metadata is not stored in the memory, the network equipment sends the information of the target metadata to a processor of the server, so that the processor of the server acquires the target metadata according to the information of the target metadata;

and the network equipment receives the target metadata sent by the processor of the server side and transmits the target metadata to the client side.

10. The method according to claim 8 or 9, wherein the metadata information cached in the memory includes an identifier of metadata and metadata or an identifier of metadata and metadata acquisition information, and the metadata acquisition information is used for acquiring the metadata.

11. The method of claim 10, wherein the metadata information is stored in the memory in the form of a Key-Value pair, wherein the identifier of the metadata is a Key in the Key-Value pair, and the metadata or the metadata acquisition information is a Value in the Key-Value pair.

12. The method according to claim 10, wherein metadata information of the applied partial data is cached in a memory of the server by using a hash table, and a hash value obtained by performing a hash operation on a data identifier of the metadata by the server is a storage location of the metadata information in the hash table.

13. The method according to claim 12, wherein the location specified by the access request is a hash value obtained by hashing the target metadata by the client, and wherein a hash function used by the client for hashing is the same as a hash function used by the server for determining the storage location of the metadata information when the server stores the metadata information.

14. The method of claim 11, wherein the information of the target metadata is an identification of the target metadata, and wherein comparing the obtained metadata information with the information of the target metadata to determine whether the target metadata is stored in the memory comprises:

the network equipment compares the identification of the target metadata with the keys in the key value pair to determine whether the target metadata is stored in the memory;

when the target metadata is stored in the memory, acquiring the target metadata and transmitting the target metadata to the client includes:

and when the identifier of the target metadata is consistent with the key, the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata, and the network equipment acquires the target metadata according to the acquisition information of the target metadata and transmits the target metadata to the client.

15. A network device, configured to connect a client and a server, where the client runs an application, and a memory of the server caches metadata information of partial data of the application, and the network device includes:

the system comprises a receiving unit, a sending unit and a receiving unit, wherein the receiving unit is used for receiving an access request of target metadata sent by a client, and the access request of the target metadata carries information of the target metadata and a position specified by the access request;

an information obtaining unit, configured to obtain metadata information at a location in the memory specified by the access request;

a determining unit configured to compare the acquired metadata information with information of the target metadata to determine whether the target metadata is stored in the memory;

and the metadata acquisition unit is used for acquiring the target metadata and transmitting the target metadata to the client when the target metadata is stored in the memory.

16. The network device of claim 15, wherein the network device further comprises a sending unit,

the sending unit is configured to send information of the target metadata to a processor of the server when the target metadata is not stored in the memory, so that the processor obtains the target metadata according to the information of the target metadata and transmits the target metadata to the client.

17. The network device according to claim 15 or 16, wherein the metadata information cached in the memory includes an identifier of metadata and metadata or an identifier of metadata and metadata acquisition information, and the metadata acquisition information is used for acquiring the metadata.

18. The network device of claim 17, wherein the metadata information is stored in the memory in the form of a Key-Value pair, wherein the identifier of the metadata is a Key in the Key-Value pair, and the metadata or the metadata acquisition information is a Value in the Key-Value pair.

19. The network device according to claim 17, wherein metadata information of the applied partial data is cached in a memory of the server by using a hash table, and a hash value obtained by performing a hash operation on a data identifier of the metadata by the server is a storage location of the metadata information in the hash table.

20. The network device according to claim 19, wherein the location specified by the access request is a hash value obtained by hashing the target metadata by the client, and wherein a hash function used by the client for hashing is the same as a hash function used by the server when determining the location where the metadata information is stored.

21. The network device of claim 18, wherein the information of the target metadata includes an identification of the target metadata,

the determining unit is configured to compare the identifier of the target metadata with a key in the key value pair, and determine whether the target metadata is stored in the memory;

the metadata acquisition unit is configured to, when the identifier of the target metadata is consistent with the key, determine that the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata, acquire the target metadata according to the acquisition information of the target metadata, and transmit the target metadata to the client;

and the sending unit is used for sending the information of the target metadata to a processor of the server when the identifier of the target metadata is inconsistent with the key, so that the processor can obtain the target metadata according to the information of the target metadata and transmit the target metadata to the client.

22. A network device comprising a processor and a communication interface, the communication interface being configured to receive an access request from a client to access target metadata, the processor being configured to perform the method of any of claims 8 to 14 to process the access request.