WO2022267909A1

WO2022267909A1 - Method for reading and writing data and related apparatus

Info

Publication number: WO2022267909A1
Application number: PCT/CN2022/098309
Authority: WO
Inventors: 晏思宇; 刘世兴; 曲迪; 冀智刚
Original assignee: 华为技术有限公司
Priority date: 2021-06-22
Filing date: 2022-06-13
Publication date: 2022-12-29
Also published as: CN115509433A

Abstract

A method for reading and writing data, comprising: a switch receiving address information of a memory controller (1100) from the memory controller (1100), and assigning corresponding switching address information to the memory controller (1100). According to the address information of the memory controller (1100) and the corresponding switching address information of the memory controller (1100), the switch establishes a direct path connection to the memory controller (1100), wherein the direct path connection is used for transmitting an input/output (I/O) message, and the I/O message is used for writing data into a memory array by means of the memory controller (1100) or reading data from the memory array by means of the memory controller (1100). Because the direct path connection established between the switch and the memory controller (1100) allows for transmission of the I/O message, the multiple address searches of I/O processing is avoided, the forwarding path of the I/O message is shortened, the time for completing data reading and writing operations is reduced, and the data reading and writing efficiency is increased. Further disclosed is a related apparatus for reading and writing data.

Description

A data reading and writing method and related device

This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on June 22, 2021, with the application number 202110694337.7, and the title of the invention is "A Data Reading and Writing Method and Related Devices", the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the technical field of storage, and in particular to a method for reading and writing data and related devices.

Background technique

Enterprise storage or distributed storage systems typically include clusters of client servers, networks, and storage arrays. A client-server cluster (referred to as a client for short) may carry various applications such as a structured query language (Structured Query Language, SQL) service or a database. The data read and written by the client is actually stored in the back-end storage array cluster. The data read and write access of the client is also called input/output (input/output, I/O) request, and each data read and write access requires the client to perform actual data read and write operations in the storage array cluster through the network.

A storage array cluster consists of many storage controllers and storage arrays. Considering factors such as high availability and data access consistency, each storage controller is often responsible for data read and write operations on some data disks. For each I/O request for data, a corresponding storage controller in the storage array cluster is required to be responsible for reading and writing the I/O request to the data disk. That is, each I/O request has a corresponding attributable storage controller, and only the attributable storage controller has the right to access the data in the storage array. Currently, the client and the storage array are usually two independent systems, and the client cannot obtain the storage controller corresponding to each I/O request in advance. The corresponding relationship between I/O requests and storage controllers is stored in each storage controller of the storage array. The client sends the I/O request to a storage controller in the storage array according to its own algorithm (such as load balancing, round robin, etc.). The storage controller determines the target storage controller by searching the corresponding relationship between the I/O request and the storage controller, and reroutes the I/O request to the target storage controller for data read and write operations.

Since the above method requires rerouting, the forwarding path of the I/O request is longer, resulting in a longer time for data read and write operations to complete, which affects the efficiency of data read and write.

Contents of the invention

The present application proposes a method for reading and writing data. A switch receives address information of the storage controller from a storage controller, and assigns corresponding switching address information to the storage controller. According to the address information of the storage controller and the exchange address information corresponding to the storage controller, the switch establishes a direct path connection with the storage controller, the direct path connection is used to transmit I/O packets, and the I The /O message is used to write data to the storage array through the storage controller or read data from the storage array through the storage control. Since the direct path connection established between the switch and the storage controller can transmit I/O packets, multiple addressing for I/O processing is avoided, the forwarding path of I/O packets is shortened, and the completion of data read and write operations is reduced. time, improve I/O access times (input/output per second, IOPS) performance, and improve data read and write efficiency.

The first aspect of the present application provides a method for reading and writing data. The switch receives the address information of the storage controller, and the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the The Internet protocol address of the storage controller, the transmission control protocol TCP port number, or the protocol number of the storage controller; the switch allocates corresponding exchange address information for the storage controller, and the exchange address information includes the following one One or more items: the queue pair port number of the switch, the Internet protocol IP address of the switch or the port number of the switch; the switch establishes a direct path connection with the storage controller, and the direct path connection uses For transmitting input and output I/O messages, the I/O messages are used to write data to the storage array through the storage controller or read data from the storage array through the storage control.

In a possible scenario, the switch establishes a communication connection with the storage controller. The address information of the storage controller is transmitted to the switch via the communication link. This communication connection may also be referred to as a control plane connection. The specific process of establishing the communication connection is as follows: (a), the switch initiates a monitoring task related to the direct path connection. (b) After the storage controller goes online, the storage controller locally allocates the queue pair port number of the storage controller (or the IP address and port number of the storage controller). (c) After the switch detects that the storage controller is online, the switch and the storage controller establish a control plane connection through a remote direct data access RDMA link establishment process (or a transmission control protocol TCP link establishment process).

When a TCP connection is established between the switch and the storage controller, the TCP connection is used to transmit address information of the storage controller. The address information of the storage controller includes: the IP address of the storage controller, the TCP port number and the protocol number of the storage controller.

When an RDMA connection is established between the switch and the storage controller, the RDMA connection is used to transmit address information of the storage controller. The address information of the storage controller includes: the queue pair port number QP of the storage controller and the IP address of the storage controller.

In this application, when the client goes online, the client establishes a network connection with the storage controller, and the switch establishes a network connection with the storage controller. After the network connection is established, the switch receives address information of the storage controller from the storage controller, and assigns corresponding exchange address information to the storage controller. According to the address information of the storage controller and the exchange address information corresponding to the storage controller, the switch establishes a direct path connection with the storage controller, the direct path connection is used to transmit I/O packets, and the I The /O message is used to write data to the storage array through the storage controller or read data from the storage array through the storage control. Since the direct path connection established between the switch and the storage controller can transmit I/O packets, multiple addressing for I/O processing is avoided, the forwarding path of I/O packets is shortened, and the completion of data read and write operations is reduced. Time, improve IOPS performance, and improve data read and write efficiency.

Optionally, the switch receives routing information from the storage controller, where the routing information includes one or more of the following information: identification information of the storage controller, input and output of the storage controller The I/O address, whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller, the I/O address includes a logical unit number, a name space identifier and/or or logical block address.

In this application, the load information of the storage array managed by the storage controller will be referred to as the load information of the storage controller for short. The load information of the storage controller includes, but is not limited to: the total storage space of the storage controller, the remaining available storage space of the storage controller, the used storage space of the storage controller, the IOPS of the storage controller, the storage Whether the controller supports active-active mode, or the temperature of the storage controller.

The switch receives the routing information of the storage controller, so that the switch can grasp the status of the storage controller. After receiving the I/O message, the switch can quickly determine the corresponding storage controller for the I/O message. Improve data read and write efficiency.

Optionally, the switch generates a first mapping relationship, where the first mapping relationship includes a mapping relationship between the identification information of the direct path connection and the I/O address. The first mapping relationship may be a hash (hash) table of key-value pairs (key-value). The key (key) of the table is the I/O address, and the value (value) of the table is the identification information of the direct path connection. After the switch receives the I/O message, it can quickly determine the corresponding storage controller for the I/O message and the first mapping relationship. Improve data read and write efficiency.

In a possible implementation manner, when only one direct path connection is established between any switch and any storage controller, the identification information of the direct path connection may be the identification information of the storage controller.

Optionally, the first mapping relationship may include: identification information of the direct path connection, identification information of the storage controller, and an I/O address.

Optionally, the first mapping relationship may include: identification information of the direct path, an I/O address, identification information of the storage controller, and identification information of the client.

Optionally, the switch receives the first I/O message from the client, and the first I/O message is used to write data to the storage array through the storage controller or read data from the storage array through the storage controller ; The switch determines the direct path connection according to the first I/O message and the first mapping relationship; the switch sends the first path connection to the storage controller through the direct path connection An I/O message.

Specifically, the switch parses the address information in the first I/O packet. The switch determines the destination of the first I/O message according to the address information in the first I/O message, that is, the storage controller corresponding to the first I/O message. The switch determines the corresponding identification information of the direct path connection according to the identification information of the storage controller. The switch sends the first I/O message to the storage controller by using the direct path connection.

Optionally, when the first mapping relationship includes: identification information of the direct path connection, identification information of the storage controller, and an I/O address. After receiving the first I/O message, the switch first determines the address information of the corresponding storage controller according to the I/O address in the first I/O message. Secondly, the switch determines the identification information of the direct path connection according to the address information of the storage controller. The switch sends the first I/O packet by using the direct path connection.

Optionally, when one or more direct path connections are established between any switch and any storage controller, different direct path connections are used to bear I/O packets of different clients. The first mapping relationship includes: identification information of the direct path, an I/O address, identification information of the storage controller, and identification information of the client.

After receiving the first I/O message, the switch first determines the address information of the corresponding storage controller according to the I/O address in the first I/O message. Secondly, the switch determines the source address information of the I/O message according to the first I/O message, that is, the address information of the client corresponding to the first I/O message. Thirdly, the switch determines the identification information of the client according to the address information of the client. In a possible implementation manner, the address information of the client is consistent with the identification information of the client. Thirdly, the switch determines the corresponding identification information of the direct path connection according to the identification information of the client. The switch sends the first I/O packet by using the direct path connection.

Optionally, the first mapping relationship includes: identification information of the direct path, an I/O address, identification information of the storage controller, and identification information of the client.

In a possible implementation, after receiving the first I/O message, the switch first determines the address information of the corresponding storage controller according to the I/O address in the first I/O message. Second, the switch detects the number of direct path connections corresponding to the storage controller. When the storage controller corresponds to only one direct path connection, the switch directly uses the direct path connection to send the first I/O packet.

When the storage controller corresponds to multiple direct path connections, the switch determines the first I/O according to the source address information in the first I/O message, that is, the address information of the client corresponding to the first I/O message /O which client the packet comes from. Thirdly, the switch determines the identification information of the client according to the address information of the client. In a possible implementation manner, the address information of the client is consistent with the identification information of the client. Thirdly, the switch determines the corresponding identification information of the direct path connection according to the identification information of the client. The switch sends the first I/O packet by using the direct path connection.

Optionally, the switch may dynamically select an appropriate storage controller to receive the first I/O packet according to status information reported by each storage controller. The suitable storage controller here can be a storage controller with larger available space, or a storage controller with stronger IOPS performance, or a storage controller with lower processing load of the central processing unit (CPU) of the storage controller. Controller, not limited here. In order to balance the workload of each storage controller.

Optionally, the switch detects the first I/O message, when the destination (storage controller) of the first I/O message is a storage controller implementing the active-active mode. Then the switch copies the first I/O packet. The switch sends the first I/O packet to a backup storage controller of the storage controller, and the storage controller and the backup storage controller work in a dual-active mode. The switch adds an identifier indicating that the message is a copy message to the first I/O message sent by the switch to the backup storage controller. Improve data security.

Optionally, the switch detects the type of the first I/O message; when the first I/O message is a data message, the switch connects to the storage controller through the direct path The switch sends the first I/O packet; when the first I/O packet is a non-data packet, the switch transparently transmits the first I/O packet. By identifying the type of packets, different processing is performed on different packets to reduce the load on the network.

Optionally, the switch receives a first reply message from the storage controller through the direct path connection, and the first reply message is a response to the first I/O message; the The switch sends the first reply packet to the client. The switch receives the reply message from the storage controller through the direct path connection, and then the switch sends the reply message to the client, so that the client does not perceive the routing path of the reply message.

Optionally, the switch receives a second reply message from the storage controller, the second reply message is a response to the first I/O message, and the purpose of the second reply message is The ground is the second storage controller, and the second storage controller is the storage controller initially allocated by the client for the first I/O message; the switch forwards the The second reply message; the switch receives the third reply message from the second storage controller, the third reply message is generated according to the second reply message; the switch forwards the message from The third reply message of the second storage controller.

The storage controller may forward the I/O reply message to the second storage controller, where the second storage controller is the storage controller initially allocated by the client to the first I/O message. The second storage controller retains the network connection with the client, so the second storage controller carries the I/O reply message on the network connection and directly sends it to the client. Perceive the routing path of the reply message.

Optionally, the second reply packet includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply packet to the client on behalf of the storage controller, The third reply packet does not include the proxy indication information. The storage controller instructs the second storage controller to send the second reply message to the client through explicit proxy indication information, so that the client does not perceive the routing path of the reply message.

Optionally, when the first I/O message needs to be copied, the switch copies the first I/O message; and the switch sends the copied copy to the backup storage controller of the storage controller. For the first I/O message, the backup storage controller and the storage controller work in the active-active mode.

The storage controller is a part of the active-active cluster, and the active-active cluster works in the active-active mode (also referred to as the active-active cluster performing the active-active task). The feature of an active-active cluster is that both clusters are running online and can support the same application load. When the client writes data to the active-active cluster, for example, the client sends an I/O message to the active-active cluster, one of the clusters in the active-active cluster performs read and write operations based on the I/O message, and the cluster Copy the I/O message and send it to another cluster, so that the other cluster stores the data in the I/O message. When the client reads data from the active-active cluster, if one cluster in the active-active cluster fails and the other cluster is still working normally, the client can directly read data through the working cluster. It can be seen that the active-active mode can effectively improve the security of stored data. Similarly, when the storage controller and the backup storage controller work in the active-active mode, after receiving the I/O message, the storage controller copies the I/O message and sends the I/O message to the backup storage controller. message. The data in the I/O message is stored by the backup storage controller.

Optionally, when the switch receives the first I/O message, the switch sends the first I/O message to the storage controller through a direct path connection with the storage controller. The switch sends first indication information to the second storage controller, where the first indication information instructs the second storage controller to update the local serial number of the received message, and the second storage controller is the client for the first I /O The memory controller to which the message is initially assigned. For example, after receiving the I/O message, the storage controller assigns a serial number to the I/O message as the serial number of the received message. The first indication information instructs the second storage controller to update the local serial number of the received message, so as to avoid the out-of-sequence problem caused by the second storage controller not receiving the first I/O message.

Optionally, the switch sending the first indication information to the second storage controller includes: the switch sending a signal message to the second storage controller, and the message header of the signal message includes the message of the first I/O message Header information, where the header of the signal message further includes first indication information.

Optionally, the switch notifies each storage controller to disable the out-of-sequence packet detection function. To avoid an out-of-sequence problem caused by the second storage controller not receiving the first I/O message, the second storage controller is the storage controller initially allocated by the client to the first I/O message.

Optionally, the direct path connection is a transmission control protocol TCP connection, or a remote direct data storage RDMA connection.

The second aspect of the present application provides a method for reading and writing data. The storage controller sends the address information of the storage controller to the switch. The address information of the storage controller includes one or more of the following: the queue pair of the storage controller Port number, the Internet protocol address of the storage controller, the transmission control protocol TCP port number of the storage controller, or the protocol number; the storage controller establishes a direct path connection with the switch, and the direct path connection uses For transmitting input and output I/O messages, the I/O messages are used to write data to the storage array through the storage controller or read data from the storage array through the storage control.

Optionally, the storage controller sends routing information of the storage controller to the switch, where the routing information includes one or more of the following information: identification information of the storage controller, the storage The input and output I/O address of the controller, whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller, the I/O address includes the logical unit number, naming ID and/or logical block address of the space. The switch receives the routing information of the storage controller, so that the switch can grasp the status of the storage controller. After receiving the I/O message, the switch can quickly determine the corresponding storage controller for the I/O message. Improve data read and write efficiency.

Optionally, the storage controller receives the first I/O packet from the switch through the direct path connection.

Optionally, the storage controller generates a first reply message according to the first I/O message, and the first reply message is a response to the first I/O message; the storage The controller sends the first reply packet to the switch. The switch receives the reply message from the storage controller through the direct path connection, and then the switch sends the reply message to the client, so that the client does not perceive the routing path of the reply message.

Optionally, the storage controller generates a second reply message according to the first I/O message, and the second reply message is a response to the first I/O message; the storage The controller sends the second reply message to the switch, the destination of the second reply message is a second storage controller, and the second storage controller is the client and the first I /O The memory controller to which the message is initially assigned. The storage controller may forward the I/O reply message to the second storage controller, where the second storage controller is the storage controller initially allocated by the client to the first I/O message. The second storage controller retains the network connection with the client, so the second storage controller carries the I/O reply message on the network connection and directly sends it to the client. Perceive the routing path of the reply message.

Optionally, the second reply packet includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply packet to the client on behalf of the storage controller. The storage controller instructs the second storage controller to send the second reply message to the client through explicit proxy instruction information, so that the client does not perceive the routing path of the reply message.

The third aspect of the present application provides a network device, including: a transceiver module and a processing module;

The transceiver module is used to receive the address information of the storage controller, the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the Internet protocol of the storage controller address, transmission control protocol TCP port number, or protocol number of the storage controller;

The processing module is configured to assign corresponding switching address information to the storage controller, and the switching address information includes one or more of the following: the queue pair port number of the switch, the Internet Protocol address of the switch, or the port number of the switch;

The processing module is further configured to establish a direct path connection with the storage controller, the direct path connection is used to transmit input and output I/O messages, and the I/O messages are used to pass through the storage controller Data is written to or read from the storage array through the storage control.

Optionally, the transceiver module is further configured to receive routing information from the storage controller, where the routing information includes one or more of the following information: identification information of the storage controller, the storage The input and output I/O address of the controller, whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller, the I/O address includes the logical unit number, naming ID and/or logical block address of the space.

Optionally, the processing module is further configured to generate a first mapping relationship, where the first mapping relationship includes a mapping relationship between the identification information of the direct path connection and the I/O address.

Optionally, the transceiver module is also configured to receive the first I/O message from the client;

The processing module is further configured to determine the direct path connection according to the first I/O message and the first mapping relationship;

The transceiver module is further configured to send the first I/O message to the storage controller through the direct path connection.

Optionally, the processing module is further configured to detect the type of the first I/O message;

The transceiver module is further configured to send the first I/O message to the storage controller through the direct path connection when the first I/O message is a data message;

The transceiver module is further configured to transparently transmit the first I/O message when the first I/O message is a non-data message.

Optionally, the transceiver module is further configured to receive a first reply message from the storage controller through the direct path connection, where the first reply message is the first I/O message the response to;

The transceiver module is further configured to send the first reply message to the client.

Optionally, the transceiver module is further configured to receive a second reply message from the storage controller, the second reply message is a response to the first I/O message, and the second reply message is The destination of the reply message is the second storage controller, and the second storage controller is the storage controller initially assigned by the client to the first I/O message;

The transceiver module is further configured to forward the second reply message to the second storage controller;

The transceiver module is further configured to receive a third reply message from the second storage controller, the third reply message is generated according to the second reply message;

The transceiver module is further configured to forward the third reply message from the second storage controller.

Optionally, the second reply packet includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply packet to the client on behalf of the storage controller, The third reply packet does not include the proxy indication information.

Optionally, the processing module is further configured to copy the first I/O message when the first I/O message needs to be copied;

The transceiver module is further configured to send the replicated first I/O message to a backup storage controller of the storage controller, and the backup storage controller and the storage controller work in a dual-active mode.

The fourth aspect of the present application provides a storage device, including: a transceiver module and a processing module;

The transceiver module is configured to send the address information of the storage controller to the switch, and the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the Internet protocol address, transmission control protocol TCP port number, or protocol number of the storage controller;

The processing module is configured to establish a direct path connection with the switch, the direct path connection is used to transmit input and output I/O messages, and the I/O messages are used to send the storage array to the storage controller through the storage controller Write data to or read data from the storage array through the storage control.

Optionally, the transceiver module is further configured to send routing information of the storage controller to the switch, where the routing information includes one or more of the following information: identification information of the storage controller, The input/output I/O address of the storage controller, whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller, the I/O address includes a logic unit number, namespace identifier and/or logical block address.

Optionally, the transceiver module is further configured to receive the first I/O message from the switch through the direct path connection.

Optionally, the processing module is further configured to generate a first reply message according to the first I/O message, and the first reply message is a response to the first I/O message;

The transceiver module is further configured to send the first reply message to the switch through the direct path connection.

Optionally, the processing module is further configured to generate a second reply message according to the first I/O message, and the second reply message is a response to the first I/O message;

The transceiver module is further configured to send the second reply message to the switch, the destination of the second reply message is a second storage controller, and the second storage controller is the client A storage controller initially allocated for the first I/O packet.

Optionally, the second reply packet includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply packet to the client on behalf of the storage controller.

A fifth aspect of the present application provides a network device, where the network device includes: a processor, configured to enable the network device to implement the method described in the foregoing first aspect or any possible implementation manner of the first aspect. The device may further include a memory, and the memory is coupled to the processor. When the processor executes the instructions stored in the memory, the network device may implement the method described in any possible implementation manner of the foregoing first aspect. The device may further include a communication interface, which is used for the device to communicate with other devices. Exemplarily, the communication interface may be a transceiver, a circuit, a bus, a module, or other types of communication interfaces.

The instructions in the memory in this application can be pre-stored or stored after being downloaded from the Internet when using the network device, and the source of the instructions in the memory is not specifically limited in this application. Coupling in this application is an indirect coupling or connection between devices, units or modules, which may be in electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.

A sixth aspect of the present application provides a storage device, where the network device includes: a processor configured to enable the storage device to implement the method described in the foregoing second aspect or any possible implementation manner of the second aspect. The device may further include a memory, and the memory is coupled to the processor. When the processor executes the instructions stored in the memory, the memory device may implement the method described in any possible implementation manner of the foregoing second aspect. The device may further include a communication interface, which is used for the device to communicate with other devices. Exemplarily, the communication interface may be a transceiver, a circuit, a bus, a module, or other types of communication interfaces.

The instructions in the memory in this application can be pre-stored or stored after being downloaded from the Internet when using the storage device, and the source of the instructions in the memory is not specifically limited in this application. Coupling in this application is an indirect coupling or connection between devices, units or modules, which may be in electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.

The seventh aspect of the present application provides a computer storage medium, which may be non-volatile; computer-readable instructions are stored in the computer storage medium, and the first aspect is realized when the computer-readable instructions are executed by a processor Or the method described in any possible implementation of the first aspect.

The eighth aspect of the present application provides a computer storage medium, which may be non-volatile; computer-readable instructions are stored in the computer storage medium, and the second aspect is realized when the computer-readable instructions are executed by a processor Or the method described in any possible implementation of the second aspect.

A ninth aspect of the present application provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the method described in the first aspect or any possible implementation manner of the first aspect.

The tenth aspect of the present application provides a computer program product including instructions, which, when run on a computer, cause the computer to execute the method described in the second aspect or any possible implementation manner of the second aspect.

The eleventh aspect of the present application provides a storage system, where the storage system includes multiple network devices according to the above third aspect or the fifth aspect, and multiple storage devices according to the above fourth aspect or the sixth aspect.

The solutions provided in the above-mentioned third aspect to the eleventh aspect are used to realize or cooperate to realize the method provided in the above-mentioned first aspect or the second aspect, so the same or corresponding beneficial effects can be achieved as in the first aspect or the second aspect. will not be repeated here.

Description of drawings

FIG. 1a is a schematic diagram of a network architecture involved in an embodiment of the present application;

FIG. 1b is a schematic diagram of a network architecture proposed by an embodiment of the present application;

FIG. 2 is a schematic diagram of an application scenario proposed by an embodiment of the present application;

FIG. 3 is a schematic diagram of another application scenario proposed by the embodiment of the present application;

FIG. 4 is a schematic flow diagram of a data reading and writing method provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of an embodiment of a data reading and writing method proposed in the embodiment of the present application;

FIG. 6 is a schematic diagram of an embodiment of another data reading and writing method proposed in the embodiment of the present application;

FIG. 7 is a schematic diagram of another embodiment of a data reading and writing method proposed in the embodiment of the present application;

FIG. 8 is a schematic structural diagram of a network device 800 provided in an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a network device 900 provided in an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a storage device 1000 provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solution and advantages of the present application clearer, the embodiments of the present application will be described below in conjunction with the accompanying drawings. Apparently, the described embodiments are only some of the embodiments of the present application, not all of them. Those skilled in the art know that, with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the descriptions so used are interchangeable under appropriate circumstances such that the embodiments can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to the expressly listed Instead, other steps or modules not explicitly listed or inherent to the process, method, product or apparatus may be included. The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time/logic sequence indicated by the naming or numbering. The execution order of the technical purpose is changed, as long as the same or similar technical effect can be achieved. The division of units presented in this application is a logical division. In actual application, there may be other division methods. For example, multiple units can be combined or integrated in another system, or some features can be ignored. , or not, in addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, and the indirect coupling or communication connection between units may be electrical or other similar forms, this Applications are not limited. Moreover, the units or subunits described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed into multiple circuit units, and some or all of them may be selected according to actual needs unit to realize the purpose of the application scheme.

Although this embodiment of the present application does not limit this, discussions using terms such as "processing", "calculating", "determining", "establishing", "analyzing" and "checking" may refer to computers, computing platforms, computing systems or The operation and/or processing of other electronic computing devices that manipulate and/or convert data represented as physical (e.g., electronic) quantities in computer registers and/or memory into data similarly represented in computer registers and/or memory Other data of physical quantities or other information in a non-transitory storage medium that may store instructions for performing operations and/or processing.

For ease of understanding, technical terms involved in the embodiments of the present application are firstly introduced below.

(1) Input and output (input/output, I/O) messages.

An I/O packet may also be called an I/O command or an I/O request. Specifically, the I/O command can be divided into a read command and a write command, and refers to a command issued by an application program running on the server for instructing to read data from the storage array or write data to the storage array. The server's processor can receive the I/O command and store it in memory so that the I/O command waits to be processed. Specifically, the I/O command may be stored in a queue of the memory.

(2), queue.

A queue is a special linear table that can be deleted at the front end of the table (front) and inserted at the back end (rear) of the table. The end of the insertion operation is called the tail of the queue, and the end of the deletion operation is called the head of the queue. When there are no elements in the queue, it is called an empty queue. The data elements of a queue are also called queue elements. Inserting a queue element into the queue is called enqueuing, and removing a queue element from the queue is called dequeuing. The queue can also be called a first in first out (FIFO) linear list.

There can be many types of queues, for example, send queue (send queue, SQ), receive queue (receive queue, RQ), completion queue (complete queue, CQ), event queue (event queue, EQ).

The sending queue of the sending end (for example: server) and the receiving queue of the receiving end (for example: storage controller) may be referred to as a queue pair (queue pair, QP).

The send queue can be used to store pending I/O commands; the receive queue is used to store and process received I/O commands.

As an example, when the sending end is a server and the receiving end is a storage controller, the sending queue in the server is used to store the I/O commands issued by the server for instructing to read or write data to the storage controller, for example , the I/O command is a read command for reading data stored in the storage controller, the server can send the processed read command to the storage controller, and the read command can include the address and length of the data to be read, etc. , so that the storage controller can send the data it needs to read to the server according to the processed read command.

As another example, the I/O command is a write command for writing data to the storage controller, the server may process the write command, and then send the processed write command to the storage controller, so that the storage controller can process the The subsequent write command searches for memory information in the receiving queue, and stores the data in the processed write command into the memory of the storage controller according to the found memory information.

Similarly, the sending queue in the storage controller may also store the I/O command issued by the storage controller for instructing to read data from the server or write data to the server, and send the command to the server through the storage controller. For details, please refer to the description above, and will not repeat them here.

Taking a distributed storage system as an example, as shown in FIG. 1a, it is a schematic diagram of a network architecture involved in the embodiment of the present application. The client cluster includes multiple clients, the storage array cluster includes multiple storage controllers (such as storage controller A and storage controller N shown in Figure 1a) and storage arrays (not shown in the figure), and the client Establish a connection with the storage controller through a switch. Since the client cluster and the storage array cluster in the distributed storage system usually belong to two independent systems, the client cannot obtain the address information of each storage controller in advance. When the client initiates an I/O request (that is, the client sends an I/O packet to the storage array cluster through the switch), since the client cannot know the storage controller corresponding to the I/O request, the client will Algorithm to determine a storage controller in a storage array cluster. The determined storage controller searches for the attributable storage controller of the I/O message, and reroutes the I/O message to the attributable storage controller to complete data reading or writing. The attributable storage controller refers to the destination of the I/O packet, and the attributable storage controller may also be called a target storage controller. The addressing address in the I/O message is the address of the storage controller (for example, logical unit number (logical unit number, LUN), logical block address (logical block address, LBA)).

Referring to FIG. 1a, the above process is as follows: the I/O message from the client first arrives at the storage controller A, and the storage controller A is randomly determined by the client. Storage controller A decapsulates the I/O packet, searches the mapping relationship table according to the metadata in the I/O packet to determine the storage controller N that processes the I/O packet, and then storage controller A sends the I/O packet to The O packet is re-encapsulated and forwarded to the storage controller N. After receiving the I/O message, the storage controller N re-decapsulates the I/O message to obtain the metadata and obtain the address information of the storage array. The storage controller N performs read and write processing tasks on the storage array according to the I/O message. Subsequent packets of this I/O packet also need to be fed back to the client through the above process.

The process of forwarding the I/O message from the storage controller receiving the I/O message to the storage controller to which the I/O message belongs is called rerouting (also called I/O rerouting). During the rerouting process, I/O packets need to be addressed and forwarded multiple times, so the forwarding path of I/O packets is long, resulting in a long time for data read and write operations to complete, and the number of I/O accesses per second (input /output per second, IOPS) is reduced, affecting data read and write efficiency.

In view of this, the present application proposes a method for reading and writing data. The switch receives address information of the storage controller from the storage controller, and assigns corresponding switching address information to the storage controller. According to the address information of the storage controller and the exchange address information corresponding to the storage controller, the switch establishes a direct path connection with the storage controller, the direct path connection is used to transmit I/O packets, and the I The /O message is used to write data to the storage array through the storage controller or read data from the storage array through the storage control. Since the direct path connection established between the switch and the storage controller can transmit the I/O message, the I/O message of the client does not need to go through the aforementioned rerouting process. It shortens the forwarding path of I/O packets, reduces the completion time of data read and write operations, increases the number of I/O accesses per second (input/output per second, IOPS), and improves the efficiency of data read and write.

In the following, the network architecture applicable to the embodiment of the present application will be introduced first. Please refer to FIG. 1 b , which is a schematic diagram of a network architecture proposed by the embodiment of the present application. The network architecture includes a client cluster, a switch 110, and a storage array cluster, wherein the client cluster includes one or more clients, the storage array cluster includes one or more storage controllers 120, and each storage controller 120 manages one or more storage array (not shown). The I/O request initiated by the client is forwarded to the storage controller 120 through the switch 110 .

In the embodiment of the present application, when the client goes online, the client will establish a network connection with one or more storage controllers through the switch, and the network connection may be a Transmission Control Protocol (TCP) connection. The switch establishes a direct path connection with the storage controller. The storage controller may be a storage controller that has established a network connection with the client, or the storage controller may not have established a network connection with the client. In the present application, the direct path connection is used for direct communication between the switch and the storage controller (that is, communication that does not need to be forwarded by other storage controllers), and the direct path connection is used for transmitting I/O packets.

Describe below in conjunction with accompanying drawing:

(A). Please refer to FIG. 2, which is a schematic diagram of an application scenario proposed by the embodiment of the present application. After the client goes online, establish network connections with storage controller A and storage controller B respectively. The switch establishes connections with the storage controller A and the storage controller B respectively.

A possible implementation manner is: after the client goes online, the client establishes network connections with storage controller A and storage controller B respectively. The switch establishes a direct path connection with each storage controller (storage controller A and storage controller B) respectively.

In another possible implementation manner: after the client goes online, the client establishes network connections with storage controller A and storage controller B respectively. When the client sends an I/O message, the client will randomly determine a storage controller for the I/O message, and the actual destination of the I/O message is the storage controller of the I/O message . When the randomly determined storage controller is consistent with the attributable storage controller, no direct path connection is established between the switch and the attributable storage controller. Optionally, a direct path connection is established between the switch and other storage controllers.

When the randomly determined storage controller is inconsistent with the attributable storage controller, a direct path connection is established between the switch and the attributable storage controller.

In another possible implementation manner: after the client goes online, the client establishes a network connection with storage controller A and storage controller B. When the client sends an I/O message, the switch establishes direct path connections with all storage controllers (storage controller A and storage controller B) according to the I/O message. (B). Please refer to FIG. 3 . FIG. 3 is a schematic diagram of another application scenario proposed by the embodiment of the present application. After the client goes online, establish network connections with storage controller A and storage controller B respectively. The storage controller C in FIG. 3 has not established a network connection with the client. The switch establishes direct path connections with the storage controller A, storage controller B and storage controller C. In a possible example, the attributable storage controller in the I/O request initiated by the client is the storage controller C, but the client has not established a network connection with the storage controller C. After the switch receives the I/O request, the switch establishes a direct path connection with the storage controller C. The switch sends the I/O request to the storage controller C through the direct path connection, so that the storage controller C completes the I/O processing. In this scenario, the switch establishes a diameter path connection with the storage controller C only when it needs to communicate with the storage controller C.

In another possible implementation manner, when the I/O request initiated by the client is not assigned to the storage controller C, and the storage controller C has not established a network connection with the client. After the switch receives the I/O request, the switch establishes a direct path connection with the storage controller C. So that the subsequent I/O request on the storage controller C can be sent to the storage controller C through the direct path connection.

Combining the scenarios shown in FIG. 1b, FIG. 2, and FIG. 3, the embodiment of the present application provides a method for reading and writing data. As shown in FIG. 4 , it is a schematic flowchart of a data reading and writing method provided in the embodiment of the present application, and the data reading and writing method includes steps 401-408.

401. The switch receives routing information from the storage controller.

In this embodiment, the switch receives the routing information from the storage controller, and the routing information includes one or more of the following information: identification information of the storage controller, I/O address of the storage controller, destination is storage control Whether the I/O packets of the controller need to be copied, or the load information of the controller is stored. The I/O address includes but is not limited to: logical unit number (logical unit number, LUN), namespace identifier (name space id, NSID), or logical block address (Logical Block Address, LBA). In this application, it will refer to the load information of the storage array managed by the storage controller, and will be referred to as the load information of the storage controller for short. The load information of the storage controller includes but not limited to: the total storage space of the storage controller, the remaining available storage space of the storage controller, the used storage space of the storage controller, the IOPS of the storage controller, the storage Whether the controller supports active-active mode, or the temperature of the storage controller.

Specifically, the switch first establishes a communication connection with the storage controller. The protocol types used in the communication connection include but are not limited to: Transmission Control Protocol (Transmission Control Protocol, TCP), Remote Direct Memory Access (Remote Direct Memory Access, RDMA) ) connection, or User Datagram Protocol (UDP).

A possible scenario is: the switch establishes a communication connection with the storage controller. When the client comes online, the client configures available storage controllers. The client establishes a communication connection with the storage controller. This communication connection may also be referred to as a control plane connection. The specific process of establishing a control plane connection is as follows:

(a). The switch initiates a monitoring task about the direct path connection.

(b) After the storage controller goes online, the storage controller locally allocates the queue pair port number of the storage controller (or the IP address and port number of the storage controller).

(c) After the switch detects that the storage controller is online, the switch and the storage controller establish a control plane connection through an RDMA link establishment process (or a TCP link establishment process).

After the communication connection (control plane connection) is established between the switch and the storage controller, the switch receives routing information from the storage controller through the communication connection. According to the storage protocol actually adopted by the storage controller, the I/O address in the routing information may be different, which will be described separately below.

In a possible implementation manner, when the storage controller adopts a high-speed non-volatile memory (Non-Volatile Memory Express, NVMe) protocol, the I/O address includes: a logical block address and a namespace identifier. That is, the switch receives the LBA of the storage controller and the NSID of the storage controller from the storage controller.

In another possible implementation manner, when the storage controller adopts the Small Computer System Interface (Small Computer System Interface, SCSI) protocol, the I/O address includes: a logical unit number and a logical block address. That is, the switch receives the LUN of the storage controller and the LBA of the storage controller from the storage controller.

The identification information of the storage controller may include an Internet Protocol (Internet Protocol, IP) address of the storage controller and/or a name of the storage controller. Exemplarily, the identification information of the storage controller is shown in Table 1:

Table 1

Whether the I/O message whose destination is the storage controller needs to be copied refers to whether the storage controller is in an active-active mode. The storage controller is a part of the active-active cluster, and the active-active cluster works in the active-active mode (also referred to as the active-active cluster performing the active-active task). The feature of an active-active cluster is that both clusters are running online and can support the same application load. When the client writes data to the active-active cluster, for example, the client sends an I/O message to the active-active cluster, one of the clusters in the active-active cluster performs read and write operations based on the I/O message, and the cluster Copy the I/O message and send it to another cluster, so that the other cluster stores the data in the I/O message. When a client reads data from an active-active cluster, if one of the active-active clusters fails and the other cluster is still working normally, the client can directly read data through the working cluster. It can be seen that the active-active mode can effectively improve the security of stored data. Similarly, when the storage controller and the backup storage controller work in the active-active mode, after receiving the I/O message, the storage controller copies the I/O message and sends the I/O message to the backup storage controller. message. The data in the I/O message is stored by the backup storage controller.

402. The switch receives address information of the storage controller from the storage controller.

In this embodiment, after the communication connection is established between the switch and the storage controller, the switch receives address information of the storage controller from the storage controller. Depending on the communication connection established between the switch and the storage controller, the address information of the storage controller received by the switch may be different. Each will be described below.

When a TCP connection is established between the switch and the storage controller, the TCP connection is used to transmit the I/O routing information in step 401 and the address information of the storage controller in step 402 . The address information of the storage controller includes: the IP address of the storage controller, the TCP port number and the protocol number of the storage controller. For example, as shown in Table 2:

Table 2

存储控制器storage controller	存储控制器的IP地址IP address of the storage controller	存储控制器的TCP端口号TCP port number of the storage controller	协议号agreement number
存储控制器AStorage controller A	192.168.1.1192.168.1.1	32603260	66
存储控制器Bstorage controller B	192.168.1.2192.168.1.2	32703270	66

When an RDMA connection is established between the switch and the storage controller, the RDMA connection is used to transmit the I/O routing information in step 401 and the address information of the storage controller in step 402 . The address information of the storage controller includes: the queue pair port number QP of the storage controller and the IP address of the storage controller. For example, as shown in Table 3:

table 3

存储控制器storage controller	存储控制器的IP地址IP address of the storage controller	存储控制器的队列对端口号The queue pair port number of the storage controller
存储控制器AStorage controller A	192.168.1.1192.168.1.1	551551
存储控制器Bstorage controller B	192.168.1.2192.168.1.2	552552

It should be noted that the execution order of step 401 and step 402 is not limited, that is, step 401 may be executed first and then step 402 may be executed, or step 402 may be executed first and then step 401 may be executed.

403. The switch allocates corresponding switching address information for the storage controller.

In this embodiment, after the switch receives the address information of the storage controller, the switch assigns the corresponding switching address information to the storage controller. According to the difference in the communication connection established between the switch and the storage controller, similar to step 402, there are differences in the exchange address information, which will be described respectively below:

When a TCP connection is established between the switch and the storage controller, the TCP connection is used to transmit the I/O routing information in step 401 and the switching address information in step 402 . The switching address information includes: the IP address of the switch, the TCP port number and the protocol number of the switch. For example, as shown in Table 4:

Table 4

交换机switch	交换机的IP地址IP address of the switch	交换机的TCP端口号TCP port number of the switch	协议号agreement number
交换机ASwitch A	192.168.1.3192.168.1.3	32903290	66

When an RDMA connection is established between the switch and the storage controller, the RDMA connection is used to transmit the I/O routing information in step 401 and the switching address information in step 402 . The switching address information includes: the queue pair port number QP of the switch and the IP address of the switch. For example, as shown in Table 5:

table 5

交换机switch	交换机的IP地址IP address of the switch	交换机的队列对端口号The queue pair port number of the switch
交换机ASwitch A	192.168.1.3192.168.1.3	556556

404. The switch establishes a direct path connection with the storage controller.

In this embodiment, the switch sends the switching address information corresponding to the storage controller to the storage controller through a communication connection (that is, a control plane connection) with the storage controller. Then the switch side and the storage controller side respectively store the exchange address information and the address information of the storage controller, and the switch and the storage controller establish a direct path connection according to the above address information. The switch may assign identification information of a direct path connection to each direct path connection. The specific process of establishing the direct path connection depends on the type of the direct path connection. When the type of the direct path connection is TCP, the direct path connection is established through a TCP link establishment procedure. When the type of the direct path connection is RDMA, the direct path connection is established through the RDMA link establishment procedure. When the type of the direct path connection is UDP, the direct path connection is established through a UDP link establishment procedure. It should be noted that the type of the direct path connection includes but not limited to TCP, UDP, or RDMA.

Exemplarily, the direct path connection between switch A and storage controller A is expressed as: "storage controller A: 192.168.1.1//3260//6; switch A: 192.168.1.3//3290//6". The identification information of the direct path connection is "controller A-switch A". The direct path connection is of type TCP.

The direct path connection between Switch A and Storage Controller B is expressed as: "Storage Controller B: 192.168.1.2//3270//6; Switch A: 192.168.1.3//3290//6". The identification information of the direct path connection is "controller B-switch A". The direct path connection is of type TCP.

In yet another example, the direct path connection between switch A and storage controller A is expressed as: "storage controller A: 192.168.1.1//551; switch A//192.168.1.3//556". The identification information of the direct path connection is "controller A-switch A". The direct path connection is of type RDMA.

The direct path connection between switch A and storage controller B is expressed as: "storage controller B: 192.168.1.2//552; switch A//192.168.1.3//556". The identification information of the direct path connection is "controller B-switch A". The direct path connection is of type RDMA.

When only one direct path connection is established between any switch and any storage controller, there is a unique corresponding relationship between the direct path connection and the storage controller. Therefore, the identification information of the direct path connection may be the identification information of the storage controller. For example, the direct path connection between switch A and storage controller B is expressed as: "storage controller B: 192.168.1.2//552; switch A//192.168.1.3//556". The identification information of the direct path connection is "controller B".

In another possible implementation manner, multiple direct path connections may be established between any switch and any storage controller. Different direct path connections can carry I/O packets of different clients. An example is given below.

Exemplarily, the direct path connection #1 between switch A and storage controller B is expressed as: "storage controller B: 192.168.1.2//3270//6; switch A: 192.168.1.3//3290//6 ; Client A: 192.168.2.3//3310//6". The identification information of the direct path connection #1 is "controller B-switch A-client A". The direct path connection #1 is of type TCP.

Direct Path Connection #2 between Switch A and Storage Controller B is represented as: "Storage Controller B: 192.168.1.2//3270//6; Switch A: 192.168.1.3//3290//6; Client B : 192.168.2.4//3320//6". The identification information of the direct path connection #2 is "controller B-switch A-client B". The direct path connection #2 is of type TCP.

In yet another example, direct path connection #1 between Switch A and Storage Controller A is represented as: "Storage Controller A: 192.168.1.1//551; Switch A//192.168.1.3//556; Customer Terminal A//192.168.2.3//540". The identification information of the direct path connection is "controller A-switch A-client A". The direct path connection is of type RDMA.

Direct Path Connection #2 between Switch A and Storage Controller B is represented as: "Storage Controller B: 192.168.1.2//552; Switch A//192.168.1.3//556; Client B//192.168.2.4 //541". The identification information of the direct path connection is "controller B-switch A-client B". The direct path connection is of type RDMA.

It should be noted that, according to different protocols used by the direct path connection, there are other implementation solutions for the direct path connection, which are not limited here. The foregoing example of the direct path connection may also include other information, for example, the status of the direct path connection, etc., which is not limited here.

405. The switch generates a first mapping relationship.

In this embodiment, the switch generates the first mapping relationship, and there are multiple possible implementations of the first mapping relationship, which will be described respectively below.

(a) When only one direct path connection is established between any switch and any storage controller, there is a unique corresponding relationship between the direct path connection and the storage controller. Therefore, the identification information of the direct path connection may be the identification information of the storage controller. The first mapping relationship includes identification information and I/O addresses of storage controllers.

Exemplarily, the first mapping relationship is shown in Table 6:

Table 6

Optionally, the first mapping relationship includes: a mapping relationship between identification information of the direct path connection, identification information of the storage controller, and an I/O address. Exemplarily, the first mapping relationship is shown in Table 7:

Table 7

Optionally, the first mapping relationship may be a hash (hash) table of key-value pairs (key-value). The key (key) of the table is the I/O address, and the value (value) of the table is the identification information of the direct path connection and the identification information of the storage controller.

(b) Multiple direct path connections can be established between any switch and any storage controller. Different direct path connections can carry I/O packets of different clients.

Optionally, the first mapping relationship includes: identification information of the direct path, an I/O address, identification information of the storage controller, and identification information of the client. Exemplarily, the first mapping relationship is shown in Table 8:

Table 8

The first mapping relationship may also include other information of the direct path connection, including but not limited to: the TCP port number of the storage controller, the IP address of the storage controller, the IP address of the switch, the port number of the switch, the QP of the storage controller, QP and protocol number of the switch, or connection status information (such as packet sequence number, etc.).

406. The client sends the first I/O packet to the switch.

In this embodiment, the client sends the first I/O packet to the switch, and the first I/O packet is used to write data to the storage array through the storage controller or read data from the storage array through the storage controller.

In a possible implementation manner, step 406 is performed before step 401, and after the client sends the first I/O packet to the switch, steps 401-405 are performed. That is, after the switch receives the I/O packet from the client, a direct path connection is established between the switch and the storage controller.

In another possible implementation manner, after steps 401-405, step 406 is performed. That is, after a direct path connection is established between the switch and the storage controller, the switch receives I/O packets from the client.

407. The switch determines the direct path connection according to the first I/O packet and the first mapping relationship.

In this embodiment, after the switch receives the first I/O message from the client, the switch determines the direct path for forwarding the first I/O message from the first mapping relationship according to the first I/O message connect.

According to different implementations of the first mapping relationship, there are multiple implementation solutions, which will be described respectively below.

Optionally, when the first mapping relationship includes: identification information and an I/O address of the direct path connection. After the switch receives the first I/O message, the switch first analyzes the address information (including source address information and destination address information) in the first I/O message. The switch determines the identification information of the direct path connection according to the I/O address in the first I/O message. The switch sends the first I/O packet by using the direct path connection. In this case, there is only one direct path connection per storage controller.

Optionally, when the first mapping relationship includes: identification information of the direct path connection, identification information of the storage controller, and an I/O address. After the switch receives the first I/O message, the switch first analyzes the address information (including source address information and destination address information) in the first I/O message. The switch determines the address information of the corresponding storage controller according to the I/O address in the first I/O message. Secondly, the switch determines the identification information of the direct path connection according to the address information of the storage controller. The switch sends the first I/O packet by using the direct path connection.

After the switch receives the first I/O message, the switch first analyzes the address information (including source address information and destination address information) in the first I/O message. Secondly, according to the I/O address in the first I/O message, the address information of the corresponding storage controller is determined. Thirdly, the switch determines the source address information of the I/O message according to the first I/O message, that is, the address information of the client corresponding to the first I/O message. Thirdly, the switch determines the identification information of the client according to the address information of the client. In a possible implementation manner, the address information of the client is consistent with the identification information of the client. Thirdly, the switch determines the corresponding identification information of the direct path connection according to the identification information of the client. The switch sends the first I/O packet by using the direct path connection.

In a possible implementation, after the switch receives the first I/O message, firstly, the switch parses address information (including source address information and destination address information) in the first I/O message. Secondly, according to the I/O address in the first I/O message, the address information of the corresponding storage controller is determined. Again, the switch detects the number of direct path connections corresponding to the storage controller. When the storage controller corresponds to only one direct path connection, the switch directly uses the direct path connection to send the first I/O packet.

When the storage controller corresponds to multiple direct path connections, the switch determines the first I/O according to the source address information in the first I/O message, that is, the address information of the client corresponding to the first I/O message /O which client the packet comes from. Thirdly, the switch determines the identification information of the client according to the address information of the client. In a possible implementation manner, the address information of the client is consistent with the identification information of the client. Thirdly, the switch determines the corresponding identification information of the direct path connection according to the identification information of the client. The switch sends the first I/O packet by using the direct path connection. Exemplarily, when the switch resolves the address information in the first I/O message as "LBA-A and NSID-A", the switch determines that the destination of the first I/O message is Storage controller A, the identification information of the storage controller A is "192.168.1.1/target100.com". The switch determines that the identification information of the corresponding direct path connection is "controller A-switch A" according to the first mapping relationship. The switch determines the direct path connection according to the identification information of the direct path connection.

Optionally, the switch may dynamically select an appropriate storage controller to receive the first I/O packet according to status information reported by each storage controller. The suitable storage controller here can be a storage controller with larger available space, or a storage controller with stronger IOPS performance, or a storage controller with lower processing load of the central processing unit (CPU) of the storage controller. Controller, not limited here.

Optionally, the switch detects the first I/O message, when the destination (storage controller) of the first I/O message is a storage controller implementing the active-active mode. Then the switch copies the first I/O packet. The switch sends the first I/O packet to a backup storage controller of the storage controller, and the storage controller and the backup storage controller work in a dual-active mode. The switch adds an identifier indicating that the message is a copy message to the first I/O message sent by the switch to the backup storage controller.

Optionally, the switch detects the first I/O message, and when the first I/O message is a data message, the switch executes step 408, and sends the first I/O message to the storage controller through a direct path connection. arts. When the first I/O packet is a non-data packet (for example, a control packet), the switch transparently transmits the first I/O packet.

408. The switch sends the first I/O packet to the storage controller through the direct path connection.

Specifically, after determining the direct path connection, the switch replaces the destination address in the packet header of the first I/O packet with the address information of the storage controller included in the direct path connection. For example: the type of the direct path connection is RDMA as an example. The destination address in the message header of the first I/O message is replaced with "192.168.2.4//541", then the first I/O message is connected to "controller B-switch A-client B" through the direct path ” to storage controller B.

Optionally, in addition to sending the first I/O packet to the storage controller through the direct path connection, the switch may also send the first indication information to the second storage controller. The first indication information instructs the second storage controller to update the local serial number of the received message, and the second storage controller is the storage controller initially allocated by the client for the first I/O message. The first indication information instructs the second storage controller to update the local serial number of the received message, so as to avoid the out-of-sequence problem caused by the second storage controller not receiving the first I/O message.

The present application proposes a method for reading and writing data. A switch receives address information of the storage controller from a storage controller, and assigns corresponding switching address information to the storage controller. According to the address information of the storage controller and the exchange address information corresponding to the storage controller, the switch establishes a direct path connection with the storage controller, the direct path connection is used to transmit I/O packets, and the I The /O message is used to write data to the storage array through the storage controller or read data from the storage array through the storage control. Since the direct path connection established between the switch and the storage controller can transmit the I/O message, the I/O message of the client does not need to go through the aforementioned rerouting process. It avoids multiple addressing for I/O processing, shortens the forwarding path of I/O packets, reduces the completion time of data read and write operations, improves IOPS performance, and improves data read and write efficiency.

With reference to the foregoing embodiments, after the storage controller receives the first I/O message, the storage controller may send a reply message to the client in various ways. Description will be made below in conjunction with the accompanying drawings.

(1) Please refer to FIG. 5 , which is a schematic diagram of an embodiment of a data reading and writing method proposed in the embodiment of the present application. The data reading and writing method includes steps 501-504.

501. The client sends a first I/O packet to the switch.

502. The switch is connected through the direct path, and sends the first I/O packet to the storage controller. The storage controller writes data to the storage array or reads data from the storage array according to the first I/O message.

503. The switch receives the first reply packet from the storage controller through the direct path connection. The first reply message is a response to the first I/O message. The destination of the first reply message is the client.

504. The switch sends a first reply message to the client.

In a possible implementation manner, the switch sends the first reply message to the client through the communication connection established between the client and the storage controller. For example, the switch sends the first reply message to the client through the TCP connection established between the client and the storage controller.

In another possible implementation manner, a direct path connection is established between the switch and the client. The specific manner of establishing the direct path connection is similar to the manner of establishing the direct path connection between the switch and the storage controller in the embodiment in FIG. 4 . The switch sends the first reply message to the client through the direct path connection.

In the embodiment of the present application, the storage controller sends the first reply message to the switch through the direct path connection, so that the client does not perceive the routing path of the first reply message.

(2) Please refer to FIG. 6 . FIG. 6 is a schematic diagram of an embodiment of another data reading and writing method proposed in the embodiment of the present application. The data reading and writing method includes steps 601-606.

601. The client sends a first I/O packet to the switch.

602. The switch is connected through the direct path, and sends the first I/O packet to the storage controller. The storage controller writes data to the storage array or reads data from the storage array according to the first I/O message.

603. The switch is connected through the direct path, and receives the second reply packet from the storage controller.

The destination of the second reply message is the second storage controller. The second storage controller is the storage controller initially allocated by the client for the first I/O packet. That is, the second storage controller is a storage controller randomly assigned to the first I/O message by the client before sending the first I/O message.

The second reply packet includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply packet to the client on behalf of the storage controller.

In another possible implementation manner, there is a network connection between the storage controllers. The storage controller sends the second reply message to the second storage controller through the network connection. At this time, the corresponding switch performs network forwarding, and forwards the second reply message from the storage controller to the second storage controller.

604. The switch forwards the second reply packet to the second storage controller.

605. After receiving the second reply message from the switch, the second storage controller generates a third reply message according to the second reply message, where the destination of the third reply message is the client, and the second The storage controller sends the modified third reply message to the switch.

The second storage controller determines that the message needs to be sent to the client according to the proxy indication information included in the second reply message. The second storage controller generates a third reply message according to the second reply message, the destination of the third reply message is the client, and the third reply message does not include the proxy indication information.

606. After receiving the third reply packet from the second storage controller, the switch sends the third reply packet to the client.

In a possible implementation manner, the switch sends the third reply message to the client through the communication connection established between the client and the storage controller. For example, the switch sends the third reply message to the client through the TCP connection established between the client and the storage controller.

In another possible implementation manner, a direct path connection is established between the switch and the client. The specific manner of establishing the direct path connection is similar to the manner of establishing the direct path connection between the switch and the storage controller in the embodiment in FIG. 4 . The switch sends the third reply message to the client through the direct path connection.

In the embodiment of the present application, the storage controller sends the third reply message to the switch through the direct path connection, so that the client does not perceive the routing path of the third reply message.

In combination with the foregoing embodiments, the data reading and writing methods proposed in the embodiments of the present application can also be applied to other systems. The following description will be made in conjunction with the accompanying drawings. Please refer to FIG. 7, which is a schematic diagram of another embodiment of a data reading and writing method proposed in the embodiment of the present application. The data reading and writing method includes steps 701-708.

701. The switch receives routing information from the front-end server.

In this embodiment, when the client accesses the storage system, for example, the user clicks a certain video in the online video software, the client (online video software) accesses the storage system. The client establishes a connection with the front-end server, and the connection may be a control plane connection, such as the TCP connection or the RDMA connection in the foregoing embodiments.

After the client establishes a connection with the front-end server, the switch receives routing information from the front-end server. The routing information of the front-end server includes but not limited to: identification information of the back-end server, I/O address of the back-end server, data storage address, or load information of the back-end server. The I/O address includes, but is not limited to: logical unit number, namespace identifier, or logical block address.

The identification information of the backend server may include the IP address of the backend server and/or the name of the backend server. The storage address of the data refers to the storage address of the data in the backend server, and the data may be business-related data. For example: video, audio, text or image, etc.

The load information of the back-end server includes, but is not limited to: the total storage space of the back-end server, the remaining available storage space of the back-end server, the used storage space of the back-end server, the IOPS performance of the back-end server, the The available bandwidth of the backend server, the total bandwidth of the backend server, or the temperature of the storage controller.

Here, the backend server is equivalent to the storage controller in the foregoing embodiments.

702. The switch receives address information from the front-end server, where the address information is address information of the back-end server.

In this embodiment, after a communication connection is established between the switch and the front-end server, the switch receives address information of the back-end server from the front-end server. Depending on the communication connection established between the front-end server and the back-end server, the address information of the back-end server received by the switch may be different. Each will be described below.

When a TCP connection is established between the front-end server and the back-end server, the address information of the back-end server includes: the IP address of the back-end server, the port number and the protocol number of the back-end server

When an RDMA connection is established between the front-end server and the back-end server, the address information of the back-end server includes: the queue pair port number QP of the back-end server and the IP address of the back-end server.

It should be noted that the execution sequence of step 701 and step 702 is not limited, that is, step 701 may be executed first and then step 702 may be executed, or step 702 may be executed first and then step 701 may be executed.

In this application, the address information of the storage controller may be sent to the switch by the storage controller itself, or may be sent to the switch through other devices, for example, the front-end server in this embodiment.

703. The switch stores switching address information corresponding to the backend server.

In this embodiment, after the switch receives the address information of the backend server, the switch assigns the corresponding switching address information to the backend server. According to the difference in the communication connection established between the front-end server and the back-end server, similar to step 702, there are differences in the exchange address information, which will be explained separately below:

When a TCP connection is established between the front-end server and the back-end server, the exchange address information includes: the IP address of the switch, the port number and the protocol number of the switch.

When an RDMA connection is established between the front-end server and the back-end server, the exchange address information includes: the queue pair port number QP of the switch and the IP address of the switch.

704. The switch establishes a direct path connection with the backend server.

In this embodiment, the switch sends the exchange address information corresponding to the back-end server to the back-end server through a communication connection (that is, a control plane connection) with the back-end server. The exchange address information and the address information of the back-end server are respectively stored on the switch side and the back-end server side, and the switch and the back-end server establish a direct path connection according to the above address information. The switch may assign identification information of a direct path connection to each direct path connection.

705. The switch generates a second mapping relationship.

In this embodiment, the switch generates a second mapping relationship, where the second mapping relationship includes: a mapping relationship between identification information of a direct path connection, identification information of a backend server, and an I/O address. The second mapping relationship is an implementation manner of the foregoing first mapping relationship.

706. The client sends the second packet to the switch.

In this embodiment, the client sends a second message to the switch, where the second message is used to read or write data to the backend server. For example, when the client needs to upload video to the storage system, the client sends a second message to the switch, and the second message carries the video data. When the client needs to watch the video, the client sends a second message to the switch, and the second message carries the address or identifier of the video data to be read. The second message is an implementation manner of the foregoing first message.

In a possible implementation manner, step 706 is performed before step 701, and after the client sends the second packet to the switch, steps 701-705 are performed. That is, after the switch receives the second message from the client, a direct path connection is established between the switch and the backend server.

In another possible implementation manner, after steps 701-705, step 706 is performed. That is, after a direct path connection is established between the switch and the backend server, the switch receives the second packet from the client.

Optionally, before the switch establishes a direct path connection with the back-end server, the switch can transparently transmit messages such as link establishment and authorization verification sent by the client to the front-end server. The front-end server processes the above-mentioned messages such as link establishment or authorization verification.

707. The switch determines the direct path connection according to the second packet and the second mapping relationship.

In this embodiment, after the switch receives the second message from the client, the switch determines the direct path connection used to forward the second message from the second mapping relationship according to the second message.

708. The switch sends the second packet to the storage backend server through the direct path connection.

This application proposes a method for reading and writing data. Since the direct path connection established between the switch and the back-end server can transmit messages, the messages of the client do not need to be re-routed by the front-end server. It avoids the communication congestion caused by the computing power of the front-end server and the network bandwidth, improves the processing efficiency of the message, improves the utilization efficiency of the back-end server, and realizes the load balancing of each back-end server.

In order to implement the foregoing embodiments, the present application further provides a network device. Refer to FIG. 8 , which is a schematic structural diagram of a network device 800 provided in an embodiment of the present application.

Although the network device 800 shown in FIG. 8 shows some specific features, those skilled in the art will realize from the embodiments of the present application that for the sake of brevity, various other features are not shown in FIG. 8 so as not to confuse the present invention. Further relevant aspects of the embodiments disclosed in the application examples. To this end, as an example, in some implementations, the network device 800 includes one or more processing modules (e.g., a CPU) 801, a network interface 802, a programming interface 803, a memory 804, and one or more communication buses 805 for Interconnect the various components. In other implementation manners, the network device 800 may also omit or add some functional components or units based on the above examples.

In some implementations, the network interface 802 is used to connect with one or more other network devices/servers in the network system. In some implementations, communication bus 805 includes circuitry that interconnects and controls communication between system components. Memory 804 may include nonvolatile memory, for example, read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM) , Electrically Erasable Programmable Read-Only Memory (electrically EPROM, EEPROM) or flash memory. Memory 804 may also include volatile memory, which may be random access memory (RAM), which acts as an external cache.

In some implementations, the memory 804 or the non-transitory computer-readable storage medium of the memory 804 stores the following programs, modules and data structures, or a subset thereof, for example including a transceiver module (not shown in the figure), a transceiver module 8041 and a processing Module 8042.

In a possible embodiment, the network device 800 may have any function of the switch in the above method embodiments corresponding to FIG. 2 to FIG. 7 .

It should be understood that the network device 800 corresponds to the switch in the above-mentioned method embodiment, and each module in the network device 800 and the above-mentioned other operations and/or functions are to implement various steps and methods implemented by the switch in the above-mentioned method embodiment, respectively, For specific details, refer to the method embodiments corresponding to the foregoing FIGS. 2-7 , and details are not repeated here for the sake of brevity.

It should be understood that in this application, the network interface 802 on the network device 800 can complete the data sending and receiving operation, or the processor can call the program code in the memory, and cooperate with the network interface 802 to realize the function of the sending and receiving module when necessary .

In various implementations, the network device 800 is configured to execute the data reading and writing method provided by the embodiment of the present application, for example, executing the data reading and writing method corresponding to the above-mentioned embodiments shown in FIGS. 2-7 .

The specific structure of the network device described in FIG. 8 of this application may be as shown in FIG. 9 .

FIG. 9 is a schematic structural diagram of a network device 900 provided by an embodiment of the present application. The network device 900 includes: a main control board 910 and an interface board 930 .

The main control board 910 is also called a main processing unit (main processing unit, MPU) or a route processor (route processor), and the main control board 910 is used for controlling and managing each component in the network device 900, including route calculation, device management , equipment maintenance, protocol processing functions. The main control board 910 includes: a CPU 911 and a memory 912 .

The interface board 930 is also called a line processing unit (line processing unit, LPU), a line card (line card), or a service board. The interface board 930 is used to provide various service interfaces and implement forwarding of data packets. Service interfaces include but are not limited to Ethernet interfaces, POS (Packet over SONET/SDH) interfaces, etc. The interface board 930 includes: a central processing unit 931 , a network processor 932 , a forwarding entry storage 934 and a physical interface card (physical interface card, PIC) 933 .

The CPU 931 on the interface board 930 is used to control and manage the interface board 930 and communicate with the CPU 911 on the main control board 910 .

The network processor 932 is configured to implement message forwarding processing. The form of the network processor 932 may be a forwarding chip.

The physical interface card 933 is used to realize the docking function of the physical layer, and the original flow enters the interface board 930 through this, and the message after processing is sent from the physical interface card 933. The physical interface card 933 includes at least one physical interface, which is also called a physical interface, and the physical interface may be a Flexible Ethernet (FlexE) physical interface. The physical interface card 933 is also called a daughter card, which can be installed on the interface board 930, and is responsible for converting the photoelectric signal into a message, checking the validity of the message and forwarding it to the network processor 932 for processing. In some embodiments, the central processing unit 931 of the interface board 930 can also execute the functions of the network processor 932 , such as implementing software forwarding based on a general-purpose CPU, so that the interface board 930 does not need the network processor 932 .

Optionally, the network device 900 includes multiple interface boards. For example, the network device 900 further includes an interface board 940 , and the interface board 940 includes: a central processing unit 941 , a network processor 942 , a forwarding entry storage 944 and a physical interface card 943 .

Optionally, the network device 900 further includes a switching fabric unit 920 . The SFU 920 may also be called a SFU (switch fabric unit, SFU). When the network device has multiple interface boards 930, the switching fabric board 920 is used to complete the data exchange between the interface boards. For example, the interface board 930 and the interface board 940 may communicate through the switching fabric board 920 .

The main control board 910 is coupled to the interface board. For example, the main control board 910, the interface board 930 and the interface board 940, and the switching fabric board 920 are connected through a system bus and/or a system backplane to implement intercommunication. In a possible implementation manner, an inter-process communication protocol (inter-process communication, IPC) channel is established between the main control board 910 and the interface board 930, and the main control board 910 and the interface board 930 communicate through the IPC channel.

Logically, the network device 900 includes a control plane and a forwarding plane. The control plane includes a main control board 910 and a central processing unit 931. The forwarding plane includes various components for performing forwarding, such as a forwarding entry storage 934, a physical interface card 933, and a network processing device 932. The control plane performs functions such as publishing routes, generating forwarding tables, processing signaling and protocol packets, configuring and maintaining device status, etc., and the control plane sends the generated forwarding tables to the forwarding plane. On the forwarding plane, the network processor 932 The forwarding table issued above looks up and forwards the packets received by the physical interface card 933. The forwarding table issued by the control plane may be stored in the forwarding table item storage 934 . In some embodiments, the control plane and the forwarding plane may be completely separated and not on the same device.

It should be understood that the transceiver module in the network device 800 may be equivalent to the physical interface card 933 or the physical interface card 943 in the network device 900; the transceiver module 8041 and the processing module 8042 in the network device 800 may be equivalent to the central processing module The processor 911 or the central processing unit 931 may also correspond to program codes or instructions stored in the memory 912.

It should be understood that the operations on the interface board 940 in the embodiment of the present application are consistent with the operations on the interface board 930 , and are not repeated for brevity. It should be understood that the network device 900 in this embodiment may correspond to the switch in each of the foregoing method embodiments, and the main control board 910, the interface board 930, and/or the interface board 940 in the network device 900 may implement the For the sake of brevity, the functions and/or various steps implemented by the switch are not repeated here.

It is worth noting that there may be one or more main control boards, and when there are multiple main control boards, it may include the main main control board and the standby main control board. There may be one or more interface boards. The stronger the data processing capability of the network device, the more interface boards it provides. There may also be one or more physical interface cards on the interface board. There may be no SFU, or there may be one or more SFUs. When there are multiple SFUs, they can jointly implement load sharing and redundant backup. Under the centralized forwarding architecture, the network device does not need a switching network board, and the interface board undertakes the processing function of the service data of the entire system. Under the distributed forwarding architecture, the network device can have at least one SFU, and the data exchange between multiple interface boards can be realized through the SFU to provide large-capacity data exchange and processing capabilities. Optionally, the form of the network device can also be that there is only one board, that is, there is no switching fabric board, and the functions of the interface board and the main control board are integrated on this board. At this time, the central processing unit and the main control board on the interface board The central processing units on the board can be combined into one central processing unit on the one board to perform the superimposed functions of the two. Which architecture to use depends on the specific networking deployment scenario, and there is no unique limitation here.

In some possible embodiments, the foregoing switch may be implemented as a virtualization device. The virtualization device may be a virtual machine (virtual machine, VM) running a program for sending packets, a virtual router or a virtual switch. Virtualization devices are deployed on hardware devices (eg, physical servers). For example, a switch may be implemented based on a common physical server combined with a network functions virtualization (network functions virtualization, NFV) technology.

It should be understood that the above-mentioned network devices in various product forms respectively have any function of the switch in the above-mentioned method embodiments, which will not be repeated here.

FIG. 10 is a schematic structural diagram of a storage device 1000 provided by an embodiment of the present application. The storage device 1000 shown in FIG. 10 is a storage array. As shown in Figure 10, the storage device 1000 may include a storage controller 1100 and a disk array 1200, wherein the disk array 1200 here is used to provide storage space, and may include a cheap redundant array of independent disk (RAID for short) Or a disk enclosure containing multiple disks. There may be multiple disk arrays 1200 , and the disk array 1200 includes multiple disks 1202 . Disk 1202 is used to store data. The disk array 1200 communicates with the controller 1100 through communication protocols such as SCSI protocol. Agreement is not limited here.

The storage controller 1100 in the storage device 1000 is configured to execute relevant steps in the foregoing method embodiments.

It can be understood that the disk array 1200 is just an example of the memory in the storage device. In this embodiment of the present application, the data may also be stored through a memory such as a tape library. It should be noted that the magnetic disk 1202 is also only an example of the memory for constructing the magnetic disk array 1200 . In practical applications, for example, in order to build a disk array between cabinets containing multiple disks, there may also be an implementation. Therefore, in the embodiment of the present application, the disk array 1200 may also include a memory, including a non-volatile storage medium, such as a solid state disk (solid state disk, SSD for short), a cabinet containing multiple disks, or a server, which will not be described here. limited.

The storage controller 1100 is the “brain” of the storage device 1000 , and mainly includes a processor 1102 , a cache 1103 , a memory 1101 , a communication bus (bus for short) 1105 and a communication interface 1104 . The processor 1102 , the cache memory 1103 , the memory 1101 and the communication interface 1104 communicate with each other through the communication bus 1105 . It should be noted that, in the embodiment of the present application, there may be one or more controllers 1100 in the storage device 1000 . It can be understood that when the storage device 1000 includes at least two controllers 1100, the stability of the storage device 1000 can be improved.

The communication interface 1104 is used for communicating with switches, clients, other network devices or other storage devices.

The memory 1101 is used to store a program 1106 . The memory 1101 may include a high-speed random access memory (random access memory, RAM for short), or may also include a non-volatile memory, such as at least one disk memory. It can be understood that the memory 1101 can be various non-transitory machine-readable media that can store program codes, such as RAM, magnetic disk, hard disk drive, optical disk, SSD or non-volatile memory.

Program 1106 may include program code including computer operating instructions.

Cache 1103 is the memory between the controller and the hard drive, which is smaller in capacity but faster than the hard drive. The cache 1103 is used to temporarily store data, such as I/O transactions received from switches or other storage devices, and temporarily store data read from the disk 1202, so as to improve the performance and reliability of the array. The cache 1103 can be various non-transitory machine-readable media that can store data, such as RAM, ROM, flash memory or SSD, which is not limited here.

The processor 1102 may be a central processing unit (central processing unit, CPU for short) or an application-specific integrated circuit (ASIC for short), or configured to implement one or more integrated circuits in the embodiments of the present application. An operating system and other software programs are installed in the processor 1102, and different software programs can be regarded as different processing modules with different functions, such as processing the input/output (input/output, referred to as I/O) requests of the disk 1202 . Perform other processing on the data in the disk 1202 , or modify the metadata stored in the storage device 1000 . The storage controller 1100 can implement various data management functions such as I/O operations, snapshots, mirroring, and duplication. In the embodiment of the present application, the processor 1102 is configured to execute the program 1106, specifically, relevant steps in the aforementioned method embodiments may be executed.

Further, an embodiment of the present application also provides a computer program product, which, when running on a network device, causes the network device to perform the method performed by the switch in the method embodiments corresponding to FIGS. 2-7 above.

The embodiment of the present application also provides a chip system, including a processor and an interface circuit, and the interface circuit is configured to receive instructions and transmit them to the processor. Wherein, the processor is configured to implement the method in any one of the foregoing method embodiments.

Optionally, the chip system further includes a memory, and there may be one or more processors in the chip system. The processor can be realized by hardware or by software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, or the like. When implemented by software, the processor may be a general-purpose processor, and implements the method in any of the above method embodiments by reading the software code stored in the memory.

Optionally, there may be one or more memories in the chip system. The memory can be integrated with the processor, or can be set separately from the processor, which is not limited in this application. Exemplarily, the memory can be a non-transitory processor, such as a read-only memory ROM, which can be integrated with the processor on the same chip, or can be respectively arranged on different chips. The setting method of the processor is not specifically limited.

The above is a detailed introduction to the embodiment of the present application. The steps in the method of the embodiment of the present application can be sequentially scheduled, merged or deleted according to actual needs; the modules in the device of the embodiment of the present application can be divided, merged or deleted according to actual needs .

It should be understood that reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Thus, appearances of "in one embodiment" or "in an embodiment" in various places throughout the specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present application. The implementation process constitutes any limitation.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

It should be understood that in this embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B may also be determined according to A and/or other information.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separated, and a component shown as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

Claims

A method for reading and writing data, characterized in that, comprising:

The switch receives the address information of the storage controller, and the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the Internet Protocol address of the storage controller, the storage control The transmission control protocol TCP port number or protocol number of the device;

The switch allocates corresponding switching address information for the storage controller, and the switching address information includes one or more of the following: the queue pair port number of the switch, the Internet protocol address of the switch, or the The port number;

The switch establishes a direct path connection with the storage controller, the direct path connection is used to transmit input and output I/O packets, and the I/O packets are used to write to the storage array through the storage controller data or read data from the storage array through the storage control.
The method according to claim 1, further comprising:

The switch receives routing information from the storage controller, and the routing information includes one or more of the following information: identification information of the storage controller, input and output I/O addresses of the storage controller . Whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller,

The I/O address includes a logical unit number, a namespace identifier and/or a logical block address.
The method according to claim 2, further comprising:

The switch generates a first mapping relationship, where the first mapping relationship includes a mapping relationship between the identification information of the direct path connection and the I/O address.
The method according to claim 3, further comprising:

The switch receives the first I/O message from the client;

The switch determines the direct path connection according to the first I/O message and the first mapping relationship;

The switch sends the first I/O message to the storage controller through the direct path connection.
The method according to claim 4, characterized in that the method further comprises:

The switch detects the type of the first I/O packet;

When the first I/O message is a data message, the switch sends the first I/O message to the storage controller through the direct path connection;

When the first I/O packet is a non-data packet, the switch transparently transmits the first I/O packet.
The method according to any one of claims 4-5, wherein the method further comprises:

The switch receives a first reply message from the storage controller through the direct path connection, and the first reply message is a response to the first I/O message;

The switch sends the first reply message to the client.
According to the method described in any one of claims 4-5, the method further comprises:

The switch receives a second reply message from the storage controller, the second reply message is a response to the first I/O message, and the destination of the second reply message is the second a storage controller, where the second storage controller is a storage controller initially allocated by the client for the first I/O message;

forwarding, by the switch, the second reply message to the second storage controller;

The switch receives a third reply message from the second storage controller, where the third reply message is generated according to the second reply message;

The switch forwards the third reply packet from the second storage controller.
The method according to claim 7, wherein the second reply message includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply message on behalf of the storage controller. The message is sent to the client, and the third reply message does not include the proxy indication information.
The method according to any one of claims 4-8, further comprising:

When the first I/O message needs to be copied, the switch copies the first I/O message;

The switch sends the duplicated first I/O packet to a backup storage controller of the storage controller, and the backup storage controller and the storage controller work in a dual-active mode.
The method according to any one of claims 1-9, characterized in that,

The direct path connection is a transmission control protocol TCP connection, or a remote direct data storage RDMA connection.
A method for reading and writing data, characterized in that, comprising:

The storage controller sends the address information of the storage controller to the switch, and the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the Internet protocol of the storage controller address, transmission control protocol TCP port number, or protocol number of the storage controller;

The storage controller establishes a direct path connection with the switch, the direct path connection is used to transmit input and output I/O packets, and the I/O packets are used to write to the storage array through the storage controller data or read data from the storage array through the storage control.
The method according to claim 11, characterized in that the method further comprises:

The storage controller sends routing information of the storage controller to the switch, where the routing information includes one or more of the following information: identification information of the storage controller, input of the storage controller Outputting the I/O address, whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller,

The I/O address includes a logical unit number, a namespace identifier and/or a logical block address.
The method according to any one of claims 11-12, wherein the method further comprises:

The storage controller receives a first I/O packet from the switch through the direct path connection.
The method according to claim 13, further comprising:

The storage controller generates a first reply message according to the first I/O message, and the first reply message is a response to the first I/O message;

The storage controller sends the first reply message to the switch through the direct path connection.
The method according to claim 13, further comprising:

The storage controller generates a second reply message according to the first I/O message, and the second reply message is a response to the first I/O message;

The storage controller sends the second reply message to the switch, the destination of the second reply message is a second storage controller, and the second storage controller is the client for the The storage controller to which the first I/O message is initially allocated.
The method according to claim 15, wherein the second reply message includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply message on behalf of the storage controller. The message is sent to the client.
The method according to any one of claims 11-16, wherein the direct path connection is a Transmission Control Protocol TCP connection, or a remote direct data storage RDMA connection.
A switch, characterized by comprising: a transceiver module and a processing module;

The transceiver module is used to receive the address information of the storage controller, the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the Internet protocol of the storage controller address, transmission control protocol TCP port number, or protocol number of the storage controller;

The processing module is configured to assign corresponding switching address information to the storage controller, and the switching address information includes one or more of the following: the queue pair port number of the switch, the Internet Protocol address of the switch, or the port number of the switch;

The processing module is further configured to establish a direct path connection with the storage controller, the direct path connection is used to transmit input and output I/O messages, and the I/O messages are used to pass through the storage controller Data is written to or read from the storage array through the storage control.
The switch according to claim 18, characterized in that,

The transceiver module is further configured to receive routing information from the storage controller, where the routing information includes one or more of the following information: identification information of the storage controller, input of the storage controller Outputting the I/O address, whether the I/O message whose destination is the storage controller needs to be copied, or the load information of the storage controller, the I/O address includes the logical unit number, the identifier of the namespace and /or logical block address.
The switch according to claim 19, characterized in that,

The processing module is further configured to generate a first mapping relationship, where the first mapping relationship includes a mapping relationship between the identification information of the direct path connection and the I/O address.
The switch according to claim 20, characterized in that,

The transceiver module is also used to receive the first I/O message from the client;

The processing module is further configured to determine the direct path connection according to the first I/O message and the first mapping relationship;

The transceiver module is further configured to send the first I/O message to the storage controller through the direct path connection.
The switch according to claim 21, characterized in that,

The processing module is further configured to detect the type of the first I/O message;

The transceiver module is further configured to send the first I/O message to the storage controller through the direct path connection when the first I/O message is a data message;

The transceiver module is further configured to transparently transmit the first I/O message when the first I/O message is a non-data message.
The switch according to any one of claims 21-22, characterized in that,

The transceiver module is further configured to receive a first reply message from the storage controller through the direct path connection, where the first reply message is a response to the first I/O message;

The transceiver module is further configured to send the first reply message to the client.
The switch according to any one of claims 22-23, characterized in that,

The transceiver module is further configured to receive a second reply message from the storage controller, the second reply message is a response to the first I/O message, and the second reply message is The destination is a second storage controller, where the second storage controller is the storage controller initially allocated by the client for the first I/O message;

The transceiver module is further configured to forward the second reply message to the second storage controller;

The transceiver module is further configured to receive a third reply message from the second storage controller, the third reply message is generated according to the second reply message;

The transceiver module is further configured to forward the third reply message from the second storage controller.
The switch according to claim 24, wherein the second reply message includes proxy indication information, and the proxy indication information instructs the second storage controller to send the second reply message on behalf of the storage controller. The message is sent to the client, and the third reply message does not include the proxy indication information.
The switch according to any one of claims 21-25, characterized in that,

The processing module is further configured to copy the first I/O message when the first I/O message needs to be copied;

The transceiver module is further configured to send the replicated first I/O message to a backup storage controller of the storage controller, and the backup storage controller and the storage controller work in a dual-active mode.
The switch according to any one of claims 21-26, wherein the direct path connection is a transmission control protocol TCP connection, or a remote direct data storage RDMA connection.
A storage device, characterized by comprising: a transceiver module and a processing module;

The transceiver module is configured to send the address information of the storage controller to the switch, and the address information of the storage controller includes one or more of the following: the queue pair port number of the storage controller, the Internet protocol address, transmission control protocol TCP port number, or protocol number of the storage controller;

The processing module is configured to establish a direct path connection with the switch, the direct path connection is used to transmit input and output I/O messages, and the I/O messages are used to send the storage array to the storage controller through the storage controller Write data to or read data from the storage array through the storage control.
The storage device of claim 28, wherein:

The transceiver module is further configured to send routing information of the storage controller to the switch, where the routing information includes one or more of the following information: identification information of the storage controller, storage control Whether the I/O address of the input and output I/O address of the storage controller, the I/O message destined for the storage controller needs to be copied, or the load information of the storage controller, the I/O address includes a logical unit number and a namespace ID and/or logical block address of .
The storage device according to any one of claims 28-29, wherein:

The transceiver module is further configured to receive the first I/O message from the switch through the direct path connection.
The storage device according to claim 30, characterized in that,

The processing module is further configured to generate a first reply message according to the first I/O message, and the first reply message is a response to the first I/O message;

The transceiver module is further configured to send the first reply message to the switch through the direct path connection.
The storage device according to claim 30, characterized in that,

The processing module is further configured to generate a second reply message according to the first I/O message, and the second reply message is a response to the first I/O message;

The transceiver module is further configured to send the second reply message to the switch, the destination of the second reply message is a second storage controller, and the second storage controller is the client A storage controller initially allocated for the first I/O message.
The storage device according to claim 32, wherein the second reply message includes proxy indication information, and the proxy indication information instructs the second storage controller to proxy the storage controller to transfer the second The reply message is sent to the client.
The storage device according to any one of claims 28-33, wherein the direct path connection is a Transmission Control Protocol TCP connection, or a remote direct data storage RDMA connection.
A storage system, characterized by comprising multiple switches according to any one of claims 18-27, and multiple storage devices according to any one of claims 28-34.