CN113326154A

CN113326154A - Connection management method, device, electronic equipment and storage medium

Info

Publication number: CN113326154A
Application number: CN202110720140.6A
Authority: CN
Inventors: 周炜; 雍帅; 霍道安; 刘烨
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-06-28
Filing date: 2021-06-28
Publication date: 2021-08-31
Anticipated expiration: 2041-06-28
Also published as: CN113326154B

Abstract

The embodiment of the application provides a connection management method, a connection management device, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving a connection request of a Remote Direct Memory Access (RDMA) connection of an application, wherein the connection request is used for requesting to establish the RDMA connection with a second device through a first device; acquiring a connection list of RDMA connection according to the connection request; the connection list comprises at least one connection multiplexing parameter; and under the condition that the target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list, establishing the RDMA connection between the first equipment and the second equipment according to the connection structure body corresponding to the target connection multiplexing parameter. Based on the connection management method provided by the application, the resource overhead of thread resources in the process of creating the RDMA connection can be reduced.

Description

Connection management method, device, electronic equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of distributed storage, in particular to a connection management method and device, electronic equipment and a computer storage medium.

Background

With the rapid rise of Cloud Computing (Cloud Computing), key services of users gradually go to the Cloud. Among the three fundamental stones of cloud computing, Distributed storage (Distributed Memory) faces challenges in terms of reliability, availability, and performance. The distributed storage is essentially network storage, and requires that each storage node is constructed into a Cluster (Cluster) in a network interconnection mode, so that more reliable high-performance storage service is provided to the outside.

In high-performance distributed storage, a traditional Transmission Control Protocol (TCP) network is used to perform network interconnection between storage nodes, and there is network overhead in terms of Input/Output (I/O) delay of distributed storage, which is easy to form a network Transmission bottleneck. With the rapid increase of network bandwidth, Remote Direct data Access (RDMA) networks that achieve high performance transmission by bypassing a Central Processing Unit (CPU) have become the choice of distributed storage.

In the related art, after a client initiates a connection request of an RDMA connection, a connection needs to be created after waiting for a response of a server, and for a scenario in which the client and the server are in the same thread, resource overhead for network resources exists in the RDMA connection process. Therefore, how to reduce the resource overhead of thread resources in the process of creating RDMA connection becomes an important problem to be solved urgently.

Disclosure of Invention

The embodiment of the application provides a connection management method, a connection management device, electronic equipment and a computer storage medium, which can reduce resource overhead of thread resources in an RDMA connection establishing process.

The connection management method provided by the embodiment of the application is applied to first equipment and comprises the following steps:

receiving a connection request for a remote direct memory access, RDMA, connection of an application, the connection request requesting establishment of an RDMA connection with the second device by the first device;

acquiring a connection list of the RDMA connection according to the connection request; the connection list comprises at least one connection multiplexing parameter;

and under the condition that the connection list has target connection multiplexing parameters consistent with the connection parameters in the connection request, establishing RDMA connection between the first equipment and the second equipment according to a connection structure body corresponding to the target connection multiplexing parameters.

In one implementation, the establishing an RDMA connection between the first device and the second device according to the connection structure corresponding to the target connection multiplexing parameter includes:

detecting the RDMA event corresponding to the connection structure at fixed time, and acquiring the detection result of the RDMA event;

determining whether to send an RDMA connection request to the second device according to the detection result of the RDMA event;

establishing an RDMA connection of the first device and the second device according to the RDMA connection request and the connection structure if it is determined to send the RDMA connection request to the second device.

In one implementation, the periodically detecting an RDMA event corresponding to the connection structure includes:

acquiring a file descriptor of an RDMA event corresponding to the connection structure in a timing manner;

and determining the RDMA event corresponding to the connection structure according to the file descriptor of the RDMA event.

In one implementation, the determining whether to send the RDMA connection request to the second device according to the detection result of the RDMA event includes:

and determining whether to send the RDMA connection request to the second device according to the detection result of the RDMA event and the event type of the RDMA event.

acquiring detection duration for detecting the RDMA event corresponding to the connection structure;

and determining whether to send the RDMA connection request to the second equipment or not according to the comparison result of the detection duration and the first duration threshold.

In one implementation, prior to receiving a connection request for a remote direct memory access, RDMA, connection of an application, the method further comprises:

and determining the descriptor of the RDMA event channel as a descriptor corresponding to the non-blocking mode.

In one implementation, the method further comprises:

receiving a connection response message sent by the second equipment; the connection response message is used for indicating that the RDMA connection is failed to be established or the RDMA connection is successfully established;

under the condition that the RDMA connection is determined to be established in failure, the detection result of the RDMA event is obtained again at an interval of a second duration threshold;

determining whether to continue sending RDMA connection requests to the second device according to the detection result of the RDMA event which is obtained again until the connection response message indicates that the RDMA connection is established successfully.

In one implementation, the second duration threshold is positively correlated with the number of times the RDMA connection request is sent to the second device.

In one implementation, the second duration threshold ranges in value between a first preset duration and a second preset duration.

An embodiment of the present application provides a connection management apparatus, which is applied to a first device, and includes:

a receiving module for receiving a connection request for a remote direct memory access, RDMA, connection of an application, the connection request for requesting establishment of an RDMA connection with the second device by the first device;

an obtaining module, configured to obtain a connection list of the RDMA connection according to the connection request; the connection list comprises at least one connection multiplexing parameter;

a processing module, configured to establish an RDMA connection between the first device and the second device according to a connection structure corresponding to a target connection multiplexing parameter when the target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list.

In one implementation, the processing module is configured to establish an RDMA connection between the first device and the second device according to a connection structure corresponding to the target connection multiplexing parameter, and includes:

In one implementation, the processing module is configured to detect an RDMA event corresponding to the connection structure at fixed time, and includes:

In one implementation, the processing module, configured to determine whether to send an RDMA connection request to the second device according to a detection result of the RDMA event, includes:

In one implementation, before receiving a connection request for a remote direct memory access, RDMA, connection of an application, the receiving module is further to:

In one implementation, the processing module is further configured to:

An embodiment of the present application provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the connection management method provided in one or more of the foregoing technical solutions is implemented.

The embodiment of the application provides a computer storage medium, wherein a computer program is stored in the computer storage medium; the computer program can implement the connection management method provided by one or more of the above technical solutions after being executed.

Receiving a connection request of Remote Direct Memory Access (RDMA) connection of an application based on the connection management method provided by the application, wherein the connection request is used for requesting to establish the RDMA connection with a second device through a first device; acquiring a connection list of RDMA connection according to the connection request; the connection list comprises at least one connection multiplexing parameter; and under the condition that the target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list, establishing the RDMA connection between the first equipment and the second equipment according to the connection structure body corresponding to the target connection multiplexing parameter. The connection structure body corresponding to the target connection multiplexing parameter does not need to be created when the RDMA connection is established every time, so that the resource overhead of thread resources in the process of creating the RDMA connection can be reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

Fig. 1 is an application scenario diagram of a connection management method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a connection management method according to an embodiment of the present application;

fig. 3 is a schematic flowchart of another connection management method according to an embodiment of the present application;

FIG. 4 is a graph of a variation of a second duration threshold provided by an embodiment of the present application;

fig. 5 is an interaction diagram of a connection management method according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another connection management method according to an embodiment of the present application;

fig. 7 is a schematic diagram of a connection management apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The present application will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the examples provided herein are merely illustrative of the present application and are not intended to limit the present application. In addition, the following examples are provided as partial examples for implementing the present application, not all examples for implementing the present application, and the technical solutions described in the examples of the present application may be implemented in any combination without conflict.

Fig. 1 shows an application scenario diagram of a connection management method according to an embodiment of the present application. Referring to fig. 1, the data transfer mode includes a legacy mode, RDMA mode. Here, the RDMA mode is a computer communication mode, and may implement data reading and writing on a storage device in a remote computer based on a shared memory, a communication coprocessor, and an RDMA controller.

In the traditional mode, network interconnection among all storage nodes is realized based on a transmission control protocol during data transmission, network overhead exists in the aspect of input/output delay of distributed storage, and a network transmission bottleneck is easily formed. In the RDMA mode, when data is transmitted, an RDMA-aware Network Interface Controller (RNIC) is used to directly transmit data with a registered memory region participating in data transmission, so that Network transmission efficiency can be improved, and Network delay in the data transmission process can be reduced.

For a first device and a second device in a distributed system, an RDMA connection between the first device and the second device may be established prior to using the RDMA mode for data transfer. For example, an RDMA connection between a first device and a second device is established over an ethernet, and an Infiniband (IB) technique is adopted, which is applied to service scenarios such as high-performance computing.

In the related art, after initiating a connection request of an RDMA connection to a second device, a first device needs to wait for a response of the second device to create the RDMA connection of the first device and the second device. In the case where the first device and the second device are located on the same thread, there is a resource overhead on thread resources when creating the RDMA connection, resulting in network congestion of the threads of the first device and the second device. In the case of network congestion in a thread, an RDMA connection initiated by a first device is in a delay waiting state for a long time, and the delay waiting may cause the RDMA connection between the first device and a second device to be established and failed.

Fig. 2 shows a flowchart of a connection management method according to an embodiment of the present application. Referring to fig. 2, the connection management method provided in the embodiment of the present application, applied to a first device, may include the following steps:

step A201: receiving a connection request of a remote direct memory access, RDMA, connection of an application, the connection request requesting establishment of an RDMA connection with the second device by the first device.

Here, the application may be an application for data interaction based on a transport protocol of an RDMA service, which is referred to as an RDMA application. The first device and the second device can belong to two different functional units in the same network device and are integrated in the same network device; or, the two different network devices belong to, for example, the first device is a client device, and the second device is a server device.

In practical applications, the first device and the second device may be network cards supporting RDMA services.

In an example, at least one virtual machine running an RDMA application is deployed in a first device and at least one virtual machine running an RDMA application is deployed in a second device. The RDMA application on the virtual machine in the first device and the RDMA application on the virtual machine in the second device can perform data interaction based on a transmission protocol of the RDMA service.

In an example, a transport protocol for an RDMA service may include any of: iWARP protocol, RoCE v1 protocol, and RoCE v2 protocol. Where, RoCE is short for RDMA (RDMA over converted Ethernet) Converged Ethernet fused with Ethernet.

In an example, an RDMA Application may call an Application Programming Interface (API) to initiate a connection request for an RDMA connection to a first device. Accordingly, the first device receives an RDMA application initiated connection request.

In an example, an RDMA application may specify a network address, a network port, a set of connection multiplexing parameters of a second device when calling an application program interface to initiate a connection request for an RDMA connection to a first device. The RDMA application may establish an RDMA connection with a different server device through the first device by entering different connection multiplexing parameters into the application program interface.

In an example, the RDMA application may query, through the application program interface, whether the connection status of the first device and the second device is connected.

Step A202: acquiring a connection list of RDMA connection according to the connection request; the connection list comprises at least one connection multiplexing parameter.

Here, the connection list of the RDMA connection may store at least one connection multiplexing parameter set. The set of connection multiplexing parameters may contain one connection multiplexing parameter, or a plurality of connection multiplexing parameters.

In an example, the number of parameters of the connection multiplexing parameter and the type of the parameters of the connection multiplexing parameter are set in the connection multiplexing parameter set according to an application scenario of the RDMA connection. The application scenario of the RDMA connection may include application of the RDMA technology in a large-scale cluster, and application of the RDMA technology in a data center.

In an example, a first device, upon receiving a connection request for an RDMA connection initiated by an RDMA application, obtains a connection list for the RDMA connection, and determines whether a target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list.

It should be appreciated that resource overhead of RDMA connections on network resources is reduced where multiple sub-modules communicate within a process based on connection multiplexing parameters.

Step A203: and under the condition that the target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list, establishing the RDMA connection between the first equipment and the second equipment according to the connection structure body corresponding to the target connection multiplexing parameter.

Here, a Work Queue Element (WQE) is used to define a connection structure body corresponding to the target connection multiplexing parameter. A WQE contains a pointer to a buffer. The work queue elements are divided into work queue elements in a sending queue and work queue elements in a receiving queue. The WQE in the sending queue contains a pointer pointing to data to be sent and a buffer area of the data to be sent; the pointer in the WQE in the receive queue points to the buffer used to store the data to be received.

In an example, the first device checks whether a target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list upon receiving a create connection request of an application. And the first equipment sends the connection request to the second equipment under the condition that the target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list.

In an example, when the first device determines that the target connection multiplexing parameter that is consistent with the connection parameter in the connection request does not exist in the connection list, the first device creates a connection structure corresponding to the target connection multiplexing parameter, and adds the connection multiplexing parameter corresponding to the connection structure to the connection list of the first device.

In an example, in a case where a connection structure corresponding to the target connection multiplexing parameter is created, the first device sets a connection state of the RDMA connection to an unconnected state, and sets a reference count of the connection structure corresponding to the target connection multiplexing parameter to 1.

Based on the connection management method provided by the application, under the condition of receiving the connection request of the RDMA connection of the application, a connection list of the RDMA connection is obtained; the connection list comprises at least one connection multiplexing parameter set; and under the condition that the target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list, establishing the RDMA connection between the first equipment and the second equipment according to the connection structure body corresponding to the target connection multiplexing parameter. The connection structure body corresponding to the target connection multiplexing parameter does not need to be created when the RDMA connection is established every time, so that the resource overhead of thread resources in the process of creating the RDMA connection can be reduced.

In practical applications, the steps a201 to a203 may be implemented by a Processor, and the Processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor.

In one implementation, the establishing an RDMA connection between the first device and the second device according to the connection structure corresponding to the target connection multiplexing parameter may include, referring to fig. 3:

step A301: and detecting the RDMA event corresponding to the connection structure at fixed time, and acquiring the detection result of the RDMA event.

In an example, a time period is set to be T in advance, and within a first time threshold, an RDMA event corresponding to a connection structure is detected at regular time according to the time period T, and a detection result of the RDMA event is obtained. In other words, within the first time period threshold, the RDMA event corresponding to the connection structure is detected once at intervals of time period T, and the detection result of the RDMA event is obtained.

It should be understood that the time interval of the RDMA event corresponding to the connection structure is controlled based on the preset time period T, and the resource overhead of creating the RDMA connection process for the thread resource can be reduced relative to the RDMA event corresponding to the connection structure detected in real time.

In an example, within a first time threshold, a file descriptor of an RDMA event channel RDMA _ cm _ event _ channel is detected once at an interval time period T, and an RDMA event corresponding to a connection structure is determined according to the file descriptor, so that a detection result of the RDMA event is obtained.

Step A302: determining whether to send the RDMA connection request to the second device according to the detection result of the RDMA event.

In an example, an RDMA connection request is sent to the second device in a case that the detection result of the RDMA event indicates that the network address of the second device is normal and the network route between the first device and the second device is normal.

For example, in a case where the detection result of the RDMA EVENT is that the RDMA _ CM _ EVENT _ ADDR _ RESOLVED EVENT is detected, it is determined that the network address of the second device is normal.

And determining that the network ROUTE between the first device and the second device is normal when the detection result of the RDMA EVENT is that the RDMA _ CM _ EVENT _ ROUTE _ RESOLVED EVENT is detected.

In an example, where the detection of the RDMA event indicates that the network address of the second device is abnormal and the network routing between the first device and the second device is abnormal, the RDMA connection request is not sent to the second device.

Step A303: and under the condition that the RDMA connection request is determined to be sent to the second device, establishing the RDMA connection between the first device and the second device according to the RDMA connection request and the connection structure body.

In an example, in an instance in which it is determined to send an RDMA connection request to the second device, an RDMA library function RDMA _ connect () is called to initiate the RDMA connection request to the second device. Here, the identification information of the RDMA connection is RDMA _ cm _ id.

In an example, in the event that it is determined to send an RDMA connection request to the second device, an RDMA library function RDMA _ create _ id () is called to create an RDMA connection according to the RDMA event channel RDMA _ cm _ event _ channel, setting a connection request parameter for the RDMA connection. Sending the RDMA connection request to the second device according to the connection request parameters of the RDMA connection.

In an example, a first device sends an RDMA connection request to a second device, calling an RDMA library function RDMA _ create _ qp () to create a connection struct upon detecting an RDMA _ CM _ EVENT _ ESTABLISHED EVENT. An RDMA connection is established for the first device and the second device according to the connection structure.

In one implementation, after a first device sends a connection request to a second device, the second device detects an RDMA connection request event on an event channel of the RDMA connection. And under the condition of detecting the RDMA connection request event, analyzing the connection request parameters of the first equipment, creating RDMA resources corresponding to the connection request parameters, adding the RDMA connection into a connection list of the second equipment, and returning a connection response to the first equipment.

In an example, the second device detects the RDMA connection request event on a File Descriptor (FD) of an RDMA event channel RDMA _ event _ channel. Here, the RDMA connection REQUEST EVENT may be referred to as an RDMA _ CM _ EVENT _ CONNECT _ REQUEST EVENT.

In an example, the second device parses the connection request parameters of the first device, creates an RDMA resource corresponding to the connection request parameters, adds the RDMA connection to a connection list of the second device, and calls an RDMA library function RDMA _ accept () to return a connection reply to the first device, when detecting an RDMA connection request event.

Here, the connection request parameter includes a queue length of a Remote Procedure Call (RPC) request queue of the first device.

In an example, the second device creating an RDMA resource corresponding to the connection request parameter may include the following steps: the second device calls RDMA library function RDMA _ create _ qp () to create an RDMA queue; the second device creates an RPC request queue for the RDMA connection.

In an example, in the case where the second device creates an RPC request queue of the RDMA connection, the minimum value of the queue lengths in the RPC request queue of the first device and the RPC request queue of the second device is taken as the queue length of the RPC request queue.

In an example, the first device detects an RDMA _ CM _ EVENT _ ESTABLISHED EVENT on a file descriptor of an RDMA EVENT channel RDMA _ EVENT _ channel, creates an RPC request queue for the RDMA connection, and updates a connection state of the RDMA connection to a connected state. Thus, an RDMA connection of the first device and the second device is established.

acquiring a file descriptor of an RDMA event corresponding to the connection structure in a timing manner; and determining the RDMA event corresponding to the connection structure according to the file descriptor of the RDMA event.

In an example, an RDMA library function RDMA _ bind _ addr () is called, RDMA _ cm _ id is bound to a listening port set by an application, an RDMA library function RDMA _ list () is called, and a file descriptor of an RDMA event channel RDMA _ event _ channel is listened.

In an example, within a first time threshold, detecting a file descriptor of an RDMA event channel RDMA _ event _ channel once every time period T, confirming whether event generation exists on the file descriptor of the RDMA event channel RDMA _ event _ channel, and determining an RDMA event corresponding to a connection structure according to the file descriptor when the event generation exists on the file descriptor.

It should be appreciated that based on detecting RDMA events corresponding to a connection structure at a fixed time, resource overhead for creating RDMA connection processes for thread resources may be reduced relative to detecting RDMA events corresponding to a connection structure in real time.

In an example, a network address of a second device which is requested to be connected by an application is analyzed, an RDMA library function RDMA _ resolve _ addr () is called, a file descriptor of an RDMA event channel RDMA _ cm _ event _ channel is detected once every time period T within a first time threshold, and a detection result of the RDMA event is obtained.

In an example, the event type of the RDMA event includes any one of: the RDMA event corresponding to the network address of the first device, and the RDMA event corresponding to the network route between the first device and the second device.

The RDMA EVENT corresponding to the network address of the first device is referred to as RDMA _ CM _ EVENT _ ADDR _ RESOLVED EVENT, and the RDMA EVENT corresponding to the network ROUTE between the first device and the second device is referred to as RDMA _ CM _ EVENT _ ROUTE _ RESOLVED EVENT.

It should be understood that, according to the detection result of the RDMA event and the event type of the RDMA event, it may be identified whether the network address of the first device is normal, whether the network route between the first device and the second device is normal, and in the case where the network address of the first device is normal and the network route between the first device and the second device is normal, the RDMA connection request may be transmitted to the second device. Thus, the resource overhead of thread resources for the process of creating an RDMA connection is reduced.

In an example, where an RDMA _ CM _ EVENT _ ADDR _ RESOLVED EVENT is detected, the RDMA _ CM _ EVENT _ ROUTE _ RESOLVED EVENT continues to be detected. In the EVENT that no RDMA _ CM _ EVENT _ ADDR _ RESOLVED EVENT is detected, the resources of the RDMA connection are released.

In an example, the method includes parsing routing information between a first device and a second device, calling an RDMA library function RDMA _ resolve _ route (), detecting a file descriptor of RDMA _ cm _ event _ channel once every time period T within a first time threshold, and obtaining a detection result of the RDMA event.

Here, the routing information of the network address of the first device to the second device specified by the application may be routing information of the network address of the first device to the second device specified by the application.

In an example, upon detecting an RDMA _ CM _ EVENT _ ROUTE _ RESOLVED EVENT, invoking an RDMA library function RDMA _ create _ qp () to create an RDMA queue, setting a connection request parameter for an RDMA connection, invoking an RDMA library function RDMA _ connect (), initiating a connection request to the second device.

In an example, the resources of the RDMA connection are released when no RDMA CM EVENT address response EVENT is detected.

acquiring detection duration for detecting the RDMA event corresponding to the connection structure; and determining whether to send the RDMA connection request to the second equipment or not according to the comparison result of the detection duration and the first duration threshold.

In an example, a detection duration of an RDMA event corresponding to a detection connection structure is obtained; and under the condition that the detection time length is larger than the first time length threshold value, stopping detecting the RDMA event corresponding to the connection structure body, and not sending the RDMA connection request to the second equipment.

In an example, a detection duration of an RDMA event corresponding to a detection connection structure is obtained; and stopping detecting the RDMA event corresponding to the connection structure under the condition that the detection time length is less than the first time length threshold value. And sending the RDMA connection request to the second device under the condition that the detection result of the RDMA event indicates that the network address of the second device is normal and the network route between the first device and the second device is normal.

It should be understood that according to the comparison result of the detection duration and the first duration threshold, the detection duration of the RDMA event corresponding to the detection connection structure may be controlled so that the detection duration is smaller than the first duration threshold, thereby reducing the resource overhead of the thread resource in the process of creating the RDMA connection.

In one implementation, prior to receiving a connection request for a remote direct memory access, RDMA, connection of an application, the method further comprises: and determining the descriptor of the RDMA event channel as a descriptor corresponding to the non-blocking mode.

In an example, the first device initializes RDMA resources, including RDMA event channels RDMA _ cm _ event _ channel and RDMA completion queues ibv _ cq.

In an example, the file descriptor of the RDMA event channel RDMA _ event _ channel is set to non-blocking mode, and the file descriptor of RDMA _ event _ channel is added to polfd.

Here, the file descriptor of the RDMA event channel RDMA _ event _ channel may be used to track RDMA events defined by the RDMA programming library.

In the embodiment of the application, the RDMA connection is created in a non-blocking manner, and when the application calls the application program interface to create the RDMA connection, the connection pointer returned by the first device is obtained, and it is not necessary to wait for the connection from the first device to the second device to be completed. Therefore, the method is suitable for a high-performance polling programming mode and a scene of connecting a second device with the same thread.

In an example, in a high Performance round robin programming mode, a Storage Performance Development Kit (SPDK) poler may be employed to share RDMA connections to save network resources for multiple sub-modules present inside a program to communicate with the same node.

Accordingly, within the threads of the first device and the second device, multiple RDMA connections may share an RDMA event channel RDMA _ cm _ event _ channel.

In one implementation, the establishing an RDMA connection between the first device and the second device according to the sending of the RDMA connection request to the second device includes:

receiving a connection response message sent by the second equipment; the connection response message is used for indicating that the RDMA connection is failed to be established or the RDMA connection is successfully established; under the condition that the RDMA connection is determined to be established in failure, the detection result of the RDMA event is obtained again at an interval of a second duration threshold; determining whether to continue sending RDMA connection requests to the second device according to the detection result of the RDMA event which is obtained again until the connection response message indicates that the RDMA connection is established successfully.

In an example, in the event of an RDMA connection failure, resources of the RDMA connection are released. Establishing a timer of RDMA connection, and after the timer passes a preset time, reacquiring a detection result of the RDMA event; determining whether to continue sending RDMA connection requests to the second device until the connection response message indicates that the RDMA connection establishment is successful, based on the detection result of the reacquiring RDMA event.

It should be understood that the first device retrieves the detection result of the RDMA event at a second duration threshold after the RDMA connection fails; and determining whether to continue sending the RDMA connection request to the second equipment according to the detection result of the RDMA event, wherein compared with the RDMA connection request sent to the second equipment in real time, the resource overhead of thread resources in the process of creating the RDMA connection can be reduced.

In an example, referring to fig. 4, the second duration threshold is lengthened as the number of reconnects increases, and the size of the second duration threshold is dynamically set, and the second duration threshold is positively correlated to the number of times the RDMA connection request is sent to the second device. Here, the time length corresponding to the second duration threshold is a reconnected time interval.

It should be understood that, since the second duration threshold is positively correlated with the number of times of sending the RDMA connection request to the second device, the larger the number of times of sending the RDMA connection request to the second device by the first device is, the larger the time length corresponding to the second duration threshold is. Thus, the resource overhead for thread resources for creating an RDMA connection process may be reduced.

In an example, according to an application scenario of the RDMA connection, a threshold range of the reconnection number is set, for example, the minimum value of the reconnection number is 10, and the maximum value of the reconnection number is 100.

In an example, the second duration threshold has a value range of [ Min, Max ], where Min corresponds to a first preset duration and Max corresponds to a second preset duration.

In an example, the numerical range of the second duration threshold is set according to an application scenario of the RDMA connection. For example, the second duration threshold has a value range of [50ms, 5s ], wherein the first preset duration is 50ms and the second preset duration is 5 s.

It should be understood that, based on setting the numerical range of the second duration threshold, the duration of the second duration threshold can be flexibly controlled, and the resource overhead of the thread resource in the process of creating the RDMA connection is reduced.

In one implementation, in the connection management method, the method may further include:

and releasing the resources of the RDMA connection according to the operation state information of the RDMA connection.

In an example, the first device determines run state information for the RDMA connection based on a reference count for the RDMA connection. The first device decrements the reference count of the RDMA connection by 1 upon receiving an application-initiated request to delete the RDMA connection.

In an example, where the reference count of the RDMA connection is greater than 0, the run state information of the RDMA connection is determined to be that the RDMA application is using the RDMA connection. In the case where the reference count of the RDMA connection is equal to 0, determining the run state information of the RDMA connection as there is no RDMA application using the RDMA connection.

In an example, where the reference count of the RDMA connection is equal to 0, the first device calls an RDMA library function RDMA _ disconnect (), disconnects the RDMA connection of the first device with the second device, and releases resources of the RDMA connection.

Further, the RDMA connection is removed from the connection list of the first device, releasing the connection structure of the RDMA connection.

In an example, after a first device disconnects a first device and a second device, the second device detects an interruption event of the RDMA connection on a file descriptor of the RDMA _ cm _ event _ channel.

In an example, the second device releases RDMA connection-related RDMA resources, deletes the RDMA connection from the connection list of the second device, and releases the connection structure of the RDMA connection, in case an interruption event of the RDMA connection is detected.

Here, the RDMA connection interrupt EVENT may be defined as an RDMA _ CM _ EVENT _ DISCONNECTED EVENT.

In one implementation, in the connection management method, before receiving a connection request of a remote direct memory access RDMA connection of an application, the first device may further include the following steps:

initializing resources of the RDMA connection.

In an example, initializing RDMA resources includes the steps of:

step 1.1: create RDMA event channel RDMA _ event _ channel, call RDMA library function RDMA _ create _ event _ channel () create an event channel for RDMA connection.

It should be understood that the RDMA _ create _ event _ channel () library function may be used to create an event channel (event channel), which is a channel used by an RDMA device to notify an application after an operation is completed or in the event of an event such as a connection request.

In the example, there is a file descriptor inside the event channel, and the file descriptor inside the event channel is monitored using the poll function.

Step 1.2: call RDMA library function ibv _ create _ cq (), create completion queue ibv _ cq for RDMA connection; and calling an RDMA library function RDMA _ get _ devices () according to the device handle ibv _ context corresponding to the RDMA connection to acquire the resource of the RDMA connection.

Step 1.3: an RDMA connected protection domain ibv _ pd is created from the device handle ibv _ context, an RDMA library function ibv _ alloc _ pd () is called, and a memory protection unit (protection domain) is created.

It should be understood that the memory protection unit is used for directly establishing an association relationship between the memory region and the queue to prevent unauthorized access.

Step 1.4: and calling an RDMA library function RDMA _ create _ id () to create identification information RDMA _ cm _ id of the RDMA connection.

In an example, the second device initializes the RDMA resource and binds the port to listen, the first device initializing the RDMA resource waiting for the application to initiate the connection. The second device and the first device respectively set the file descriptor of the RDMA event channel to be in a non-blocking mode; and under the condition that the application calls the application program interface for creating the RDMA connection, the first device creates a connection structure body to return the application and starts to asynchronously connect the second device.

Correspondingly, after receiving the connection request of the first equipment, the second equipment creates a connection structure body of the second equipment and returns a response of the first equipment; and after receiving the response, the first device completes the RDMA connection between the first device and the second device and updates the connection state of the first device and the second device.

Based on the same technical concept as the foregoing embodiment, referring to fig. 5, the connection management method provided in the embodiment of the present application may include:

step A501: the first device initializes RDMA resources.

Step A502: the first device sets a file descriptor of the RDMA connection channel to a non-blocking mode;

step A503: the first device creates a connection structure for the RDMA connection.

Step A504: the first device creates a resource for the RDMA connection.

Step A505: the first device sends a connection request to the second device.

Step A506: the second device creates a connection structure for the RDMA connection and a resource for the RDMA connection.

In an example, the second device may initialize the RDMA resources, setting a file descriptor FD of the RDMA connection channel to a non-blocking mode, before creating the connection structure of the RDMA connection and the resources of the RDMA connection.

Step A507: the second device returns a connection reply to the first device.

Step A508: the first device updates a connection state of the RDMA connection.

Based on the same technical concept as the foregoing embodiment, referring to fig. 6, the connection management method provided in the embodiment of the present application may include:

step A601: the first device receives a connection request for an RDMA connection of an application.

Step A602: the first device creates a connection structure for the RDMA connection.

Step A603: the first device creates identification information for the RDMA connection.

Step A604: the network address of the second device to which the application requests a connection is resolved.

Step A605: whether an RDMA event corresponding to the network address is detected.

In an example, the RDMA EVENT corresponding to the network address may be the RDMA _ CM _ EVENT _ ADDR _ RESOLVED EVENT described above.

In an example, the following step a606 is performed in case an RDMA _ CM _ EVENT _ ADDR _ RESOLVED EVENT is detected, otherwise the following step a611 is performed.

Step A606: and analyzing the routing information from the first equipment to the second equipment.

Step A607: whether an RDMA event corresponding to the routing information is detected.

In an example, the RDMA EVENT corresponding to the routing information may be the RDMA _ CM _ EVENT _ ROUTE _ RESOLVED EVENT described above.

In an example, the following step a608 is performed in case an RDMA _ CM _ EVENT _ ROUTE _ RESOLVED EVENT is detected, otherwise the following step a611 is performed.

Step A608: the first device sends an RDMA connection request to the second device.

Step A609: whether RDMA connection creation completion corresponding RDMA events are detected.

In an example, the RDMA EVENT corresponding to the RDMA connection creation completion may be the RDMA _ CM _ EVENT _ ESTABLISHED EVENT noted above.

In an example, the following step a610 is performed in case an RDMA _ CM _ EVENT _ ESTABLISHED EVENT is detected, otherwise the following step a611 is performed.

Step A610: connection state information for the RDMA connection is updated.

Step A611: network resources for the RDMA connection are released.

Based on the same technical concept as the foregoing embodiment, referring to fig. 7, the connection management apparatus provided in the embodiment of the present application may include:

a receiving module 701, configured to receive a connection request of a remote direct memory access, RDMA, connection of an application, where the connection request is used to request establishment of an RDMA connection with the second device through the first device;

an obtaining module 702, configured to obtain a connection list of the RDMA connection according to the connection request; the connection list comprises at least one connection multiplexing parameter;

a processing module 703, configured to, when a target connection multiplexing parameter consistent with the connection parameter in the connection request exists in the connection list, establish an RDMA connection between the first device and the second device according to a connection structure corresponding to the target connection multiplexing parameter.

In one implementation, the processing module 703 is configured to establish an RDMA connection between the first device and the second device, and includes:

establishing an RDMA connection between the first device and the second device according to the RDMA connection request, if it is determined to send the RDMA connection request to the second device.

In one implementation, the processing module 703 is configured to detect an RDMA event corresponding to the connection structure at fixed time, and includes:

In one implementation, the processing module 703 is configured to determine whether to send an RDMA connection request to the second device according to the detection result of the RDMA event, and includes:

In one implementation, before receiving a connection request of a remote direct memory access RDMA connection of an application, the receiving module 701 is further configured to:

In one implementation, the processing module 703 is configured to establish an RDMA connection between the first device and the second device according to the sending of the RDMA connection request to the second device, and includes:

In practical applications, the receiving module 701, the obtaining module 702, and the processing module 703 may all be implemented by a processor of an electronic device, where the processor may be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor, and the embodiment of the present application is not limited thereto.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present application may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

Based on the same technical concept as the foregoing embodiment, referring to fig. 8, an electronic device 800 provided in an embodiment of the present application may include: a memory 810 and a processor 820; wherein the content of the first and second substances,

a memory 810 for storing computer programs and data;

a processor 820 for executing a computer program stored in the memory to implement any one of the connection management methods in the foregoing embodiments.

In practical applications, the memory 810 may be a volatile memory (volatile memory), such as RAM; or a non-volatile memory (non-volatile memory), illustratively a ROM, a flash memory, a Hard Disk Drive (HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories. The memory 810 may provide instructions and data to the processor 820.

The foregoing descriptions of the various embodiments are intended to highlight different aspects of the various embodiments, which have the same or similar aspects, and thus, for brevity, detailed descriptions thereof are omitted

The methods disclosed in the method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

Features disclosed in various product embodiments provided by the application can be combined arbitrarily to obtain new product embodiments without conflict.

The features disclosed in the various method or apparatus embodiments provided herein may be combined in any combination to arrive at new method or apparatus embodiments without conflict.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, and for example, the division of the unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication between the components shown or discussed may be through some interfaces, and the indirect coupling or communication between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of grid units; some or all of the units can be selected according to actual conditions to achieve the purpose of the scheme of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing module, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A connection management method applied to a first device, the method comprising:

2. The method of claim 1, wherein the establishing the RDMA connection of the first device and the second device according to the connection structure corresponding to the target connection multiplexing parameter comprises:

3. The method of claim 2, wherein the timing detection of the RDMA event corresponding to the connection structure comprises:

4. The method of claim 2, wherein the determining whether to send the RDMA connection request to the second device based on the detection of the RDMA event comprises:

5. The method of claim 2, wherein the determining whether to send the RDMA connection request to the second device based on the detection of the RDMA event comprises:

6. The method of claim 1, wherein prior to receiving the connection request for the remote direct memory access, RDMA, connection of the application, the method further comprises:

7. The method of claim 1, further comprising:

determining whether to continue sending RDMA connection requests to the second device according to the re-acquired detection result of the RDMA event until the connection response message indicates that the RDMA connection establishment is successful.

8. The method of claim 7, wherein the second duration threshold positively correlates with a number of times the RDMA connection request is sent to the second device.

9. The method according to claim 7 or 8, wherein the second duration threshold has a value ranging between a first preset duration and a second preset duration.

10. A connection management apparatus, applied to a first device, includes:

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the connection management method of any one of claims 1 to 9 when executing the program.

12. A computer storage medium storing a computer program; characterized in that said computer program is capable of implementing the connection management method of any one of claims 1 to 9 when executed.