CN110647293B

CN110647293B - Method for sending target instruction and storage system

Info

Publication number: CN110647293B
Application number: CN201910775367.3A
Authority: CN
Inventors: 卢玥; 赵聪; 周坤; 余思; 龚骏辉
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-08-21
Filing date: 2019-08-21
Publication date: 2021-07-09
Anticipated expiration: 2039-08-21
Also published as: CN110647293A

Abstract

The application provides a method and a storage system for sending a target instruction, wherein the method comprises the following steps: acquiring the forwarding time delay of a first control device for forwarding a target instruction to other control devices of a production center and the minimum round-trip time delay of a link from each control device of the production center to a second control device; determining a third control device, wherein the third control device is a control device corresponding to a path with the minimum round-trip delay from the production center to the second control device; determining a target link from the N links from the third control device to the second control device; the target instruction is sent to the second control device through the target link. The technical scheme can improve the overall performance of the storage system.

Description

Method for sending target instruction and storage system

Technical Field

The present application relates to the field of information technology, and more particularly, to a method and a storage system for sending a target instruction.

Background

In order to ensure data security against various natural or man-made disasters, a storage system generally includes a production center and a disaster preparation center.

A production center may also be referred to as production storage, production storage equipment, and the like. The production center is responsible for the local storage of data. The production center is also responsible for copying the data needing backup to the work of the disaster recovery center.

The disaster backup center can also be called disaster backup storage, disaster backup storage equipment, and the like. And the disaster recovery center receives the write request sent by the production center, and stores data corresponding to the write request in storage equipment of the disaster recovery center. When needed, the disaster recovery center can send the stored data to the production center so as to recover the data stored by the production center.

In order to improve the reliability of communication between the production center and the disaster recovery center, a multilink redundancy technology is adopted for communication between the production center and the disaster recovery center. Multiple links are included between the production center and the disaster recovery center. If one or part of the links in the plurality of links are failed, the rest links can be used between the production center and the disaster recovery center to complete the data copying or recovery work.

In addition, because a plurality of links exist between the production center and the disaster recovery center, the production center can also select one link from the plurality of links to send the write request to the disaster recovery center. The currently common selection mechanism for links is a load-based link selection mechanism. The production center selects the least loaded link from the plurality of links as the link to send the write request. In a storage system using such a link selection mechanism, if one link fails, the throughput of the system as a whole is reduced.

Disclosure of Invention

The application provides a method for sending a target instruction and a storage system, which can improve the overall performance of the storage system.

In a first aspect, an embodiment of the present application provides a method for sending a target instruction, where the method includes: acquiring the forwarding time delay of a first control device for forwarding a target instruction to other control devices of a production center, wherein the production center comprises a plurality of control devices, and the first control device is the control device which receives the target instruction in the production center; acquiring the minimum round-trip delay of a link from each control device of the production center to a second control device, wherein the second control device is one control device of the disaster recovery center; determining a third control device, wherein the third control device is a control device corresponding to a path with the minimum round-trip delay from the production center to the second control device; determining a target link from the N links from the third control device to the second control device, where N is a positive integer greater than or equal to 1; the target instruction is sent to the second control device through the target link.

In the above technical solution, the production center of the storage system performs global selection, selects the control device with the link having the smallest round-trip delay from the plurality of control devices included in the production center, and then selects an appropriate link from the control device as the target link for sending the target instruction, so that the throughput from the production center to the disaster recovery center can be improved. For example, if the round-trip delay of a link between the production center and the disaster recovery center is significantly increased, the above-mentioned solution does not significantly delay the throughput from the production center to the disaster recovery center, and if the existing load-based target instruction distribution scheme is adopted, the throughput from the production center to the disaster recovery center is reduced.

In one possible design, the determining a target link from the N links from the third device to the second control device includes: determining a first link set, wherein the first link set comprises N links of the N links₁A link of N₁The link comprises a first link which is the link with the minimum round trip delay among the N links, wherein N is₁Is a positive integer greater than or equal to 1 and less than or equal to N, N being a positive integer greater than or equal to 1; a target link is determined from the first set of links.

In one possible design, the N₁The link also comprises at least one second link, and the difference between the round trip delay of any one second link in the at least one second link and the round trip delay of the first link is smaller than a preset threshold value.

In the above technical solution, the target link is a link with a smaller round-trip delay from N links. Therefore, the target instruction can be quickly sent to the disaster recovery center, and feedback information corresponding to the target instruction can also be quickly returned to the production center, so that the overall throughput efficiency and the working efficiency of the storage system are improved.

Optionally, the target link may be the least loaded link in the first set of links.

In one possible design, prior to determining the first set of links, the method further includes: determining a second set of links, wherein the second linkIncluding N of the N links in the set₂A link of N₂The link is the link that can be used to transmit the target instruction in the N links, where N is₂Is greater than or equal to N₁And is less than or equal to a positive integer of N; the determining the first set of links includes: the first set of links is determined based on the second set of links.

Based on the above technical solution, all links in the first link set are links that can be used for sending the target instruction. This avoids the selection of a destination link that does not have sufficient transmission resources to transmit the destination instruction.

In one possible design, the determining the second set of links includes: determining the N links according to the target instruction and the remaining transmission resources of each of the N links₂A bar link, wherein the N₂The size of the remaining transmission resources of any one of the links is not less than the size of the target instruction.

In one possible design, the method further includes: and allocating transmission resources for the nth link according to the load capacity information of the nth link in the N links, wherein N is 1, … and N.

In one possible design, the load capability information of the nth link includes at least one of the following information: a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay; the allocating transmission resources for the nth link according to the load capability information of the nth link in the N links includes: determining a target link capacity evaluation value of the nth link according to the load capacity information of the nth link; and allocating transmission resources for the nth link according to the target link capability evaluation value.

In one possible design, the reference throughput is the total throughput in the previous statistical period, or an average of M total throughputs in the previous M statistical periods, where M is a positive integer greater than or equal to 2; the reference link capacity evaluation value is a link capacity evaluation value in a previous statistical period, or an average value of M link capacity evaluation values in the previous M statistical periods; the first reference round trip delay is an average round trip delay in a previous statistical period, or an average value of M average round trip delays in previous M statistical periods; the second reference round trip delay is an average round trip delay when the nth link does not exceed the maximum load capacity.

Based on the technical scheme, the control device in the production center allocates appropriate transmission resources for the link by evaluating the link responsibility in real time, actively protects the link, can reduce the probability that the link flow exceeds the link bandwidth, and improves the reliability of the storage system.

In one possible design, the determining the first set of links includes: determining the first link from the second set of links; according to the time delay of the first link and the preset threshold value, from N₂The at least one second link is determined among the links.

In a second aspect, an embodiment of the present application provides a storage system, where the storage system includes a production center and a disaster recovery center, the production center includes a first control device and a third control device, and the disaster recovery center includes a second control device: the first control device is used for receiving a target instruction; the first control device is also used for obtaining the forwarding time delay of the first control device for forwarding the target instruction to other control devices of the production center; the first control device is further used for acquiring the minimum round-trip delay of a link from each control device of the production center to the second control device; the first control device is also used for determining a third control device, wherein the third control device is a control device corresponding to a path with the minimum round-trip delay from the production center to the second control device; the third control device is used for determining a target link from N links from the third control device to the second control device, wherein N is a positive integer greater than or equal to 1; the third control device is further configured to send the target command to the second control device via the target link.

In one possible design, the third controlling means is specifically configured to determine a first link set, where the first link set includes N links of the N links₁A link of N₁The link comprises a first link which is the link with the minimum round trip delay among the N links, wherein N is₁Is a positive integer greater than or equal to 1 and less than or equal to N, N being a positive integer greater than or equal to 1; a target link is determined from the first set of links.

In one possible design, the third controlling means is further configured to determine a second set of links, where the second set of links includes N of the N links₂A link of N₂The link is the link that can be used to transmit the target instruction in the N links, where N is₂Is greater than or equal to N₁And is less than or equal to a positive integer of N; the third control deviceAnd is specifically configured to determine the first link set according to the second link set.

In one possible design, the third controlling means is specifically configured to determine the N links according to the target instruction and the remaining transmission resources of each of the N links₂A bar link, wherein the N₂The size of the remaining transmission resources of any one of the links is not less than the size of the target instruction.

In one possible design, the third controlling means is further configured to allocate transmission resources to an nth link of the N links according to load capability information of the nth link, where N is 1, …, N.

In one possible design, the load capability information of the nth link includes at least one of the following information: a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay; the third control device is specifically configured to determine a target link capability evaluation value of the nth link according to the load capability information of the nth link; and allocating transmission resources for the nth link according to the target link capability evaluation value.

In one possible design, the third control means is specifically configured to determine the first link from the second set of links; according to the time delay of the first link and the preset threshold value, from N₂The at least one second link is determined among the links.

In a third aspect, a control device in an embodiment of the present application includes a transceiver and a processor. Optionally, the control device further comprises a memory. The processor is used for controlling the transceiver to transmit and receive signals, the memory is used for storing a computer program, and the processor is used for calling the computer program from the memory and running the computer program, so that the control device executes the method in any one of the possible implementation manners in the method design of the first aspect.

In a fourth aspect, the present embodiment provides a control device, which may be the first control device and/or the third control device in the method design of the first aspect, or a chip disposed in the first control device and/or the third control device. The control device includes: a processor, coupled to the memory, is configured to execute the instructions in the memory to implement the method performed by the first control device and/or the third control device in any one of the possible implementations of the method design of the first aspect.

Optionally, the control device further comprises a memory. Optionally, the control device further comprises a communication interface, the processor being coupled to the communication interface.

When the control device is a chip, the communication interface may be an input/output interface.

Alternatively, the transceiver may be a transmit-receive circuit. Alternatively, the input/output interface may be an input/output circuit.

In a fifth aspect, an embodiment of the present application provides a computer program product, where the computer program product includes: computer program code which, when run on a computer, causes the computer to perform the method of the first aspect described above.

In a sixth aspect, the present application provides a computer-readable medium, which stores program code, and when the computer program code runs on a computer, the computer executes the method in the first aspect.

Drawings

FIG. 1 is a schematic diagram of a storage system to which embodiments of the present application can be applied.

Fig. 2 is a schematic diagram of another storage system to which the present solution can be applied.

FIG. 3 is a schematic flow chart diagram of a method for sending a target instruction according to an embodiment of the present application.

FIG. 4 is a graph of throughput for the memory system after injection of a link delay using the method shown in FIG. 3 and a prior art method.

Fig. 5 is a schematic flow chart of a method for allocating transmission resources provided according to an embodiment of the present application.

Fig. 6 is a schematic structural block diagram of a control device provided according to an embodiment of the present application.

Fig. 7 is a schematic structural diagram of a storage system provided according to an embodiment of the present application.

Detailed Description

The technical solution in the present application will be described below with reference to the accompanying drawings.

This application is intended to present various aspects, embodiments or features around a system that may include a number of devices, components, modules, and the like. It is to be understood and appreciated that the various systems may include additional devices, components, modules, etc. and/or may not include all of the devices, components, modules etc. discussed in connection with the figures. Furthermore, a combination of these schemes may also be used.

In addition, in the embodiments of the present application, words such as "exemplary", "for example", etc. are used to mean serving as examples, illustrations or explanations. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term using examples is intended to present concepts in a concrete fashion.

In the embodiment of the present application, information (information), signal (signal), message (message), channel (channel) may be mixed, and it should be noted that the intended meanings are consistent when the differences are not emphasized. "corresponding" and "corresponding" may sometimes be used in combination, it being noted that the intended meaning is consistent when no distinction is made.

In the examples of the present application, the subscripts are sometimes as W₁It may be mistaken for a non-subscripted form such as W1, whose intended meaning is consistent when the distinction is de-emphasized.

The network architecture and the service scenario described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not form a limitation on the technical solution provided in the embodiment of the present application, and as a person of ordinary skill in the art knows that along with the evolution of the network architecture and the appearance of a new service scenario, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

In the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.

FIG. 1 is a schematic diagram of a storage system to which embodiments of the present application can be applied. The storage system 100 shown in fig. 1 includes a production center 101 and a disaster recovery center 102.

A plurality of control devices, such as control device 111 and control device 112, may be included in the production center 101. A plurality of control devices, such as control device 121 and control device 122, may also be included in disaster recovery center 102.

In the storage system 100 shown in fig. 1, the production center 101 and the disaster recovery center 102 are connected in a fully connected manner. In other words, each control device in the production center 101 and each control device in the disaster recovery center 102 are connected by at least one link.

For example, there are two links between control device 111 and control device 121, link 1a-1 and link 1a-2, respectively. There is a link between control device 112 and control device 121, which is link 1 b. There is a link between control device 111 and control device 122, which is link 1 c. Control device 112 and control device 122 have links, link 1d-1 and link 1d-2, respectively.

Further, two control apparatuses in the production center 101, i.e., the control apparatus 111 and the control apparatus 112, are also connected to each other by a link 1 e. Two control devices in disaster recovery center 102, i.e., control device 121 and control device 122, are also connected via link 1 f.

It is understood that production center 101 may also include more control devices and disaster recovery center 102 may also include more control devices.

In the storage system shown in fig. 1, the production center 101 and the disaster recovery center 102 are connected in a fully connected manner. In other implementations of the storage system, although there are multiple links between the production center and the disaster recovery center, the production center and the disaster recovery center may not be connected in a full connection manner. Such as the storage system 200 shown in fig. 2.

Fig. 2 is a schematic diagram of another storage system to which the present solution can be applied. As shown in fig. 2, storage system 200 includes a production center 201 and a disaster recovery center 202.

A plurality of control devices, such as control device 211, control device 212, and control device 213, may be included in production center 201. Disaster recovery center 202 may also include a plurality of control devices, such as control device 221 and control device 222.

The storage system 200 shown in fig. 2 is connected in a non-fully connected manner. For example, the control device 212 and the control device 221 do not have a link directly, and the control device 213 and the control device 221 do not have a link.

There is a link between control device 211 and control device 221, which is link 2 a. One link between control device 211 and control device 222 is link 2 c. There are four links between control device 212 and control device 222, link 2b-1, link 2b-2, link 2b-3, and link 2b-4, respectively. Two links, link 2d-1 and link 2d-2, are between the control device 213 and the control device 222.

The control device 211 and the control device 212 are connected by a link 2e, the control device 211 and the control device 213 are connected by a link 2f, and the control device 212 and the control device 213 are connected by a link 2 g. The control device 221 and the control device 122 are connected by a link 2 h.

The application provides a method for sending a target instruction, which comprises the following steps: acquiring the forwarding time delay of a first control device for forwarding a target instruction to other control devices of a production center, wherein the production center comprises a plurality of control devices, and the first control device is the control device which receives the target instruction in the production center; acquiring the minimum round-trip delay of a link from each control device of the production center to a second control device, wherein the second control device is one control device of the disaster recovery center; determining a third control device, wherein the third control device is a control device corresponding to a path with the minimum round-trip delay from the production center to the second control device; determining a target link from the N links from the third control device to the second control device, where N is a positive integer greater than or equal to 1; the target instruction is sent to the second control device through the target link.

In other words, the first control device (i.e., the control device in the production center that receives the target instruction) needs to determine the third control device first. The third control means determines a target link for transmitting the target instruction, and transmits the target instruction to the second control means through the determined target link.

Next, a method of sending a target instruction provided by the present application will be described with reference to fig. 3 to 5.

FIG. 3 is a schematic flow chart diagram of a method for sending a target instruction according to an embodiment of the present application. The method shown in fig. 3 may be applied to the memory system 100 shown in fig. 1 and also to the memory system 200 shown in fig. 2.

301, a first control device in the production center receives a target command from a host.

A plurality of control devices may be included in the production center. The first control device receives a target command from the host. In other words, the target instruction transmitted by the host computer may be transmitted to any one of the plurality of control apparatuses of the generation center. The first control device is a control device of the plurality of control devices that receives the target instruction.

The target instruction may be a copy request, a write instruction, a store instruction, or other instruction that requires intervention by the disaster recovery center and feedback of the result.

302 which determines a second control device, which is one of the at least one control devices in the disaster recovery center.

The second control device is a destination node of the target instruction.

In some embodiments, the target instruction may include an identifier of a destination node, and the first control device may determine that a control device in the disaster recovery center corresponding to the identifier of the destination node is the second control device.

In other embodiments, the first control device may determine, according to the load conditions of a plurality of control devices in the disaster recovery center, that the control device with the smallest load is the second control device of the destination control device.

In other embodiments, the first control device may determine that any of the control devices in the disaster recovery center is the second control device of the destination control device.

303, the first control means may be according to P₁Round trip delay of a link between a control device and the second control device, from the P₁A third control means is determined in the respective control means.

Assuming that the production center includes P control devices in total, P is a positive integer greater than or equal to 2. Each of the P control devices can be connected to one or more control devices in the disaster recovery center via at least one link.

The first control device may determine the control device of the P control devices connected to the second control device after determining the second control device. Suppose that the P control devices share P₁A control device connected to the second control device, P₁Is a positive integer greater than or equal to 2 and less than or equal to P. In other words, P control devices include P₁A control device, P₁Each of the control devices includes at least one link with the second control device.

Taking the memory system 100 shown in FIG. 1 as an example, assuming that the second control device is the control device 121, the P is₁Each control means comprises a control means 111 and a control means 112.

Taking the memory system 200 shown in FIG. 2 as an example, assume that the second control device is the control device 222, then P is₁Each control means comprises a control means 211, a control means 212 and a control means 213.

The P is₁Each of the individual control devices may monitor the round trip delay of each link between the individual control device and the second control device.

Also taking the storage system 200 shown in fig. 2 as an example, assuming that the second control device is the control device 222, the control device 211 may monitor the round trip delay of the link 2c, the control device 212 may monitor the round trip delay of the link 2b-1 to the link 2b-4, and the control device 213 may monitor the round trip delay of the link 2d-1 and the link 2 d-2.

In some embodiments, the P₁The control means may send the round trip delay of the link having the lowest round trip delay to the first control means. The P is₁The round trip delay of the link fed back by the individual control means may be referred to as the first round trip delay. In other words, the first control means can acquire P₁A first round trip delay, P₁A first round trip delay time of P₁A minimum round trip delay between the control device and the second control device. In other words, the P₁The pth of the first round trip delays is the P₁Minimum round-trip time between the pth control device of the control devices and the second control device, P being 1, …, P₁. The first control means may be in accordance with P₁A first round trip delay determines the third control means.

It is understood that if P is present₁If there is only one link between one or more of the first control devices and the second control device, the one or more control devices may send the round trip delay of the one link to the first control device.

In other embodiments, the P₁Each of the plurality of control devices may count an average round trip delay of the plurality of links with the lowest round trip delay (e.g., count the round trip delay of the 3 links with the lowest round trip delay, or the round trip delay of the 4 links with the lowest round trip delay), and transmit the average round trip delay as the first round trip delay to the control deviceThe first control device. If P is₁If the number of links between one or more of the plurality of control devices and the second control device is only one, the one or more control devices may transmit the round trip delay of the one link to the first control device. If P is₁If the total number of links between one or more of the plurality of control devices and the second control device is less than the predetermined number, the one or more control devices may transmit the round trip delay of the link having the lowest round trip delay to the first control device, or may transmit the round trip delays of all the links to the first control device.

In some embodiments, the P₁Each of the plurality of control means selects from the at least one link a remaining transmission resource greater than a remaining resource threshold in determining the minimum round trip delay. In other words, the first round trip delay that each control device sends to the first control device is the minimum round trip delay in the candidate link. The candidate links include one or more of at least one link between the each control device and the second control device. The remaining transmission resources for each of the candidate links is greater than the remaining resource threshold.

Taking the control device 212 in the storage system 200 shown in fig. 2 as an example, the second control device is assumed to be the control device 222. Assume that the predetermined remaining resource threshold is 512B, the remaining transmission resources of link 2B-1 are 1MB, the remaining transmission resources of link 2B-2 are 4MB, the remaining transmission resources of link 2B-3 are 2MB, and the remaining transmission resources of link 2B-4 are 256B. In this case the smallest round trip delay sent by the control means 212 is the smallest value of the round trip delay in link 2b-1 to link 2 b-3.

In some embodiments, the remaining resource link threshold may be predetermined, for example, the average size of all target instructions received by the production center. As another example, it may be the average size of the majority of target instructions received by the production center.

In other embodiments, the first control device may increase the target instructionSend the target instruction to the P little or directly₁A control device. The remaining resources are the size of the target instruction.

In other embodiments, the P₁Each of the control means does not take into account whether the remaining transmission resources of the link with the smallest round trip delay are available for transmitting the target instruction when determining the smallest round trip delay. In other words, the P₁Each of the control means may transmit the round trip delay with the smallest round trip delay to the first control means.

Taking the control device 212 in the storage system 200 shown in fig. 2 as an example, the second control device is assumed to be the control device 222. The smallest round trip delay sent by the control means 212 is the smallest value of the round trip delay in the link 2b-1 to the link 2 b-4.

The P is₁The control device may or may not include the first control device. In other words, in some embodiments, there may be no link connection between the control device that received the target instruction and the destination node of the target instruction. In other embodiments, the control device receiving the target instruction has a link connection with a destination node of the target instruction.

The third control device may be the same as the first control device or may be different from the first control device. In other words, in some embodiments, the control device for sending the target instruction to the disaster recovery center (i.e., the third control device) and the control device for receiving the target instruction from the host (i.e., the first control device) may be the same. In other embodiments, the control device for sending the target instruction to the disaster recovery center (i.e., the third control device) and the control device for receiving the target instruction from the host (i.e., the first control device) may be different. If the first control device is different from the third control device, the first control device needs to send the target instruction to the third control device. Accordingly, the third control device receives the target command from the first control device.

The first control device is based on the P₁A first round trip delay determining the third controlling means, comprising: the first control device is based on the P₁A first round trip delay determination P₁A second round trip delay, P₁The pth second round trip delay of the pth second round trip delays is the pth first round trip delay and P₁The sum of round trip delays of the p-th control device of the plurality of control devices to the first control device. The first control means determines the P₁The control device corresponding to the smallest round trip delay among the second round trip delays is the third control device.

Taking the control device 212 in the storage system 200 shown in fig. 2 as an example, it is assumed that the first control device is the control device 211 and the second control device is the control device 222. It is assumed that the round trip delay transmitted from control device 212 to control device 211 is the round trip delay of link 2b-1, and the round trip delay transmitted from control device 213 to control device 211 is the round trip delay of link 2 d-1. In this case, the first control means (i.e., the control means 211) may acquire three first round trip delays, respectively, the round trip delay of the link 2c, the round trip delay of the link 2b-1, and the round trip delay of the link 2 d-1. The control means 211 may determine three second round trip delays based on the three first round trip delays. The first of the three second round trip delays is the round trip delay of link 2c, the second is the sum of the round trip delay of link 2b-1 and the round trip delay of link 2e, and the third is the sum of the round trip delay of link 2d-1 and the round trip delay of link 2 g. Assuming that the second one of the three second round trip delays (i.e., the sum of the round trip delay of link 2b-1 and the round trip delay of link 2 e) is the smallest, control device 211 may determine control device 212 as the third control device.

The P is₁A second round trip delay of P₁Round trip delay of the path of the strip from the first control means to the second control means. P₁Some of the paths may be a link, and other paths may be composed of a plurality of links and a control device.

Taking the control device 212 in the storage system 200 shown in fig. 2 as an example, it is assumed that the first control device is the control device 211 and the second control device is the control device 222. Three paths are included between the first control device 211 and the second control device 222: path 1 is link 2 c; path 2 consists of link 2e, control device 212, and link 2 b-1; path 3 is comprised of link 2g, control device 213 and link 2 d-1. The round trip delay of path 1 is the first second round trip delay of the three round trip delays, the round trip delay of path 2 is the second round trip delay, and the round trip delay of path 3 is the third second round trip delay. As described above, the second round trip delay is minimal. Thus, path 2 is the path with the least round trip delay from the production center to the second control device. The control device corresponding to the path having the smallest round trip delay from the production center to the second control device is the control device 212.

304, the third control means determining a second set of links, the second set of links comprising N₂Bar link, N₂Is a positive integer greater than or equal to 1 and less than or equal to N.

The third controlling device may determine a link belonging to the second link set from among at least one link included between the third controlling device and the second controlling device. For convenience of description, it may be assumed that N links are included between the third control device and the second control device. N included in the second set of links₂The links belong to the N links.

N included in the second set of links₂The bar link is the link of the N links that can be used to transmit the target instruction.

In some embodiments, the third controlling means determining the second set of links may comprise: the third control means determines the N links based on the target command and the remaining transmission resources of each of the N links₂A bar link, wherein the N₂The size of the remaining transmission resource of any one of the links is not smaller than the size of the target instruction.

In other words, the third controlling means may determine, as the link in the second link set, a link in which the size of the transmission resource remaining in the N links is larger than the size of the target instruction.

The transmission resources of each of the N links are allocated by the third control means. The third control means may allocate transmission resources for each link according to the load capability information of each link. In other words, the third controlling means may allocate transmission resources for the nth link according to the load capability information of the nth link among the N links. Thus, the third control means may determine the remaining transmission resources for each link.

For example, in some embodiments, the third control means may employ a token bucket algorithm to allocate transmission resources for the nth link. The third control means may issue tokens for the nth link at a rate, each token allowing a certain number of bits to be sent. Every time a packet is sent through the nth link, the number of tokens having the same size as the data packet is deleted from the token bucket of the nth link. The third control means may determine the number of tokens remaining for the nth link. If the number of bits corresponding to the number of remaining tokens of the nth link is greater than or equal to the size of the target instruction, the nth link can be used for transmitting the target instruction. In this case, the third controlling means may determine that the nth link belongs to the second link set. If the number of bits corresponding to the remaining number of tokens in the nth link is smaller than or equal to the size of the target instruction, the nth link cannot be used for transmitting the target instruction, and in this case, the third controlling device may determine that the nth link does not belong to the second link set.

Assuming that each token can be used to send NUM _ P bits of data, the size of the target instruction is NUM _ Q bits, NUM _ P and NUM _ Q are positive integers greater than or equal to 1, and NUM _ P is less than NUM _ Q. Take the memory system 200 shown in fig. 2 as an example. Assume that the third control device is control device 212 and the second control device is control device 222. Assume that the remaining number of tokens for links 2b-1 and 2b-2 is greater than

Then links 2b-1 and 2b-2 belong to the second set of links where the symbol is

Represents rounding up; assume that the number of remaining tokens for links 2b-3 and 2b-4 is less than

Then links 2b-3 and 2b-4 do not belong to the second set of links.

In other embodiments, the third controlling means determining the second set of links may comprise: the third control means determines the N links based on the target instruction and the load of each of the N links₂And (4) a link. The N is₂The load of each of the links is less than the load threshold of each of the links.

For example, the third control means may periodically or aperiodically stress the N links to determine the maximum load of each of the N links. After determining the maximum load of each link, the third controlling means may determine a load threshold for each link, for example, the load threshold may be 60%, 70%, 80%, etc. of the maximum load. If the current load of the nth link does not exceed the load threshold of the nth link, it may be determined that the nth link belongs to the second link set; if the current load of the nth link exceeds the load threshold of the nth link, it may be determined that the nth link does not belong to the second link set.

The third control means determines 305 a first set of links based on the second set of links.

The first set of links includes the N₂N in a link₁And (4) a link.

In some embodiments, the third control means may determine the N first₂One of the links (which may be referred to as a first link) with the smallest round trip delay is the smallest link. The third controlling means may determine the N in case that the first link is determined₂N of the bar link except the first link₂-the difference between the round trip delay of each of the 1 links and the round trip delay of the first link. If N is present₂-round trip delay of one of 1 links with the first linkIf the difference of the round trip delay of the link is smaller than a preset threshold value, the link belongs to the first link set, otherwise, the link does not belong to the first link set.

In some embodiments, the preset threshold may be a fixed preset time, such as 1ms, 3ms, or 5ms, etc.

In other embodiments, the predetermined threshold may be a value determined based on a round trip delay of the first link. For example, the preset threshold may be RTT₁×1％，RTT₁X 2%, or RTT₁X 5% where RTT₁Is the round trip delay of the first link.

In other embodiments, the third control means may determine the N₂The K links with the smallest round trip delay among the links constitute the first set of links.

In some embodiments, K may be a preset fixed value. For example, K may have a value of 3, 5, or 7.

In other embodiments, K may be according to N₂A determined value. For example, K may be equal to

Where k' may equal 3, 4, or 5, etc., symbols

Indicating rounding up.

The third control means determines 306 a target link from the first set of links.

In some embodiments, the first set of links may include only one link, i.e., the first link. In other words, in some embodiments, N in the second set of links₂-the round trip delay of any of the 1 links and the round trip delay of the first link are both greater than the preset threshold. In this case, only one link may be included in the first set of links.

In other embodiments, the first link set may include N in addition to the first link ₁1 link. The first part isThe links in a wire bond other than the first link may be referred to as second links. In other words, in some embodiments, the first link set may include one first link and N₁-1 second link. Optionally, the N₁-the difference between the round trip time domain of each of the 1 second links and the round trip delay of the first link is smaller than a preset threshold.

If only one link is included in the first link set, the first link is the first link. The third control means may determine that the target link is the first link.

If the first link set includes a plurality of links, the third control device may determine that the target link is the link with the smallest load among the plurality of links.

In some embodiments, if the number of the least loaded links in the plurality of links is greater than 1, the third control device may determine one of the least loaded links having the smallest round trip delay as the target link.

In other embodiments, if the least loaded link number of the links is greater than 1, the third control device may determine any link of the least loaded links as the target link.

307, the third control device sends the target command to the second control device through the target link.

After the target command is sent to the second control device, the second control device needs to feed back whether the target command is successfully processed to the third control device. For example, if the target instruction is a copy request, the second control device needs to feed back whether to store the data corresponding to the copy request in the disaster recovery center. After receiving the feedback information, the production center may determine a subsequent operation according to the feedback information. For example, the production center may create a log after receiving the target instruction. If the feedback information indicates that the target instruction is successfully executed (for example, the data corresponding to the replication request is successfully stored in the disaster recovery center), the production center may delete the log; otherwise, the production center keeps the log and carries out subsequent processing.

As described above, the production center needs to obtain feedback information corresponding to the target instruction sent by the disaster recovery center, and perform subsequent operations according to the feedback information. Therefore, the production center needs to select a link with a smaller round-trip delay to send the received target instruction to the disaster recovery center. In the method shown in fig. 3, the production center performs global selection, selects a control device having the link with the smallest round-trip delay from a plurality of control devices included in the production center, and then selects an appropriate link from the control devices as a target link for sending a target instruction, so that the throughput from the production center to the disaster recovery center can be improved. For example, if the round-trip delay of a link between the production center and the disaster center is significantly increased, the throughput from the production center to the disaster center is not significantly compromised by the method shown in fig. 3, and if the existing load-based target instruction distribution scheme is used, the throughput from the production center to the disaster center is reduced.

For example, FIG. 4 is a graph comparing simulation results using the method of FIG. 3 and a prior art method. The production center and the disaster recovery center of the storage system are assumed to be double controllers, and the production center and the disaster recovery center are connected in a full-connection mode. And a Fiber Channel (FC) with the bandwidth of 8G is arranged between each controller of the production center and each controller of the disaster recovery center. In other words, each controller in the production center has two links that are connected to two controllers of the disaster recovery center, respectively. Assume that one of the two links of each controller of the production center is injected with a link delay of 10 ms. FIG. 4 is a graph of throughput for the memory system after injection of a link delay using the method shown in FIG. 3 and a prior art method. It can be seen that the throughput from the production center to the disaster recovery center using the method shown in fig. 3 is significantly greater than the throughput from the production center to the disaster recovery center using the prior art method. Therefore, the method shown in fig. 3 can reduce the throughput reduction from the production center to the disaster recovery center due to the link delay.

In the method shown in fig. 3, the third control device determines a second link set (i.e. a set of N links capable of transmitting the target command), and then determines the target link from the second link set. In this case, the target link must be a link that can be used to transmit the target instruction. In other words, the size of the remaining transmission resources of the target link must not be smaller than the size of the target instruction.

In other embodiments, the third controlling means may further determine the link with the smallest round trip delay among the N links, and then determine whether the size of the remaining transmission resource of the link with the smallest round trip delay is larger than the size of the target instruction. If the size of the remaining transmission resources of the link is not smaller than the size of the target instruction, determining the link as the target link; if the size of the remaining transmission resources of the link is smaller than the size of the target instruction, continuing to determine whether the size of the remaining transmission resources of the link with the second smallest round trip delay among the N links is larger than the size of the target instruction, and if the size of the remaining transmission resources of the link is not smaller than the size of the target instruction, determining that the link is the target link; if the size of the remaining transmission resource of the link is smaller than the size of the target instruction, continuously determining whether the size of the remaining transmission resource of the link with the third smallest round trip delay in the N links is larger than the size of the target instruction, and so on.

In the method shown in fig. 3, the third control device may allocate transmission resources to each link according to the load capability information of each link, and further may determine the remaining transmission resources of each link, and determine the second link set according to the remaining transmission resources. Fig. 5 will be gathered to describe how the third control means allocates transmission resources for each link.

Fig. 5 is a schematic flow chart of a method for allocating transmission resources provided according to an embodiment of the present application. Fig. 5 is a diagram illustrating an nth link of the N links, and illustrates how the third control device allocates transmission resources for the nth link.

The third control means obtains load capability information 501.

The load capability information includes at least one of a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay.

The link capability evaluation value of the nth link reflects the maximum flow rate which can be transmitted by the nth link in unit time.

As described above, the method shown in fig. 5 is to describe how the third control device allocates transmission resources for the nth link, so that the load capability information is also the load capability information of the nth link. Accordingly, the reference throughput is a reference throughput of the nth link, the reference link capability assessment is a reference link capability assessment of the nth link, the first reference round trip delay is a first reference round trip delay of the nth link, and the second reference round trip delay is a second reference round trip delay of the nth link.

In some embodiments, the reference throughput is the total throughput in the previous statistical cycle.

In other embodiments, the reference throughput is top M₁M within one statistical period₁Average of individual total throughputs, M₁Is a positive integer greater than or equal to 2. M₁The average of the total throughput may be M₁Arithmetic mean of individual total throughputs, M₁Moving average of individual total throughputs, or M₁Geometric mean of individual total throughputs, etc.

In some embodiments, the reference link capability estimate is the link capability estimate in the previous statistical period.

In other embodiments, the reference link capability estimate is preceded by M₂M within one statistical period₂Average of individual link capability evaluation values, M₂Is a positive integer greater than or equal to 2. M₂The average of the individual link capability evaluation values may be M₂Arithmetic mean of individual link capability evaluation values, M₂Moving average of individual link capability evaluation values, or M₂Geometric mean of individual link capability estimates, etc.

In some embodiments, the first reference round trip delay is an average round trip delay over a previous statistical period.

In other embodiments, the first reference round trip delay is first M₃M within one statistical period₃Average value of average round trip delay, M₃Is a positive integer greater than or equal to 2. M₃The average value of the average round trip delay can be M₃Arithmetic mean of individual average round trip delays, M₃Moving average of the average round trip delay, or M₃Geometric mean of the individual round trip delays, etc.

In some embodiments, the second reference round trip delay is an average round trip delay when the nth link does not exceed the maximum load capacity.

In some embodiments, M₁、M₂And M₃May be the same, e.g. M₁＝M₂＝M₃M is a positive integer greater than or equal to 2.

In other embodiments, M₁、M₂And M₃Any two of which may be different. For example, M₁May be equal to M₂And M is₁Is not equal to M₃. As another example, M₁Is not equal to M₂，M₁Is not equal to M₃，M₂Is not equal to M₃。

502, the third control device determines the target link capability evaluation value of the nth link according to the load capability information.

In some embodiments, the load capability information includes a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay. In this case, the third controlling means determines the target link capability evaluation value of the nth link based on the load capability information, including:

the third control means determines whether the reference throughput is greater than a first evaluation parameter determined based on the reference link capability evaluation value, the first evaluation parameter being less than the reference link capability evaluation value;

if the reference throughput is greater than the first evaluation parameter, determining the target link capability evaluation value according to the reference link capability evaluation value, wherein the target link capability evaluation value is greater than the reference link capability evaluation value;

if the reference throughput is not greater than the first evaluation parameter, determining whether the reference throughput is greater than a second evaluation parameter, the second evaluation parameter being determined according to the reference link capability evaluation value, the second evaluation parameter being less than the first evaluation parameter;

if the reference throughput is greater than the second evaluation parameter, determining that the target link capability evaluation value is the same as the reference link capability evaluation value;

if the reference throughput is not greater than the second evaluation parameter, determining whether the first round trip delay is greater than a second round trip delay;

if the first reference round trip delay is larger than the second reference round trip delay, determining the target link capacity evaluation value according to the reference link capacity evaluation value, wherein the target link capacity evaluation value is smaller than the reference link capacity evaluation value;

if the first reference round trip delay is not greater than the second reference round trip delay, it may be determined that the target link capability assessment value is the same as the reference link capability assessment value.

The first evaluation parameter and the second evaluation parameter may be determined according to the following formula:

EST₁＝C_{est_r}x α, formula 1.1

EST₂＝C_{est_r}X beta, equation 1.2

Wherein the EST₁Representing the first evaluation parameter, EST₂Representing the second evaluation parameter. C_{est_r_}Indicating the reference link capability assessment value, and alpha and beta indicating throughput attenuation threshold coefficients. The values of α and β may be greater than 0 and less than 1, with α being greater than β. For example, α may equal 0.95 and β may equal 0.8.

In the case where the reference throughput is greater than the first evaluation parameter, the third control means may determine the target link capacity evaluation value according to the following formula:

C_est＝C_{est_r}x (1+ lambda), equation 1.3

Wherein, C_estRepresents the target link capability evaluation value, C_{est_r_}Representing the reference link capability assessment value, λ is a link capability assessment value adjustment step size, and λ may be a number greater than 0 and less than 1. For example, λ may be equal to 0.2.

In the case where the first reference round trip delay is greater than the second reference round trip delay, the third control means may determine the target link capacity evaluation value according to the following formula:

C_est＝C_{est_r}x (1-. eta.), equation 1.4

Wherein, C_estRepresents the target link capability evaluation value, C_{est_r_}Represents the reference link capability assessment value, η is a link capability assessment value adjustment step size, and η may be a number greater than 0 and less than 1. For example, η may be equal to 0.2. The values of η and λ may be the same or different.

if the first reference round trip delay is not greater than the second reference round trip delay, determining whether the reference throughput is greater than a first evaluation parameter, wherein the first evaluation parameter is determined according to the reference link capability assessment value, and the first evaluation parameter is less than the reference link capability assessment value;

if the reference throughput is not greater than the second evaluation parameter, the target link capability evaluation value may be determined according to the reference link capability evaluation value, wherein the target link capability evaluation value is smaller than the reference link capability evaluation value.

In the case where the reference throughput is not greater than the second evaluation parameter, the third control means may determine the target link capability evaluation value according to equation 1.4.

In other embodiments, the load capability information may also include portions of the reference throughput, the reference link capability assessment value, the first reference round trip delay, and the second reference round trip delay.

For example, the load capability information may include only the reference link capability assessment value, the first reference round trip delay, and the second reference round trip delay. In this case, the third controlling means determines the target link capability evaluation value of the nth link based on the load capability information, including: the third control means determines whether the first round trip delay is greater than the second round trip delay; if the first reference round trip delay is greater than the second reference round trip delay and the difference between the first reference round trip delay and the second reference round trip delay is greater than a first preset delay difference, determining the target link capacity evaluation value according to the reference link capacity evaluation value, wherein the target link capacity evaluation value is less than the reference link capacity evaluation value; if the first reference round trip delay is greater than the second reference round trip delay and a difference between the first round trip reference delay and the second reference round trip delay is not greater than the first preset delay difference, or the first reference round trip delay is less than the second reference round trip delay and a difference between the second round trip reference delay and the first reference round trip delay is not greater than a second preset delay difference, it may be determined that the target link capability evaluation value is the same as the reference link capability evaluation value; if the first reference round trip delay is smaller than the second reference round trip delay and the difference between the second round trip reference delay and the first reference round trip delay is larger than the second preset delay difference, determining the target link capability evaluation value according to the reference link capability evaluation value, wherein the target link capability evaluation value is larger than the reference link capability evaluation value.

In other words, if the first reference round trip delay is much larger than the second reference round trip delay, the link capability assessment value may be decreased (e.g., the target link capability assessment value is determined using equation 1.4); if the first reference round trip delay is far less than the second reference round trip delay, the link capability assessment value may be increased (e.g., the target link capability assessment value is determined using equation 1.3); if the difference between the first reference round trip delay and the second reference round trip delay is not large, the reference link capability assessment value can be kept unchanged.

As another example, the load capability information may include a reference throughput and a reference link capability assessment value. In this case, the third controlling means determines the target link capability evaluation value of the nth link based on the load capability information, including:

The first evaluation parameter and the second evaluation parameter may be determined according to the above equation 1.1 and equation 1.2, respectively. In the case where the reference throughput is greater than the first evaluation parameter, the third control means may determine the target link capability evaluation value according to equation 1.3. In the case where the reference throughput is not greater than the second evaluation parameter, the third control means may determine the target link capability evaluation value according to equation 1.4.

503, the third controlling means allocates transmission resources for the nth link according to the target link capability evaluation value of the nth link.

In some embodiments, the first control parameter may use a token bucket algorithm to allocate transmission resources for the nth link. The third controlling means may allocate transmission resources for the nth link according to the target link capability evaluation value of the nth link, including: the third controlling means may determine the speed of issuing the token for the nth link based on the target link capability evaluation value for the nth link.

For example, the target link capability evaluation value and the token issuance speed may have the following relationship:

C_estv × To, formula 1.5

Wherein, C_estIndicates the target link capability evaluation value, V indicates the token issuance speed, and To indicates the number of bits allowed To be transmitted per token.

For example, if the target capability evaluation value of the nth link is 1GB/s and the amount of data allowed to be transmitted per token is 1B, the third control means may determine that the speed of issuing tokens for the nth link is 10⁶Token/ms. In other words, the third control means may be provided every other1ms for nth link distribution 10⁶A token.

In other embodiments, the third controlling means may allocate transmission resources for the nth link according to the target link capability evaluation value, and the allocating may include: the third controlling means may periodically allocate a transmission resource of a certain size to the nth link, and the allocation period or the size of the transmission resource allocated each time may be determined according to the target link capability evaluation value.

In some implementations, the allocation period for the third control device to allocate the transmission resource for the nth link may be determined according to the target link capability evaluation value. For example, assuming that the target link capability evaluation value is 1GB/s, the transmission resource of a specific size allocated in each period is 10⁸B, the period for allocating transmission resources may be 100 ms. In other words, the first control resource may allocate the nth link with a size of 10 every 100ms⁸Transmission resources of B.

In other implementations, the size of the transmission resource allocated by the third control means in each period may be determined according to the target capability evaluation value. For example, assuming that the target link capability evaluation value is 1GB/s and the period for allocating transmission resources is 100ms, the transmission resources of a specific size allocated in each period is 10⁸B. In other words, the first control resource may allocate the nth link with a size of 10 every 100ms⁸Transmission resources of B.

The third controlling means may record the total size of the transmission resources allocated for the nth link and the size of the transmission resources consumed. In this way, the third control means may determine the size of the transmission resource remaining for the nth link based on the total size of the allocated transmission resources and the size of the consumed transmission resource.

The control devices other than the third control device in the production center may also allocate transmission resources to each link in the control device using the method shown in fig. 5.

Fig. 6 is a schematic structural block diagram of a control device provided according to an embodiment of the present application. The control apparatus 600 shown in fig. 6 includes a transceiving unit 610 and a link management unit 620. The control apparatus 600 shown in fig. 6 may be any one of the control apparatuses in the production center shown in fig. 1 or fig. 2. The control means 600 may be the first control means or the third control means in the method as shown in fig. 3.

The transceiver unit 610 is used for receiving target commands from a host or other control devices. The transceiving unit 610 may also be configured to send the target instruction to a control device of the disaster recovery center or another control device in the production center.

And the link management unit 620 is configured to obtain a forwarding delay for the control device 600 to forward the target instruction to other control devices of the production center.

And the link management unit 620 is further configured to obtain a minimum round-trip delay of a link from each control device of the production center to a second control device, where the second control device is a control device of the disaster recovery center.

The link management unit 620 is further configured to determine a third control device, where the third control device is a control device corresponding to a path with a minimum round-trip delay from the production center to the second control device.

The link management unit 620 is further configured to determine a target link from N links from the control device 600 to the second control device, where N is a positive integer greater than or equal to 1.

The link management unit 620 is further configured to send the target instruction to the second control device through the target link.

Optionally, the link management unit 620 is specifically configured to determine a first link set, where the first link set includes N links of the N links₁A link of N₁The link comprises a first link which is the link with the minimum round trip delay among the N links, wherein N is₁Is a positive integer greater than or equal to 1 and less than or equal to N, N being a positive integer greater than or equal to 1; a target link is determined from the first set of links.

Optionally, the N₁The link also comprises at least one second link, and the round trip delay of any one of the at least one second link is equal to that of the first linkThe difference of the round trip delay is smaller than a preset threshold value.

Optionally, the link management unit 620 is further configured to determine a second link set, where the second link set includes N links of the N links₂A link of N₂The link is the link that can be used to transmit the target instruction in the N links, where N is₂Is greater than or equal to N₁And is less than or equal to a positive integer of N; the link management unit 620 is specifically configured to determine the first link set according to the second link set.

Optionally, the link management unit 620 is specifically configured to determine the N links according to the target instruction and the remaining transmission resources of each link in the N links₂A bar link, wherein the N₂The size of the remaining transmission resources of any one of the links is not less than the size of the target instruction.

Optionally, the link management unit 620 is further configured to allocate transmission resources to an nth link of the N links according to the load capability information of the nth link, where N is 1, …, and N.

Optionally, the load capability information of the nth link includes at least one of the following information: a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay. A link management unit 620, configured to determine a target link capability evaluation value of the nth link according to the load capability information of the nth link; and allocating transmission resources for the nth link according to the target link capability evaluation value.

Optionally, the reference throughput is a total throughput in a previous statistical period, or an average of M total throughputs in the previous M statistical periods, where M is a positive integer greater than or equal to 2; the reference link capacity evaluation value is a link capacity evaluation value in a previous statistical period, or an average value of M link capacity evaluation values in the previous M statistical periods; the first reference round trip delay is an average round trip delay in a previous statistical period, or an average value of M average round trip delays in previous M statistical periods; the second reference round trip delay is an average round trip delay when the nth link does not exceed the maximum load capacity.

Optionally, the link management unit 620 is specifically configured to determine the first link from the second link set; according to the time delay of the first link and the preset threshold value, from N₂The at least one second link is determined among the links.

Assuming that P control devices are collectively included in the production center, P is a positive integer greater than or equal to 2. Among the P control devices, P₁At least one link, P, between the control device and the second control device₁Is a positive integer greater than or equal to 2 and less than or equal to P. If the control device 600 is the first control device in the method of FIG. 3, then in some embodiments the P is₁A control device may include control device 600, and in other embodiments P₁The control device 600 may not be included in the control device.

At the P₁In the case where the control apparatus includes the control apparatus 600, the link management unit 620 may obtain the round trip delay of the link with the smallest round trip delay fed back by the link performance monitoring unit 621, and obtain the P through the transceiving unit 610₁A first round trip delay of transmissions of a control device other than the control device 600 among the control devices.

The link performance monitoring unit 621 may be included in the link management unit 620. The link performance monitoring unit 621 may be configured to detect a round trip delay of a link between the control apparatus 600 and each control apparatus in the disaster recovery center. For example, assuming that the control device 600 is the control device 111 shown in fig. 1, the link performance monitoring unit 621 may detect the round trip delay of the link 1a-1, the round trip delay of the link 1a-2, and the round trip delay of the link 1 c.

The link performance monitoring unit 621 may count a time delay of a target instruction passing through a link to the control device of the disaster recovery center and then receiving feedback information corresponding to the target instruction, the time delay being taken as a round trip time delay. In other words, the round trip delay may be a delay from when the control device transmits the target instruction to when the feedback information corresponding to the target instruction is received. In some embodiments, the round trip delay of each link acquired by the link performance monitoring unit 621 may be the round trip delay of the last transmission target instruction of each link. In other embodiments, the round trip delay of each link obtained by the link performance control unit 621 may be an average value of the round trip delay of each link from the beginning of transmission of the target instruction to the last transmission of the target instruction. Further assuming that the control device 600 is the control device 111 shown in fig. 1, the link performance monitoring unit 621 may detect that the round trip delay of the link 1a-1 is an average value of round trip delays of the link 1a-1 from the first transmission of the target instruction to the disaster recovery center to the last transmission of the target instruction. For example, assuming that at time 1, link 1a-1 is commonly used to transmit 10 target instructions, the round trip delay of link 1a-1 is the average of the 10 round trip delays. Assuming that at time 2, link 1a-1 completes the transmission of the target command and the reception of the corresponding feedback information for 11 th time, link performance monitoring unit 621 may update the round trip delay of link 1a-1 according to the round trip delay of this time. The average may be an arithmetic average, a sliding average, a geometric average, or the like. In other embodiments, the round trip delay of each link obtained by the link performance control unit 621 may be an average value of round trip delays of the first Num _ k times transmission target instructions of each link, where Num _ k is a positive integer greater than or equal to 2. The average may be an arithmetic average, a sliding average, a geometric average, or the like.

The link performance monitoring unit 621 may also transmit the round trip delay of the link with the smallest round trip delay to other control units of the production center through the transceiving unit 610.

At the P₁In the case where the individual control device does not include the control device 600, the transmitting/receiving unit 610 may receive P₁P transmitted by a control device₁A round trip delay. The link management unit 620 may acquire the P received by the transceiving unit 610₁A first round trip delay.

The link performance monitoring unit 621 is also capable of acquiring a round trip delay of a link for connecting the control apparatus 600 and another control apparatus belonging to the production center. For example, assuming that the control device 600 is the control device 111 shown in fig. 1, the link performance monitoring unit 621 may acquire the round trip delay of the link 1 e. The link performance monitoring unit 621 may acquire a one-way delay of a link from the control device 600 to another control device of the production center, and then multiply the one-way delay by 2 as a round trip delay of the link.

Optionally, the link management unit 620 may include a link capability evaluation unit 622. The link capability evaluating unit 622 may be configured to determine a target link capability evaluation value of the nth link according to the load capability information of the nth link.

Optionally, the link management unit 620 may further include a traffic scheduling unit 623. The traffic scheduling unit 623 is configured to allocate a transmission resource for the nth link according to the target link capability evaluation value of the nth link.

Optionally, the operations of determining the first link set, determining the second link set, determining the third control device and/or determining the target link, which are implemented by the link management unit 620, may be implemented by the traffic scheduling unit 623.

For the specific functions and advantages of the transceiving unit 610 and the link management unit 620, reference may be made to the description in the foregoing method embodiments, and for brevity, detailed description is omitted here.

Optionally, the control device 600 is a chip. The transceiver unit 610 may be a communication interface on the chip. The link management unit 620 may be a processor or microprocessor or an integrated circuit integrated on the chip.

Optionally, the production center and the disaster recovery center in the storage system are relative to the host. Taking the storage system 100 shown in fig. 1 as an example, for the host 1, in fig. 1, the production center 101 is a production center, and the disaster recovery center 102 is a disaster recovery center; for the host 2, the production center 101 in fig. 1 is a disaster recovery center, and the disaster recovery center 102 is a production center. In other words, the identity of a hub in a storage system is also different in the case of different hosts. Therefore, the control device 600 may be a control device in a disaster recovery center in the storage system shown in fig. 1 or fig. 2.

Fig. 7 is a schematic structural diagram of a storage system provided according to an embodiment of the present application. The storage system 700 shown in fig. 7 includes a production center 710 and a disaster recovery center 720. The production center 710 includes a first control device 701 and a third control device 703. Disaster recovery center 720 includes a second control device 702. The first control means 701, the second control means 702 and the third control means 703 may be control means as shown in fig. 6.

The first control device 701 is configured to receive a target instruction.

The first control device 701 is further configured to obtain a forwarding time delay for the first control device 701 to forward the target instruction to other control devices of the production center.

The first control device 701 is further configured to obtain a minimum round trip delay of a link from each control device of the production center to the second control device 702.

The first control device 701 is further configured to determine a third control device 703, where the third control device 703 is a control device corresponding to a path with a minimum round-trip delay from the production center to the second control device 702.

A third controlling means 703 for determining a target link from N links from the third controlling means 703 to the second controlling means 702, where N is a positive integer greater than or equal to 1.

The third control device 703 is further configured to send the target command to the second control device through the target link.

Optionally, the third controlling means 703 is specifically configured to determine a first link set, where the first link set includes N links of the N links₁A link of N₁The link comprises a first link which is the link with the minimum round trip delay among the N links, wherein N is₁Is a positive integer greater than or equal to 1 and less than or equal to N, N being a positive integer greater than or equal to 1; a target link is determined from the first set of links.

Optionally, the N₁The link also comprises at least one second link, and the difference between the round trip delay of any one second link in the at least one second link and the round trip delay of the first link is less than a preset threshold value

Optionally, the third controlling means 703 is further configured to determine a second link set, wherein the second link set isThe two-link set comprises N of the N links₂A link of N₂The link is the link that can be used to transmit the target instruction in the N links, where N is₂Is greater than or equal to N₁And is less than or equal to a positive integer of N. The third controlling means 703 is specifically configured to determine the first link set according to the second link set.

Optionally, the third control device 703 is specifically configured to determine the N links according to the target instruction and the remaining transmission resources of each link in the N links₂A bar link, wherein the N₂The size of the remaining transmission resources of any one of the links is not less than the size of the target instruction.

Optionally, the third controlling means 703 is further configured to allocate transmission resources to an nth link of the N links according to the load capability information of the nth link, where N is 1, …, N.

Optionally, the load capability information of the nth link includes at least one of the following information: a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay. A third control device 703, configured to determine a target link capability evaluation value of the nth link according to the load capability information of the nth link; and allocating transmission resources for the nth link according to the target link capability evaluation value.

Optionally, the reference throughput is a total throughput in a previous statistical period, or an average of M total throughputs in the previous M statistical periods, where M is a positive integer greater than or equal to 2. The reference link capacity evaluation value is a link capacity evaluation value in a previous statistical period, or an average value of M link capacity evaluation values in the previous M statistical periods; the first reference round trip delay is an average round trip delay in a previous statistical period, or an average value of M average round trip delays in previous M statistical periods; the second reference round trip delay is an average round trip delay when the nth link does not exceed the maximum load capacity.

Optionally, third controlling means 703 is specifically configured to determine the first link from the second link set(ii) a According to the time delay of the first link and the preset threshold value, from N₂The at least one second link is determined among the links.

The chip in this embodiment of the application may be a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a system on chip (SoC), a Central Processing Unit (CPU), a Network Processor (NP), a digital signal processing circuit (DSP), a Microcontroller (MCU), a programmable logic controller (PLD), other programmable logic devices, a discrete gate or transistor logic device, a discrete hardware component, or other integrated chips.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor. To avoid repetition, it is not described in detail here.

It should be noted that the processor in the embodiments of the present application may be an integrated circuit chip having signal processing capability. In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

It will be appreciated that the memory in the embodiments of the subject application can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The non-volatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), double data rate SDRAM, enhanced SDRAM, SLDRAM, Synchronous Link DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

According to the method provided by the embodiment of the present application, the present application further provides a computer program product, which includes: computer program code which, when run on a computer, causes the computer to perform the method of the embodiment shown in fig. 3 or 5.

According to the method provided by the embodiment of the present application, the present application also provides a computer readable medium storing program code, which when run on a computer causes the computer to execute the method of the embodiment shown in fig. 3 or fig. 5.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of issuing a target instruction, the method comprising:

acquiring the forwarding time delay of a first control device for forwarding a target instruction to other control devices of a production center, wherein the production center comprises a plurality of control devices, and the first control device is the control device which receives the target instruction in the production center;

acquiring the minimum round-trip delay of a link from each control device of the production center to a second control device, wherein the second control device is one control device of a disaster recovery center;

determining a third control device according to the forwarding time delay and the minimum round-trip time delay, wherein the third control device is a control device corresponding to a path with the minimum round-trip time delay from the production center to the second control device;

determining a target link from N links from the third control device to the second control device, where N is a positive integer greater than 1;

and sending the target instruction to a second control device through the target link.

2. The method of claim 1, wherein the determining a target link from the N links from the third control device to the second control device comprises:

determining a first link set, wherein the first link set comprises N of the N links₁A bar link of N₁The link comprises a first link which is the link with the minimum round trip delay in the N links, wherein N is₁Is a positive integer greater than or equal to 1 and less than or equal to N, N being a positive integer greater than 1;

a target link is determined from the first set of links.

3. The method of claim 2, wherein said N is₁The link also comprises at least one second link, and the difference between the round trip delay of any one second link in the at least one second link and the round trip delay of the first link is smaller than a preset threshold value.

4. The method of claim 3, wherein prior to determining the first set of links, the method further comprises:

determining a second set of links, wherein the second set of links includes N of the N links₂A bar link of N₂The link is a link which can be used for transmitting the target instruction in the N links, wherein N is₂Is greater than or equal to N₁And is less than or equal to a positive integer of N;

the determining a first set of links comprises

And determining the first link set according to the second link set.

5. The method of claim 4, wherein the determining the second set of links comprises:

determining the N links according to the target instruction and the remaining transmission resources of each of the N links₂A bar link, wherein said N₂The size of the remaining transmission resources of any one of the links is not less than the size of the target instruction.

6. The method of claim 5, wherein the method further comprises: and allocating transmission resources for the nth link according to the load capacity information of the nth link in the N links, wherein N is 1, … and N.

7. The method of claim 6, wherein the load capability information of the nth link comprises at least one of: a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay;

the allocating transmission resources for the nth link according to the load capability information of the nth link in the N links includes:

determining a target link capacity evaluation value of the nth link according to the load capacity information of the nth link;

and allocating transmission resources for the nth link according to the target link capacity evaluation value.

8. The method of claim 7, wherein the reference throughput is a total throughput in a previous statistical period, or an average of M total throughputs in the previous M statistical periods, M being a positive integer greater than or equal to 2;

the reference link capacity evaluation value is a link capacity evaluation value in a previous statistical period, or an average value of M link capacity evaluation values in the previous M statistical periods;

the first reference round trip delay is an average round trip delay in a previous statistical period, or an average value of M average round trip delays in previous M statistical periods;

the second reference round trip delay is an average round trip delay when the nth link does not exceed the maximum load capacity.

9. The method of any of claims 4 to 8, wherein the determining the first set of links comprises:

determining the first link from the second set of links;

according to the round trip delay of the first link and the preset threshold value, starting from the N₂The at least one second link is determined among the links.

10. The storage system is characterized by comprising a production center and a disaster recovery center, wherein the production center comprises a first control device and a third control device, and the disaster recovery center comprises a second control device:

the first control device is used for receiving a target instruction;

the first control device is further configured to obtain a forwarding time delay from the first control device to forward the target instruction to other control devices of the production center;

the first control device is further configured to obtain a minimum round-trip delay of a link from each control device of the production center to the second control device;

the first control device is further configured to determine the third control device according to the forwarding time delay and the minimum round-trip time delay, where the third control device is a control device corresponding to a path with the minimum round-trip time delay from the production center to the second control device;

the third control device is used for determining a target link from N links from the third control device to the second control device, wherein N is a positive integer greater than 1;

the third control device is further used for sending the target instruction to the second control device through the target link.

11. The storage system according to claim 10, wherein the third control means is specifically configured to determine a first set of links, wherein the first set of links comprises N of the N links₁A bar link of N₁The link comprises a first link which is the link with the minimum round trip delay in the N links, wherein N is₁Is a positive integer greater than or equal to 1 and less than or equal to N, N being a positive integer greater than 1; a target link is determined from the first set of links.

12. The storage system of claim 11, wherein said N₁The link also comprises at least one second link, and the difference between the round trip delay of any one second link in the at least one second link and the round trip delay of the first link is smaller than a preset threshold value.

13. The storage system of claim 12, wherein said third control means is further for determining a second set of links, wherein said second set of links comprises N of said N links₂A bar link of N₂The link is a link which can be used for transmitting the target instruction in the N links, wherein N is₂Is greater than or equal to N₁And is less than or equal to a positive integer of N;

the third controlling means is specifically configured to determine the first link set according to the second link set.

14. The memory system of claim 13, wherein said third control means is specifically configured to determine said N links based on said target command and remaining transmission resources of each of said N links₂A bar link, wherein said N₂The size of the remaining transmission resource of any one of the links is not smallAt the size of the target instruction.

15. The storage system according to claim 14, wherein the third control means is further configured to allocate transmission resources for an nth link of the N links according to load capability information of the nth link, where N is 1, …, N.

16. The storage system according to claim 15, wherein the load capability information of the nth link comprises at least one of: a reference throughput, a reference link capability assessment value, a first reference round trip delay, and a second reference round trip delay;

the third control device is specifically configured to determine a target link capability evaluation value of the nth link according to the load capability information of the nth link; and allocating transmission resources for the nth link according to the target link capacity evaluation value.

17. The storage system of claim 16, wherein the reference throughput is a total throughput in a previous statistical cycle, or an average of M total throughputs in the previous M statistical cycles, M being a positive integer greater than or equal to 2;

18. The storage system according to any of claims 13 to 17, the third control means, in particular for determining the first link from the second set of links; according to round trip time of the first linkDelaying said preset threshold from said N₂The at least one second link is determined among the links.