JP2010283780A - Method of controlling number of tcp connections in iscsi connection, iscsi host device, and program for controlling the number of tcp connections in iscsi connection - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To search for the optimal number of TCP connections to second and subsequent links at high speed when determining the number of TCP connections for maximizing throughput for a plurality of links. <P>SOLUTION: A distribution control section 3 is built in the initiator of an iSCSi host device which establishes an iSCSi session with an iSCSi target. When the times to start using connections Link 1 and 2 are different, the distribution control section 3 first searches for an optimal value of the number of TCP connections for maximizing the throughput for the first initiated Link 1. The distribution control section 3 next applies the optimal value of the first initiated Link 1 to the number of TCP connections for the next initiated Link 2. Then, on the basis of a throughput difference between the Links 1 and 2, the number of TCP connections for the Link 2 is adjusted. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an iSCSI technique for connecting an iSCSI host apparatus and an iSCSI device apparatus using an IP (Internet protocol) network, and more particularly to a technique for optimizing a write or read throughput value at high speed.

≪iSCSI (Internet SCSI, SCSI over TCP) ≫
A storage device composed of a single or a collection of a plurality of hard disk drives or a disk array device that controls a plurality of hard disk drives with a dedicated control unit has been put into practical use.

  Examples of interface technologies for connecting a storage apparatus and a host apparatus (server, PC) include Fiber Channel (FC), InfiniBand, and SCSI (Small Computer Systems Interface). The SCSI interface is excellent as an application for economically connecting a short distance, and is already widely used.

  In this interface, a function corresponding to a client is called an initiator, and a function corresponding to a server is called a target, and a SCSI command (CDB: Command Descriptor Block) is exchanged between the initiator and the target. As shown in Non-Patent Document 1, iSCSI (Internet SCSI) technology was standardized in 2004 by The Internet Engineering Task Force (RFC3720) as an upper protocol of TCP / IP, which is a network protocol.

  At present, the iSCSI technology is used for a line quality of gigabit or more, particularly in a backbone line such as in a data center, between data centers, and a large-scale LAN system. In the iSCSI technology, a method of using a plurality of TCP connections is defined in Non-Patent Document 1, but this is not necessarily implemented because of an optional function.

≪iSCSI speedup technology≫
On the other hand, the effectiveness of using multiple TCP connections is known as a method for preventing a decrease in iSCSI throughput due to network delay on an IP network having high delay characteristics (Non-patent Document 2.3). In addition, a method using a multilink (see Non-Patent Document 4), a method of changing a transport layer protocol (Non-Patent Document 5), and the like have been proposed as means for realizing multiple use of TCP connections.

  For example, Non-Patent Document 4, which is a technique using a multilink, proposes a technique for establishing a parallel TCP connection through a plurality of routes and using a VPN multihoming function to improve throughput. In Non-Patent Document 5, which is a method for changing the protocol of the transport layer, throughput is improved by physically multiplexing links using a plurality of LAN ports and establishing TCP connections through different paths. I am trying.

  However, the technology of Non-Patent Document 4 has a limitation in the environment in which multilink can be used, and the technology of Non-Patent Document 5 still has a problem that it is difficult to connect to a multi-vendor device.

≪TCP multiple connection technology≫
In iSCSI connection, a technique has been proposed in which the multiplicity of parallel TCP connections is dynamically controlled to maximize throughput (see Non-Patent Document 6).

(1) Link aggregation
First, there is link aggregation as a technique for bundling a plurality of physical links connecting between a server and a LAN switch, or between two LAN switches, without using a higher-speed (and expensive) interface. It becomes possible to increase the bandwidth.

  Link aggregation was standardized as IEEE 802.3ad in March 2000 (see Non-Patent Document 6). The use of this link aggregation itself is called trunking.

(2) Teaming, Bonding
A technique that allows a plurality of network interface cards to be mounted on a server to appear virtually on a single network interface by software is generally called teaming.

  Bonding realizes teaming at the kernel level of Linux (registered trademark) (see Non-Patent Document 7). In any case, when the virtual network interface is realized by software and the two are used, the bandwidth can be doubled, but such virtualization is limited to a static configuration.

RFC 3720 “Internet Small Computer Systems Interface (iSCSI)”, Apr. 2004. Q. K. Yang “On performance of parallel iSCSI informed storage systems,” In Proceedings of the 20th International Conference on Advanced Information. 1, pp. 629-636, Apr. 2006. G. Motwani and K.M. Gopinath, “Evaluation of advanced TCP stacks in the iSCSI envi- ronment using simulation model,” Proceedings of the 22nd IEEE / 13th NASA Standard. 210-217, 2005. B. K. Kancheria, G .; M.M. Narayan, and K.K. Gopinath, “Performance evaluation of Multiple TCP connection in iSCSI,” in Proceedings of the 24th IEEE Conference on Mass Storage Systems and Tech. 239-244, IEEE Computer Society, Sept. 2007. Nozomi Chishima, Minoru Yamaguchi, Masato Oguchi, “Performance and TCP parameter analysis during VPN multi-path access in iSCSI storage,” IEICE 18th Data Engineering Workshop DEWS 2007), Feb. 2007. “IEEE Std 802.3ad-2000”. Amendment to Carrier Sense Multiple Access with Collision Detection (CSMA / CD) Access Method and Physical Layer Specification-Aggregation of Migration. Sponsor: LAN MAM Standards Committee of the IEEE Computer Society. Applied 30 March 2000 IEEE-SA Standards Board “HP Serviceguard Management Linux (6th edition)” Manufacturing Part Number: B9903-90052. Aug. 2006, pp. 132-137 T.A. Ito, H.C. Ohsaki, and M.M. Imas, “GridFTP-APT: Automatic parallel tuning mechanism for data transfer protocol (GTP)” in Proceedings of 6th IEEE Criteria. 454-461, May. 2006.

  In an IP network having high delay characteristics, the problem of a decrease in iSCSI throughput is widely known, and it is also known that a system using TCP multiple connections defined in iSCSI improves the throughput.

  However, with iSCSI connections based on wide-area and wide-band IP networks, to improve throughput, parameters such as iSCSI / TCP are tuned in advance based on the bandwidth and delay information of the line used, or protocol conversion is performed between the server and storage. There is a risk that operation settings are complicated and economical operation is difficult, such as improving the throughput by installing a machine.

  In addition, the conventional improvement is based on fixed information, so it cannot follow the fluctuation of the bandwidth, it is difficult to transfer data efficiently, and there is a possibility that the best-effort type economical line cannot be used. .

  Therefore, the present inventors have proposed an invention for performing tuning for maximizing the number of TCP connections with respect to throughput by adding a function to the iSCSI initiator (Japanese Patent Application No. 2008-037692).

  When this method is used for a plurality of links between the same initiator and target, particularly in an environment in which each link is reserved in advance by bandwidth reservation, a case where the use start time for each link is different is assumed. Under such an environment, the number of TCP connections must be tuned for each link, and an efficient method for reducing the total tuning time is required.

  An object of the present invention is to search for the optimum number of TCP connections for the second and subsequent links at a high speed when determining the number of TCP connections that maximizes the throughput for a plurality of links in such an environment. .

  Although the iSCSI technology defines a function for establishing a plurality of TCP connections in one iSCSI session, issuing a plurality of SCSI commands in parallel, and simultaneously transferring data in each TCP connection, the number of TCP connections is It is not defined how to determine or to which connection a data read / write command from an upper layer is distributed to a plurality of established TCP connections.

  Therefore, the present invention uses the number of TCP connections tuned in the first link in a situation where a plurality of links can be used as the base point of the subsequent tuning, thereby optimizing the number of TCP connections for the second and subsequent links. Search time is shortened. That is, a TCP connection used for iSCSI connection when performing data transfer by iSCSI connection in a server and an IP storage device having a plurality of physical links having different paths and delay amounts, and a dedicated network interface accommodated for each physical link. The initial value of the multiplicity used for the network interface to be set next is managed using the multiplicity optimized for the network interface used for the link to be set in advance. And the multiplicity of TCP connections is optimized at high speed by using the throughput difference for each network interface.

  Here, an algorithm for managing to which of a plurality of TCP connections the read / write commands from the upper layer are arranged in the initiator. The built-in algorithm is the same as that of Japanese Patent Application No. 2008-37692, and in the case of a read command, the link-by-link throughput calculated at a predetermined time is calculated and recorded by referring to the reception buffer for each link. By repeating this process, the optimum value for the number of TCP connections is searched for the throughput through the specific link from the maximum throughput value in the history.

  When a plurality of links having different use start times are available, the TCP connection determination process determined above is used for the first link, and the first link is determined for the second and subsequent available links. By applying the number of TCP connections (the optimum value of Link1), the optimum value search time for the number of TCP connections for the second and subsequent links is shortened.

  According to the present invention, when determining the number of TCP connections that maximizes the throughput for a plurality of links, the optimum number of TCP connections for the second and subsequent links is searched at high speed.

FIG. 3 is a schematic diagram of a connection form that uses a plurality of TCP connections in one iSCSI connection (session) in the iSCSI protocol. The control block diagram for the number of TCP connections of iSCSi which comprises the basic principle of this invention. The control block diagram of the optimization process of the number of TCP connections which concerns on embodiment of this invention. (A) is a model figure which shows the use reservation situation of Link1 in FIG. 3, (b) is a model figure which shows the use reservation situation of Link2 in FIG. The adjustment principle figure of the number of TCP connections at the time of Link2 starting.

≪iSCSI connection form≫
FIG. 1 shows a basic form in which an iSCSI initiator 1 and an iSCSI target 2 are iSCSI-connected via an IP (Internet Protocol) network. Here, in the iSCSI protocol, a plurality of TCP connections are used in one iSCSI connection (iSCSI session).

  The iSCSI initiator 1 is mounted on a file server or the like constituting the iSCSI host device, while the iSCSI target 2 is mounted on a disk storage device or the like constituting the iSCSI device device. The function of the iSCSI initiator 1 is realized through a network interface such as an Ethernet (registered trademark) interface.

  The protocol defined in Non-Patent Document 1 defines a form in which a plurality of TCP connections are used. However, when an iSCSI command from an upper layer is transmitted through a specific TCP connection, the protocol is attributed to the command. It only specifies that "Reception of response from target" and "Response to response" are completed in the same connection.

  That is, it is not determined whether to change the number of TCP connections during data transmission / reception in an established iSCSI session, or to which of the established TCP connections an iSCSI (Read / Write) command is sent.

≪Basic control≫
FIG. 2 shows a basic control configuration S1 of an optimization method for the throughput of the number of TCP connections, and a distribution control unit (iSCSI-APT) 3 for the number of TCP connections is added to the iSCSI initiator 1. Here, the number of network interfaces, that is, the number of links is assumed to be one (Link1), and the optimization process similar to that of Japanese Patent Application No. 2008-37692 is performed. Hereinafter, the operation will be described with reference to FIG.

  First, when an iSCSI (Read) command is issued from an upper layer (read request command queue) in the file server as indicated by an arrow a in FIG. 2, the command is transferred to the iSCSI layer as indicated by an arrow b. It is sent to the connection distribution function unit 4 to add functions.

  The connection distribution function unit 4 distributes the iSCSI (Read) command received from the upper layer to the TCP connection. Here, first, an iSCSI (Read) command is transmitted through a specific TCP connection as indicated by an arrow c. Next, as shown by an arrow d, an iSCSI (Read) command is transmitted through another TCP connection.

  As a result of each iSCSI (Read) request, “Read Data” received from the iSCSI target 2 is stored in the reception buffer 5. The “Read Data” of the reception buffer 5 is converted into a throughput value. Here, the throughput value is calculated from the data amount of “Read Data” and the transfer time. As shown by an arrow e, the converted throughput value is notified as a TCP multiplicity control signal to the distribution control unit 3 incorporated in the iSCSI initiator 1.

  Then, the distribution control unit 3 applies the processing of the built-in algorithm to the history of the throughput values obtained by repeating the process described above. That is, the golden search method is applied to narrow down and a correction value (maximum value) of the TCP connection number (multiplicity number) that maximizes the throughput value is extracted. Here, the method of Non-Patent Document 8 is mainly used. In Non-Patent Document 8, the golden search method is used for another technique called GridFTP instead of iSCSI.

  The correction value extracted here is reflected in the processing of the connection distribution function unit 4 as indicated by an arrow f. By such processing, when the number of links used is one, the optimum value of the number of TCP connections is searched.

≪When using multiple links≫
FIG. 3 shows the control configuration S2 according to the embodiment of the present invention, and shows a method for optimizing the number of TCP connections when a plurality of links are available. Here, if multiple links can be reserved at different usage start times, the optimum number of TCP connections for the second and subsequent links is searched at a high speed when determining the number of TCP connections that maximizes the throughput for each link. To do.

  That is, since the control configuration S1 assumes the use of one link, in order to optimize the number of TCP connections for each of a plurality of links having different usage start times, 20 to 20 for each link. A convergence time of about 40 seconds is required.

  At that time, if the link is between the same initiator and the target, the probability that the delay amount is greatly different is often small, and if the bandwidth difference between each link becomes large, the number of previously established TCP connections becomes a new link. Probably close to the number of TCP connections.

  Based on this concept, the number of TCP connections initially tuned to improve the control configuration S1 and reduce the optimum value search time for the number of TCP connections for the second and subsequent links is used as a starting point for the subsequent tuning. Use to compress search time. As a result, the total data transfer amount can be increased or the data transfer time can be shortened.

Specifically, the control configuration S2 incorporates a monitoring control unit 6 that detects activation of Link2 (second network interface). The monitoring control unit 6 establishes an iSCSI connection by using the number N of TCP connections operated on Link1 (first network interface) as an initial value of the number of TCP connections on Link2. Hereinafter, an overall processing process of the control configuration S2 will be described. Here, it is assumed that Link2 is added after the optimization process of the number of TCP connections for Link1 is completed, and the use start times of each Link1.2 are different.
(1) Optimization of Link 1 First, when Link 1 is activated at the use start time as indicated by an arrow A, the activation of Link 1 is detected and an iSCSI / TCP connection is established. As the number of TCP connections at that time, the maximum value in the negotiation between the iSCSI initiator 1 and the iSCSI target is selected as the initial value. However, the actual connection usage is not always operated at the maximum value.

  When an iSCSI (Read) command issued from an upper layer (read request queue) in the file server is issued to the connection distribution function unit 4 as indicated by an arrow B, the distribution function unit 4 As a result, the iSCSI (Read) command is distributed to the TCP connection. In general, there are many cases where distribution is performed equally in a round robin method regardless of which link is used. The iSCSI (Read) command distributed here is transmitted through a specific TCP connection as indicated by an arrow C. Also, an iSCSI (Read) command is transmitted through another TCP connection as indicated by an arrow D.

  As a result of each iSCSI (Read) request, “Read Data” stored in the reception buffer 5a is converted into a throughput value and notified to the distribution control unit 3 as a TCP multiplicity control signal as indicated by an arrow E.

  The distribution control unit 3 performs a built-in algorithm process on the throughput history. In other words, a correction value (maximum value) of the number of TCP connections is searched from the history of throughput values, and reflected in the processing of the connection distribution function unit 4 as indicated by an arrow F (limits the connection to which the “iSCSI Read” command is sent) To do).

As a result, the operation is performed with the number N of TCP connections in which the iSCSI throughput through Link 1 is optimized. Hereafter, the operation is continued without correcting the number of TCP connections for Link1.
(2) Optimization of Link 2 Next, when Link 2 is activated at the use start time as indicated by an arrow G, the monitoring control unit 6 detects the activation of Link 1 as indicated by an arrow H. At this time, the monitoring control unit 6 monitors the number N of TCP connections operated on the Link 1 and operates on the Link 1 as indicated by an arrow I as an initial number of TCP connections for establishing an iSCSI / TCP connection through the Link 2. The number N of TCP connections being used is used.

  Accordingly, in Link2, an iSCSI / TCP connection is established with the number of N TCP connections. As a result, an iSCSI (Read) command is transmitted for each TCP connection, and “Read Data” received through Link 2 is accumulated in the reception buffer 5 b. The “Read Data” stored in the reception buffer 5b is converted into a throughput value and, as indicated by an arrow J, notified to the distribution control unit 3 as a TCP multiplicity signal.

  The distribution control unit 3 determines whether the bandwidth of the link 2 is larger or smaller than the link 1 based on the amount of fluctuation (throughput) of the data accumulation of each of the links 1 and 2, and determines the optimum TCP from the history of the throughput of the link 2. After searching for the number of connections (execution of the built-in algorithm), as indicated by an arrow K, it is reflected in the distribution process of the connection distribution control unit 3.

  That is, the connection distribution function unit 4 determines whether the arrow L.P is set according to the number N ′ of TCP connections searched by the distribution control unit 3. As shown by M, an iSCSI (Read) command is transmitted. As a result, the “Read Data” received through the Link 2 under the TCP connection number N ′ is accumulated in the reception buffer 5 b. “Read Data” stored in the reception buffer 5b is converted into a throughput value, and as shown by an arrow O, the distribution control unit 3 is notified again as a TCP multiplicity signal.

Then, the distribution control unit 3 corrects the number of TCP connections again from the history of the throughput value of Link 2 (re-executes the processing of the built-in algorithm), and the distribution processing of the connection distribution function unit 4 as indicated by the arrow P To reflect. As a result, the number of TCP connections (multiplicity) N ″ for optimizing the throughput value through Link 2 is searched, and thereafter, the operation is continued with the number of TCP connections.
≪Adjustment principle when starting Link2≫
4 (a) shows the usage reservation status of Link1 in FIG. 3, FIG. 4 (b) shows the usage reservation status of Link2 in FIG. 3, the usage start time of Link1 is indicated as t1, The use start time is indicated as t2. Here, since “t1 <t2” is established, use of Link1 is started earlier than Link2.

  Also, according to FIGS. 4 (a) and 4 (b), since Link 1.2 indicates that the reserved bandwidth is the same (bw1), TCP optimized by Link 1 (throughput is maximized) The number of connections N is estimated to be close to the optimum value of the number of TCP connections of Link2.

  At this time, the control configuration S2 uses the number N of TCP connections in the link 1 as the initial value, that is, the starting point for adjusting the number of TCP connections, so that the number of TCP connections in the control configuration S1 is different for each link in the control configuration S1. Convergence time (about 20 to 40 seconds) is shortened.

  FIG. 5 shows the principle of adjusting the initial value when Link 2 is activated. Here, under the reservation status shown in FIGS. 4 (a) and 4 (b), throughput fluctuation from point A → A ′ or point B → B ′ is shown depending on whether the delay amount of Link2 is larger or smaller than Link1. ing. The execution bandwidth (Bandwidth) of data transfer through Link 2 can be determined based on the throughput fluctuation, and the amount of fluctuation of the arrived data can be determined as compared with the operation time of Link 1, and the number of TCP connections is adjusted. That is, in order to adjust the number of TCP connections to the optimum value, a search should be performed from N1 to N2 when the point A changes to the point A ′, and N2 when the point B changes to the point B ′. → A search may be performed on N1.

  If there is no significant difference in the reserved bandwidth for multiple links reserved for use in this way, the time required to determine the number of TCP connections that maximizes throughput for each link can be reduced, and data transfer can be made more efficient. It becomes.

  The present invention is not limited to the above-described embodiment. For example, the same effect can be obtained not only for an iSCSI (Read) command but also for an iSCSI (Write) command. The present invention can also be configured as a program that causes a computer to function as the control configuration S2 including the distribution control unit 3 and the monitoring control unit 6. In this case, the program is read by a processing unit (for example, CPU) of the computer, and each operation process in FIG. 3 is executed.

  This program can be provided through a network such as a website or e-mail. The program is recorded on a recording medium such as a CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, MO, HDD, Blu-ray Disk (registered trademark). It is also possible to save and distribute. This recording medium is read using a recording medium driving device, and the program code itself realizes the processing of the above embodiment, so that the recording medium also constitutes the present invention.

  Currently, there is an increasing demand for large-capacity data backup for disaster countermeasures mainly by companies, and there is a great expectation for realizing a simple and economical system. The optimization method for the throughput of the number of TCP connections according to the present invention is an Internet standard, and is an improved technique for the iSCSI technique in which commercially available storage apparatuses have already started to spread.

  In particular, it is assumed that bandwidth-reservable network services that are expected to make progress in the future will be used. Therefore, network operation efficiency is improved by automating operations (parameter settings) when transferring high-speed and large-capacity data through cooperation between the two, and shortening the transfer time. It is effective in improving

DESCRIPTION OF SYMBOLS 1 ... iSCSI initiator 2 ... iSCSI target 3 ... Distribution control part 4 ... Connection distribution function part 5.5a. 5b ... Reception buffer 6 ... Monitoring control unit

Claims (5)

A method of controlling the number of TCP connections for each connection link between an iSCSI host device and an iSCSI device device having an iSCSI session with an initiator constructed in the host device,
A first step of searching for an optimum value of the number of TCP connections that maximizes throughput with respect to a previously activated link when the use start times of the connection links are different;
Apply the optimal value of the previously activated link searched in the first step to the number of TCP connections of the activated link, and maximize the throughput for the activated link based on the throughput difference between the two links. A second step of adjusting the number of TCP connections to
A method for controlling the number of TCP connections in an iSCSI connection. Monitoring control means for monitoring the activation of each of the connection links and detecting the link to be activated next;
Establishing an iSCSI connection with the optimum value as an initial value for a link to be activated next;
The method for controlling the number of TCP connections in the iSCSI connection according to claim 1, further comprising: In the iSCSI host device in which an initiator that performs an iSCSI session with the iSCSI device is constructed,
Means for searching the initiator for the optimum value of the number of TCP connections that maximizes the throughput for the link that was activated first when the use start time of each link is different;
Apply the optimal value of the link activated first to the number of TCP connections of the next activated link, and adjust the number of TCP connections to maximize the throughput for the next activated link based on the throughput difference between the two links Means to
An iSCSI host device characterized by comprising: It further comprises monitoring control means for monitoring activation of each link and detecting a link to be activated next,
The iSCSI host device according to claim 3, wherein the monitoring control means establishes an iSCSI connection with the optimum value as an initial value for a link to be activated next.   A program for causing a computer to function as the iSCSI host device according to claim 1.

Images