CN116360675A

CN116360675A - SAS frame routing method and device in wide port scene

Info

Publication number: CN116360675A
Application number: CN202211511468.8A
Authority: CN
Inventors: 李越峰; 宫晓渊; 王剑铎; 刁永翔; 汪宏志; 刘欢
Original assignee: Wuxi Zhongxing Microsystem Technology Co ltd
Current assignee: Wuxi Zhongxing Microsystem Technology Co ltd
Priority date: 2022-11-29
Filing date: 2022-11-29
Publication date: 2023-06-30
Anticipated expiration: 2042-11-29
Also published as: CN116360675B

Abstract

The invention provides a SAS frame routing method and device in a wide port scene, wherein the method comprises the following steps: configuring a Phy selection field in the SAS frame head, wherein the Phy selection field is used for designating a channel Phy between the SAS HBA and the SAS expander; receiving an SAS frame, and analyzing a Phy selection domain segment in the SAS frame to obtain a designated channel Phy; sending a link establishment request with the target SAS disk device through the SAS expander; I/O transmission between the SAS HBA and the target SAS disk device is initiated through a designated channel Phy. The proposal of the invention establishes the connection between the designated Phy and the SAS disk device by analyzing the Phy selection field section in the frame header, thereby avoiding the waste of physical link resources and realizing the I/O order-preserving management.

Description

SAS frame routing method and device in wide port scene

Technical Field

The invention belongs to the technical field of disk connection, and particularly relates to a SAS frame routing method and device in a wide port scene.

Background

SAS (serial attached SCSI) is a new generation of SCSI technology, belongs to a point-to-point full duplex serial protocol, has built-in data and command checking units, has strong error correction capability, supports hot plug, has the characteristics of small pin count, fast data transmission rate, high reliability, good compatibility, and the like, and is widely used in the industry for a main I/O serial interface between a storage device and a host at present. Meanwhile, the SAS design considers the downward compatible SATA technology, achieves interconnection and data transmission between the SAS controller and the SATA device through an STP protocol (SATA channel protocol), and has good expansibility and compatibility. In order to support higher transmission efficiency, SAS standard protocols introduce a concept of wide ports, and bandwidth performance can be improved by combining multiple links into one wide port between SAS devices. One port supports one or more physical layers Phy, each individual physical layer Phy being referred to as a physical channel (lane), a plurality of phys having the same physical address making up a wide port.

In a typical SAS data storage topology, SAS controllers support large-scale disk management through Expander (disk Expander) expansion. The wide port provides a plurality of channels (Phy) to access the external magnetic disk, so that the utilization rate of multiple channels is improved, and the transmission rate is faster. A plurality of disks can be connected with the Expander to form an array.

FIG. 1 illustrates a typical wide port array architecture. The host bus adapter HBA serves as an integrated circuit adapter that provides input/output (I/O) processing and physical connections between the server and the storage device, and may be in the form of a circuit board card or an integrated circuit chip. The HBA reduces the burden of a main processor in data storage and retrieval tasks, can improve the performance of a server, and is widely applied to storage systems.

According to SAS standards protocols, a connection must be successfully established between two ends each time before a command or data is transceived. Connections are frequently established and closed during transmission, so it is important to optimize connection management reasonably. For example, in the wide port array structure of fig. 1, a wide port consisting of physical channels Phy0 to Phy7 is shown, which allows access to a maximum of 8 disks at the same time, and the SAS HBA may perform data interaction with SAS disk #0 to SAS disk #7 after expansion by an Expander. When the SAS HBA and the SAS device conduct data transmission, whether Phy is idle or not can be inquired. When a certain Phy needs to be selected to establish connection, after receiving an application layer sending command or a data sending command, the SAS controller transmission layer requests to establish connection to the SAS controller port layer. The SAS controller port layer queries whether Phy of the wide port is idle, sends a request to establish a connection to the peer SAS device, and after sending an OPEN address frame OPAF and receiving an OPEN ACCEPT primitive, indicates that a connection was successfully established, may dispatch an I/O to the connection. For example, phy0 is selected to serve I/O#1 for data transfer with SAS disk device #0, or Phy1 is selected to serve I/O#2 for data transfer with SAS disk device # 1.

However, when both I/O#1 and I/O#2 need to be dispatched to the same SAS disk device #0, I/O#1 and I/O#2 cannot be dispatched at the same time, but must wait for the current I/O dispatch to complete before other I/Os can be dispatched. In the prior art, if a plurality of I/os are distributed to the same SAS disk device end, the Phy with failed connection establishment is occupied and cannot be used by other devices, thus wasting channel resources.

Taking the wide-port storage structure shown in fig. 2 as an example, if the I/o#1 is already dispatched to Phy0, when the I/o#2 needs to be dispatched, an idle Phy can be selected from Phy1 to Phy7, if Phy1 is selected at this time, since the device #0 is already occupied by Phy0, phy1 cannot establish a connection, waiting for timeout always, interrupting the link, and continuing to select other phys for dispatch, which is equivalent to wasting channel resources. In addition, the prior art cannot perform order preserving management on multiple I/O arrival orders. If the I/o#1 and the I/o#2 are respectively dispatched to Phy0 and Phy1, if Phy1 establishes a connection prior to Phy0, the I/o#2 may be dispatched first, and the I/o#1 may be dispatched later, so that the sequential relationship between the I/o#1 and the I/o#2 when they arrive at the device cannot be ensured, and if the SAS disk device #0 has a requirement on the I/O receiving sequence, a data receiving error may occur in this case.

Disclosure of Invention

Aiming at a storage structure of a wide port, the invention aims to provide a SAS frame routing method for solving the problems of reasonable utilization of SAS HBA and SAS Expander multichannel resources and I/O order-preserving management.

According to a first aspect of the present invention, there is provided a SAS frame routing method in a wide port scene, comprising:

configuring a Phy selection field in the SAS frame head, wherein the Phy selection field is used for designating a channel Phy between the SAS HBA and the SAS expander;

receiving the SAS frame, and analyzing a Phy selection domain segment in the SAS frame to obtain a designated channel Phy; sending a link establishment request with the target SAS disk device through the SAS expander;

and initiating I/O transmission between the SAS HBA and the target SAS disk device through the designated channel Phy.

Preferably, the Phy select field is a 5bit wide field selected from unused reserved field in the SAS frame header, the Phy select field configured by a software application layer.

Preferably, the sending, by the SAS expander, a link establishment request with the target SAS disk device further includes:

and finishing framing operation of the SAS frame by a transmission layer, requesting to a port layer to establish connection with the target SAS disk device based on the Phy selection domain segment obtained by analysis, and sending the SAS frame to the port layer.

Preferably, the parsing the Phy selection field in the SAS frame to obtain a designated channel Phy further includes:

judging the value of a preset zone bit in the Phy selection field section;

if the preset flag bit is 1, determining the designated channel Phy according to the mapping relation between the preset Phy selection field segment and the channel Phy;

if the preset flag bit is 0, an idle channel Phy is selected based on polling all channels Phy.

when the SAS expander comprises a multi-stage SAS expander, determining the designated channel Phy by performing a modulo-N operation on the value of the Phy selection field, where N is the Phy number of each stage expander.

According to a second aspect of the present invention, there is provided a SAS frame routing device in a wide port scene comprising:

a configuration unit, configured to configure a Phy selection field in a SAS frame header, where the Phy selection field is used to specify a channel Phy between a SAS HBA and a SAS expander;

the chain building unit is used for receiving the SAS frame and analyzing the Phy selection domain segment in the SAS frame to obtain a designated channel Phy; sending a link establishment request with the target SAS disk device through the SAS expander;

and the transmission unit is used for initiating I/O transmission between the SAS HBA and the target SAS disk device through the designated channel Phy.

Preferably, the chain building unit is further configured to:

Preferably, the transmission unit is further configured to:

judging the value of a preset zone bit in the Phy selection field section;

Preferably, the transmission unit is further configured to:

Compared with the prior art, the SAS frame routing method and device in the wide port scene are used for establishing connection between the designated Phy and the SAS disk device by analyzing the frame header Phy_SEL domain segment in different modes of SMP/SSP/STP on the basis of the connection management rule defined by compatible SAS standard protocol, so that I/O read-write operation is completed, physical link resource waste caused by competition of a plurality of Phys in the wide port for the same target SAS device is effectively avoided, I/O order-preserving management is realized, and the utilization efficiency of Phy channels in the wide port scene can be remarkably improved.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent to those having ordinary skill in the art from the description or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will briefly explain the embodiments of the present invention or the drawings needed in the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 shows a schematic diagram of a typical wide port array structure according to the prior art.

Fig. 2 shows a schematic diagram of a multi-channel I/O connection establishment procedure in a wide port scenario according to the prior art.

Fig. 3 shows a frame header format schematic after configuring Phy selection field in SMP mode according to the present invention.

Fig. 4 shows a frame header format diagram after configuring Phy selection field in SSP mode according to the present invention.

Fig. 5 shows a frame header format schematic after configuring Phy selection field segments in STP mode according to the present invention.

Fig. 6 shows a schematic view of an OPAF frame after configuring a Phy selection field in a multi-level expander scene according to the present invention.

Fig. 7 shows a schematic diagram of the mapping relationship between Phy selection field values and the actual selected Phy according to the present invention.

Fig. 8 illustrates a flowchart of a SAS frame routing method and apparatus in a wide port scenario in accordance with the present invention.

Fig. 9 shows a flow chart of a connection management procedure in a wide port scenario according to the present invention.

Fig. 10 illustrates a read I/O operation timing diagram in SSP mode for the SAS frame routing method in accordance with the present invention.

Fig. 11 illustrates an array structure diagram of a SAS frame routing method in a multi-level expander scenario in accordance with the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which are derived by a person skilled in the art from the embodiments according to the invention without creative efforts, fall within the protection scope of the invention.

In order to solve the problems, the invention provides a SAS frame routing method and device under a wide port scene aiming at a storage structure of a wide port, and aims at the defects of port routing under a wide port mode in an SAS standard protocol, and the problems of channel resource waste and incapability of carrying out order-preserving dispatch on I/O are solved by optimizing a configuration mode of a software application layer on a SAS frame head through designating an I/O dispatch path.

Because SAS modes include three modes, namely SMP (Serial Management Protocol ), SSP (Serial SCSI Protocol, serial SCSI protocol), STP (Serial ATA Tunneled Protocol, serial ATA pipeline protocol), in the three SAS modes, the invention selects a field segment with a bit width from reserved field segments not used in a frame header specified by the SAS protocol, and the field segment is named phy_sel as a selection field segment of Phy, and is used for specifying a specific Phy in a plurality of Phy channels with wide ports. In one embodiment, the Phy_SEL bit width is 5 bits, the value of which can be configured in software, for example, to identify the number specifying Phy. When the SAS controller port layer performs Phy dispatch, the Phy selection domain segment is analyzed, so that the corresponding Phy is designated to perform I/O dispatch. In this way, the SAS HBA may transmit with the SAS device over the designated Phy channel of the SAS expander.

If a multi-stage expander exists, when the number of the expanding wide ports Phy and the number of the expanding wide ports Phy are not matched, the configured Phy_SEL value can be subjected to modulo N operation, wherein N is the Phy number of each stage expander so as to further realize the selection of Phy, achieve the purpose of designating Phy and realize the route management in the multi-stage expander.

For example, FIG. 3 shows an example of a modified REQUEST frame header format in SMP mode, where the lower 5 bits of a byte may be selected as the Phy selection field segment Phy_SEL [4:0] in Additional REQUEST bytes.

FIG. 4 shows an example of a modified transport layer frame header format in SSP mode, where the 8 th byte low order 5 bits may be selected as the Phy selection field segment Phy_SEL [4:0].

Fig. 5 shows an example of a modified Register Host to Device FIS in STP mode. Since PM Port is only used in SATA mode and not used in STP mode, PM Port field and one R field can be used to make up Phy_SEL [4:0].

In addition, for an application scenario where multiple levels of expanders exist, a phy_sel field in the OPAF frame needs to be further defined. The phy_sel field in the OPAF frame may use the lower 5 bits of the COMPATIBLE FEATURES field in 25 bytes as shown in fig. 6 for the expander designation Phy.

FIG. 7 illustrates an exemplary mapping of Phy_SEL field values and selected Phy. When Phy_SEL is configured as 5' h0x (independent of other bit fields if the most significant bit field is 0), this indicates that Phy need not be specified; when Phy_SEL is configured to be 5' h10, it indicates that channel Phy0 is selected. Phy may be selected in a round robin fashion. In fig. 7, 16 Phy cases are listed, and those skilled in the art will understand that the mapping relationship may be modified and extended according to actual requirements. In the two-stage expander mode, phy_SEL field values can be used for carrying out the operation of the mode N according to the port width between the expanders, namely Phy channel number N, so as to ensure the fixed routing relationship.

Fig. 8 shows a flowchart of a SAS frame routing method in a wide port scenario in accordance with the present invention. Specifically, the SAS frame routing method in the wide port scene according to the present invention includes:

step S1, a Phy selection field is configured at the head of the SAS frame, wherein the Phy selection field is used for designating a channel Phy between the SAS HBA and the SAS expander.

In connection with the specific wide port connection management flow proposed by the present invention shown in fig. 9, in step 101, the phy_sel field may be configured by the SAS controller software application layer, to designate a channel Phy between the SAS HBA and the SAS expander to be subjected to I/O transmission for the target SAS disk device.

S2, receiving the SAS frame, and analyzing a Phy selection domain segment in the SAS frame to obtain a designated channel Phy; and sending a link establishment request with the target SAS disk device through the SAS expander.

Referring to step 102 of fig. 9, the SAS controller transmission layer completes framing operation of SAS frames, sends the SAS frames to the SAS controller port layer, and requests the SAS controller port layer to establish connection with the target SAS disk device based on the Phy selection field obtained by parsing.

And S3, initiating I/O transmission between the SAS HBA and the target SAS disk device through the designated channel Phy.

Referring to step 103 of fig. 9, when the SAS controller port layer receives a chain establishment request from the SAS controller transport layer, the Phy channel may be selected by initiating an arbitration mechanism. At this time, the value of Phy_SEL needs to be judged, and the designated Phy can be selected according to the value of Phy_SEL to open the link, and the I/O read-write operation is started.

In a preferred embodiment of the present invention, phy_SEL may indicate whether a specific Phy needs to be specified by a preset flag bit. As previously described, the highest bit Phy_SEL [4] may be used as a preset flag bit according to the Phy_SEL field value and the selected Phy mapping table illustrated in FIG. 7. Specifically, in step 103 of FIG. 9, it may be determined whether the preset flag Phy_SEL [4] is 1.

If Phy_SEL [4] is 1, that is, in step 104 of FIG. 9, a specific Phy needs to be specified, a corresponding Phy may be selected according to the Phy_SEL [3:0] value and the predetermined mapping relationship, connection is established, and I/O reading and writing is completed.

Alternatively, if Phy_SEL [4] is 0, indicating that Phy is not specified in the SAS frame, in step 105 of FIG. 9, each Phy may be polled in a conventional routing manner, and an idle Phy is selected to establish a connection to complete the I/O read/write operation.

Finally, in step 106, when the I/O data read/write is completed, the SAS expander and the SAS device may perform a DONE primitive handshake, CLOSE the connection with the SAS device by means of CLOSE or BREAK, and then release the Phy channel. The method ends.

According to the embodiment of the invention, the selected channel Phy is determined by analyzing the Phy_SEL field in the SAS frame, so that the SAS frame route channel from the SAS Expander to the SAS disk can be definitely designated, the channel is ensured to be fixed when the SAS HBA accesses the SAS device, the competition and the deadlock which occur when the Phy is automatically selected can be avoided, and the I/O sequence received by the SAS device is ensured to be consistent with the data transmission sequence.

To further clarify the technical solution of the present invention, the following describes an exemplary whole process of performing I/O transmission in connection with a read I/O operation in SSP mode.

Assuming that there are two I/Os I/O #1 and I/O #2 to be dispatched, and Phy_SEL [4:0] is preset to be 5' h10, according to the mapping table of FIG. 7, the speech segment indicates that Phy0 is selected to dispatch I/O #1 and I/O #2, and the chain transmission process is shown in the timing chart 10.

Event T1: the SAS controller port layer analyzes Phy_SEL, determines to select to send an OPAF frame to the SAS disk device #0 through a Phy0 channel, and the opposite-end SAS device returns an OPEN_ACCEPT primitive after receiving the OPEN_ACCEPT primitive, namely the connection is successfully established.

Event T2: the SAS HBA sends SSP read command frames for I/O #1 to SAS disk device #0 over the Phy0 connection.

Event T3: SAS disk device #0 data is ready and returns the SSP data frame sequence of I/O #1 until the SAS HBA receives all the data frame sequence.

Event T4: after returning all data, the SAS disk device #0 returns a response frame, notifies the SAS HBA of the end state of data transmission, and then completes the read operation of the I/O # 1.

Event T5: after I/O#1 is completed, SAS HBA dispatches I/O#2, determines that the target channel is still Phy0 by parsing Phy_SEL field segment, at this time, I/O#2 dispatch can be continued using the current connection without contention of Phy channel. The I/O#2SSP read command then continues to be dispatched.

Event T6: SAS disk device #0 returns all data of I/O #2, similar to event T3, until SAS HBA completes receipt of all read data.

Event T7: after the SAS disk device #0 has transmitted the I/O #2 data, replies with a response frame, notifies the end state of SAS HBA data transmission similarly to event T4, and then completes the read operation of I/O #2.

Event T8: and under the condition that the SAS HBA does not have data to be transmitted, firstly transmitting a DONE primitive to the SAS disk device #0, and returning the DONE primitive to the SAS HBA after the SAS disk device #0 transmits the data, so as to complete the DONE handshake operation.

Event T9: after the DONE primitive handshake is completed, both SAS HBA and SAS disk device #0 send CLOSE primitive to the opposite end, and after the CLOSE handshake is completed, the link is normally closed.

Fig. 11 shows the application of the method of the present invention in a secondary expander scenario. Referring to fig. 11, SAS HBA and SAS expander_1 are transmitted through x8 wide links, and SAS expander_1 and SAS expander_1_1 are transmitted through x4 wide links. The number of ingress and egress ports of SAS expander_1 does not match. Thus, in x8 wide links, SAS HBAs may route by modulo-8 computing a preconfigured phy_sel value followed by Phy selection. In the x4 wide link, SAS expander_1 may select Phy for routing by modulo-4 operation on the phy_sel value.

For example, when phy_sel [4:0] is 5'h15 and SAS disk #2 needs to be accessed, the input channels of SAS expander_1 are 8, and the modulo-8 operation is performed on 5' h15 to obtain 5, which corresponds to expander_1ingress Phy 5. However, the number of output channels is 4, that is, the number of input channels of the next SAS expander_1_1 is 4, so that the modulus 4 operation of 5' h15 can be performed to obtain 1, which corresponds to expander_1egress Phy 1. The final I/O route from SAS HBA to SAS disk #2 is:

SAS HBA Phy

SAS Expander_1ingress Phy/>

SAS Expander_1egress Phy

SAS Expander_1_1ingress Phy/>

SAS Expander_1_1egress Phy/>

SAS disk #2.

In this way, by analyzing the preconfigured phy_sel field segment by each stage of Expander to determine the selected Phy, the routing channel from each stage of Expander to the SAS disk can be specified explicitly, and the channel is ensured to be fixed when the SAS HBA accesses the SAS device, so that competition and deadlock occurring when Phy is selected in the prior art can be avoided, and the I/O sequence entering the SAS device can be ensured.

According to the embodiment, the SAS frame routing method in the wide port scene is simple, clear and easy to realize in principle, and on the basis of being compatible with the connection management rules defined by the SAS standard protocol, the designated Phy and the SAS disk device are connected by analyzing the frame header Phy_SEL domain segment in the SMP/SSP/STP mode, so that I/O read-write is completed, the physical link resource waste caused by competition of a plurality of Phys in the wide port for the same target SAS device is effectively avoided, and the utilization efficiency of the Phy channel can be remarkably improved. Meanwhile, the method of the invention also realizes the I/O order-preserving management, and if the same Phy is bound with the I/O of the SAS device, the order of the data reaching the device side can be ensured, which is important for the condition of requiring the I/O receiving order.

Accordingly, the present invention provides, in a second aspect, a SAS frame routing device in a wide port scene comprising:

a configuration unit 201, configured to configure a Phy selection field in the SAS frame header, where the Phy selection field is used to specify a channel Phy between the SAS HBA and the SAS expander;

the link establishment unit 202 is configured to receive the SAS frame, and parse the Phy selection domain segment in the SAS frame to obtain a designated channel Phy; sending a link establishment request with the target SAS disk device through the SAS expander;

and the transmission unit 203 is configured to initiate an I/O transmission between the SAS HBA and the target SAS disk device through the designated channel Phy.

The flowcharts or block diagrams shown above are only for illustrating the technical solution of the present invention, and are not limited to the embodiments. It will be appreciated by those skilled in the art that any modifications may be made to the above-described method flow details, etc. as may be readily appreciated by those skilled in the art based on the present invention as desired, without limiting the invention to the specific structures or parameters of the examples described above.

While the invention has been described in detail with reference to the foregoing embodiments, it will be appreciated by those skilled in the art that variations may be made in the techniques described in the foregoing embodiments, or equivalents may be substituted for elements thereof; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A SAS frame routing method in a wide port scenario for implementing I/O transmissions between a SAS HBA and a target SAS disk device, comprising:

2. The method of claim 1, wherein the Phy selection field is a 5bit wide field selected from among reserved fields that are not used in the SAS frame header, the Phy selection field configured by a software application layer.

3. The method of SAS frame routing in a wide port scenario of claim 1 wherein the sending a link establishment request with the target SAS disk device further comprises:

4. The method of claim 1, wherein said parsing Phy select field segments in the SAS frame to obtain a designated channel Phy, further comprises:

judging the value of a preset zone bit in the Phy selection field section;

5. The method of claim 1, wherein said parsing Phy select field segments in the SAS frame to obtain a designated channel Phy, further comprises:

when the SAS expander comprises a multi-stage expander, determining the designated channel Phy by performing a modulo-N operation on the value of the Phy selection field, where N is the Phy number of each stage expander.

6. A SAS frame routing apparatus in a wide port scenario comprising:

7. The SAS frame routing device of claim 6 wherein the Phy select field is a 5bit wide field selected from unused reserved field in the SAS frame header, the Phy select field configured by a software application layer.

8. The SAS frame routing device of claim 6 wherein the chaining unit is further configured to:

9. The SAS frame routing device of claim 6 wherein the transmission unit is further configured to:

judging the value of a preset zone bit in the Phy selection field section;

10. The SAS frame routing device of claim 6 wherein the transmission unit is further configured to: