WO2017201742A1 - 存储系统及设备扫描方法 - Google Patents

存储系统及设备扫描方法 Download PDF

Info

Publication number
WO2017201742A1
WO2017201742A1 PCT/CN2016/083701 CN2016083701W WO2017201742A1 WO 2017201742 A1 WO2017201742 A1 WO 2017201742A1 CN 2016083701 W CN2016083701 W CN 2016083701W WO 2017201742 A1 WO2017201742 A1 WO 2017201742A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing unit
scanned
current processing
storage
address
Prior art date
Application number
PCT/CN2016/083701
Other languages
English (en)
French (fr)
Inventor
尹泽生
陈明
付晓飞
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP16890129.6A priority Critical patent/EP3291096B1/en
Priority to CN201680003259.9A priority patent/CN107851058B/zh
Priority to PCT/CN2016/083701 priority patent/WO2017201742A1/zh
Priority to US15/797,687 priority patent/US10437473B2/en
Publication of WO2017201742A1 publication Critical patent/WO2017201742A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/40—Bus structure
    • G06F13/4004—Coupling between buses
    • G06F13/4022—Coupling between buses using switching circuits, e.g. switching matrix, connection or expansion network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/40—Bus structure
    • G06F13/4004—Coupling between buses
    • G06F13/4027—Coupling between buses using bus bridges
    • G06F13/404—Coupling between buses using bus bridges with address mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0626—Reducing size or complexity of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0635—Configuration or reconfiguration of storage systems by changing the path, e.g. traffic rerouting, path reconfiguration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/54—Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026—PCI express

Definitions

  • Embodiments of the present invention relate to the field of storage, and more particularly, to a storage system and a device scanning method.
  • a storage system in the prior art may include a plurality of storage devices, and the plurality of storage devices may be interconnected in pairs.
  • the storage system 100 includes N storage devices 110, and there is a direct link between any two of the N storage devices 110, and can be directly connected between the two.
  • the links communicate with each other. Therefore, there are N ⁇ (N ⁇ 1)/2 direct links between the N storage devices in the storage system 100. If the number of storage devices included in the storage system is large, that is, the value of N is large, the number of connection links of the storage system is large, and the link structure is complicated.
  • Embodiments of the present invention provide a storage system and a device scanning method, which can reduce system complexity and construction cost.
  • the storage system is specifically a PCIe storage system.
  • the at least two uplink ports may be specifically PCIe uplink ports
  • the at least two downlink ports may be specifically PCIe downlink ports.
  • the storage system provided by the embodiment of the present invention forms a ring connection structure by connecting N storage devices end to end, so that any two storage devices of the N storage devices can communicate with each other, and the related art Compared with the direct connection of the storage devices, the complexity and construction cost of the physical link of the system can be reduced.
  • a connection link exists between any one of the at least two downlink ports and any one of the at least two uplink ports.
  • the at least two downlink ports are shared downlink ports, and correspondingly, at least one storage unit in the storage device is a shared storage unit, that is, can be shared by all processing units in the storage device.
  • the storage unit can be shared by all processing units in the system.
  • the processing unit can directly perform read and write operations on any storage unit in the system, and does not need to interact with other processing units, thereby simplifying the data read and write process of the processing unit, saving processing resources of the processing unit, and speeding up data reading and writing efficiency.
  • the processing unit can uniformly manage the resources of each storage unit in the storage system, which is beneficial to improving the overall performance of the system.
  • connection link exists between the at least two uplink ports.
  • the processing unit is configured to: scan a device that the processing unit can access; record between the devices scanned by the processing unit during the scanning process A connection relationship is generated, and a PCIe tree of the processing unit is generated according to the recorded connection relationship.
  • the processing unit when scanning the accessible device, is specifically configured to: scan the device connected to the processing unit according to the depth-first search algorithm DFS .
  • the processing unit is specifically configured to: determine whether the scanned device has been scanned by the processing unit; if it is determined that the scanned device has been scanned by the processing unit, return to the uplink connection with the scanned device. The device is scanned.
  • the device connected to the uplink end of the scanned device may be specifically a device connected to the scanned device in the uplink direction.
  • the processing unit may return to the upstream device of the device to continue scanning.
  • the processing unit is specifically configured to: if it is determined that the scanned device has been scanned by the processing unit Then, it is determined that the scanned device is an exchange unit directly connected to the processing unit.
  • the processing unit is specifically configured to: if it is determined that the scanned device has been scanned by the processing unit, return to the upper switching unit connected to the uplink port of the scanned device, and scan the connection with the upper switching unit. The device being scanned.
  • the processing unit is further configured to: after the device connected to the upper-level switching unit is scanned, returning to the upper-level switching unit connected to the uplink port of the upper-level switching unit, scanning the unscanned device until the processing All devices connected to all switching units connected to the unit are scanned.
  • the processing unit may allocate the device address to the device and scan the device connected to the device in the downlink direction.
  • the processing unit may further add a device address allocated for the device to the allocated address set of the processing unit.
  • the uplink direction may be specifically a direction close to the processing unit, and the downlink direction may be specifically a direction away from the processing unit.
  • the processing unit is specifically configured to: determine whether the scanned device is assigned a device address; if it is determined that the scanned device is allocated a device address, determining whether the device address to which the scanned device is assigned exists in a device address set corresponding to the processing unit, where the device address set corresponding to the processing unit includes each of the at least one device that the processing unit has scanned The device address to which the device is assigned; if the device address to which the scanned device is assigned exists in the device address set corresponding to the processing unit, it is determined that the scanned device has been scanned by the processing unit.
  • the processing unit is further configured to: if it is determined that the scanned device is not assigned a device address, allocate a device address for the scanned device, And adding the allocated device address to the device address set corresponding to the processing unit.
  • the processing unit may continue to scan for devices connected to the scanned device.
  • the processing unit is further configured to: if it is determined that the device address allocated by the scanned device does not exist in the device address set corresponding to the processing unit, add the device address to which the scanned device is assigned to the process The device address set corresponding to the unit.
  • the processing unit is further configured to: after generating the PCIe tree of the processing unit, send a verification message, the school The destination address of the message is the device address of the outermost leaf node device in the PCIe tree, where the root node of the PCIe tree is the processing unit, and the outermost leaf node of the PCIe tree is The node in the PCIe tree that is the farthest from the processing unit; if the processing unit receives the check message, it is determined that the PCIe tree is successfully established.
  • the processing unit is further configured to: if the processing unit does not receive the verification message, determine to be used for generating Repeating the scanned device during the scanning process of the PCIe tree; assigning the device to the repeated scan to a new device address different from the current device address of the repeatedly scanned device, and in the device address set corresponding to the processing unit The current device address is replaced with the new device address; the device connected to the repeatedly scanned device is scanned starting from the repeatedly scanned device, and the connection relationship between the scanned devices is recorded during the scanning process, and A new PCIe tree of the processing unit is generated based on the recorded connection relationship.
  • a second aspect provides a device scanning method, which can be applied to a PCIe storage system according to the first aspect or any possible implementation of the first aspect, the method comprising: the current of the N storage devices that are ring-connected
  • the processing unit scans the device that the current processing unit can access; records the connection relationship between the devices scanned by the current processing unit during the scanning process, and generates the current processing according to the recorded connection relationship between the scanned devices.
  • the PCIe tree of the unit is a device scanning method, which can be applied to a PCIe storage system according to the first aspect or any possible implementation of the first aspect, the method comprising: the current of the N storage devices that are ring-connected
  • the processing unit scans the device that the current processing unit can access; records the connection relationship between the devices scanned by the current processing unit during the scanning process, and generates the current processing according to the recorded connection relationship between the scanned devices.
  • the PCIe tree of the unit is a device scanning method, which can be applied to a PCIe storage system according to
  • the scanning the device that the current processing unit can access comprises: scanning, according to the depth-first search algorithm DFS, a device directly or indirectly connected to the current processing unit.
  • the scanning the device that the current processing unit can access comprises: if it is determined that the scanned device has been scanned by the current processing unit, returning to the device connected to the uplink of the scanned device for scanning.
  • the scanned device if it is determined that the scanned device has been scanned by the current processing unit, it is determined that the scanned device is a switching unit directly connected to the current processing unit, and returns to the switching unit directly connected to the current processing unit.
  • An upper-level switching unit connected to the uplink port, and scanning the unscanned device connected to the upper-level switching unit, and returning to the upper-level port connection of the upper-level switching unit after the device connected to the upper-level switching unit is scanned.
  • the switching unit scans the unscanned devices until all the devices connected to the switching units in the storage system are scanned.
  • the determining that the scanned device has been scanned by the current processing unit includes: determining whether the scanned device is assigned a device address If it is determined that the scanned device is assigned a device address, it is determined whether the device address to which the scanned device is assigned exists in the device corresponding to the current processing unit.
  • the device address set corresponding to the current processing unit includes a device address allocated by each device in the at least one device that the current processing unit has scanned; if the scanned device is assigned the device address exists in the In the device address set corresponding to the current processing unit, it is determined that the scanned device has been scanned by the current processing unit.
  • the scanning the device that the current processing unit can access further includes: if it is determined that the scanned device is not assigned a device address, Scanning the device to assign a device address, and adding the assigned device address to the device address set corresponding to the current processing unit; and/or if it is determined that the scanned device is assigned the device address does not exist in the current processing unit In the corresponding device address set, the device address to which the scanned device is assigned is added to the device address set corresponding to the current processing unit.
  • the method further includes: sending a verification message, where the destination address of the verification message is in the PCIe tree The device address of the outermost leaf node device, wherein the root node of the PCIe tree is the current processing unit, and the outermost leaf node of the PCIe tree is the node farthest from the current processing unit in the PCIe tree;
  • the current processing unit receives the verification message and determines that the PCIe tree is successfully established.
  • the method further includes: determining, in the scanning process for generating the PCIe tree, if the current processing unit does not receive the verification message Repeating the scanned device; assigning the device to the repeated scan to a new device address different from the current device address of the device that is repeatedly scanned, and the current device address in the device address set corresponding to the current processing unit Replace with the new device address; scan the device connected to the repeatedly scanned device from the repeatedly scanned device, record the connection relationship between the scanned devices during the scanning process, and according to the recorded connection Relationship, generating a new PCIe tree for the current processing unit.
  • the device scanning method provided by the embodiment of the present invention verifies that the establishment of the PCIe tree is correct by sending a verification message, and re-establishes the PCIe tree when determining that the current PCIe tree has an error, thereby avoiding establishment of an incorrect PCIe tree, thereby improving system performance. .
  • the device that is repeatedly scanned in the previous scanning process (that is, the device that interrupts the scanning) is scanned as a starting point, thereby avoiding the need to start repeated scanning from the first processing device, thereby improving the PCIe tree.
  • the speed and efficiency of the establishment further improve system performance.
  • a computer readable medium for storing a computer program comprising instructions for performing the method of any of the second aspect or any of the possible implementations of the second aspect.
  • FIG. 1 is a schematic diagram of the architecture of a storage system in the prior art.
  • FIG. 2 is a schematic structural diagram of a storage system according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an example of a storage system in accordance with an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a PCIe tree established by a CPU of the storage device A in the storage system example shown in FIG. 3.
  • FIG. 5 is a schematic flowchart of a device scanning method according to an embodiment of the present invention.
  • FIG. 6 is a schematic flowchart of a device scanning method according to another embodiment of the present invention.
  • FIG. 2 is a schematic structural diagram of a storage system 200 according to an embodiment of the present invention.
  • the storage system 200 includes N storage devices 210 in a ring connection, N ⁇ 2, wherein each storage device 210 can include at least one processing unit 212, an exchange unit 214, and at least one storage unit 216.
  • the switching unit 214 can include at least two uplink ports (shown as U in the figure) and at least two downlink ports (shown as D in the figure), wherein the at least two uplink ports can include at least one a first uplink port and at least one second uplink port, the at least one first uplink port may be connected to the at least one processing unit 212; the at least two PCIe downlink ports may include at least one first downlink port and a second downlink port The at least one first downlink port may be connected to the at least one storage unit 216.
  • the storage system 200 can be specifically a PCIe storage system.
  • the PCIe storage system may include N PCIe storage devices 210.
  • the switching unit 214 in the PCIe storage device 210 may be specifically a PCIe switching unit, and the PCIe switching unit may specifically connect to the processing unit through the PCIe uplink port and the PCIe downlink port respectively.
  • 212 is coupled to storage unit 216.
  • the processing unit 212 can serve as a PCIe master device, and the storage unit 216 can serve as a PCIe slave device, but the embodiment of the present invention is not limited thereto.
  • the switching unit 214 may include a switch chip, and may have multiple uplink ports and multiple downlink ports, where the multiple uplink ports may include at least one first uplink port and one or more second uplink ports, where the first The uplink port may be an uplink port for connection with a processing unit in the same storage device, and the second uplink port may be an uplink port for connection with a downlink port of the switching unit 214 in another storage device.
  • the multiple downlink ports may include at least one first downlink port and one or more second downlink ports, where the first downlink port may be a downlink for connecting with a storage unit in the same storage device.
  • the port, the second downlink port may be a downlink port for connecting to an uplink port of a switching unit in another storage device, but the embodiment of the present invention is not limited thereto.
  • the processing unit 212 can include a central processing unit (CPU) that can be used to connect with a first uplink port of a switching unit in the storage device.
  • the number of the at least one first uplink port of the switching unit 214 may be equal to the number of the at least one processing unit 212.
  • the at least one first uplink port may be in one-to-one correspondence with the at least one processing unit 212.
  • the number of the at least one first uplink port of the switching unit 214 may not be equal to the number of the at least one processing unit 212.
  • the number of the at least one first uplink port of the switching unit 214 is greater than the number of the at least one processing unit 212.
  • the number of the first uplink ports of the at least one first uplink port may be a one-to-one connection with the at least one processing unit 212, and the other part of the first uplink port may be idle, but this embodiment of the present invention Not limited.
  • the storage unit 216 can be used to connect to a first downstream port of the switching unit 214 in the same storage device.
  • at least one of the switching units 214 is first The number of row ports may be equal to the number of the at least one storage unit 216.
  • the at least one first downlink port may be connected to the at least one storage unit 216 in one-to-one correspondence; or at least one of the switching units 214
  • the number of the at least one storage unit may not be equal to the number of the at least one storage unit.
  • the number of the at least one first downlink port of the switching unit 214 may be greater than the number of the at least one storage unit 216.
  • a portion of the first downlink port of the first downlink port may be in a one-to-one correspondence with the at least one storage unit 216, and another portion of the first downlink port may be idle, but the embodiment of the present invention is not limited thereto.
  • different storage devices of the N storage devices may include different numbers of processing units 212 and/or storage units 216, and switching units in different storage devices may have different numbers of uplinks.
  • the port and/or the downlink port are not limited in this embodiment of the present invention.
  • the N storage devices are connected end to end through the switching unit to form a ring connection structure.
  • the two storage devices can communicate with each other through the ring connection structure, and the number of physical links can be reduced compared with the direct connection of multiple storage devices in the storage system of the prior art. Reduce system complexity and construction costs.
  • a connection link may exist between at least one processing unit 212 in the storage device, and a connection link may also exist between at least two uplink ports in the switching unit.
  • the uplink port and the downlink port in the switching unit may be specifically a PCIe port.
  • the processing unit may be specifically used as a PCIe master device.
  • the at least two uplink ports may be connected by a non-transparent bridge, and the at least one processing unit may be connected by a non-transparent bridge, but the embodiment of the present invention is not limited thereto.
  • the downlink port may be a shared downlink port that can be shared or a common downlink port that cannot be shared, where a common downlink port of the switching unit can only have a connection with one uplink port of the switching unit. a link or a connection channel, and correspondingly, only one processing unit in the storage device can access the downlink port, and access the storage unit connected to the downlink port through the downlink port, where the processing unit may specifically be the downlink port a processing unit corresponding to the connected uplink port; and the shared downlink port of the switching unit may have a connection link or a connection channel with all uplink ports in the switching unit, and accordingly, all processing units in the storage device may Accessing the shared downlink port, and accessing the storage unit connected thereto through the shared downlink port, so that the storage unit can be shared by all processing units in the storage device, but the present invention The example does not limit this.
  • At least two downlink ports in the switching unit may be shared downlink ports.
  • a connection link may exist between any one of the at least two downlink ports and any one of the at least two uplink ports.
  • any processing unit in the storage device can access any storage unit in the storage device through the shared downlink port, and can access any storage unit in other storage devices through the ring connection structure, so that the storage unit can be stored in the storage system.
  • Any processing unit in the access, and the processing unit can directly read and write operations to any storage unit, without interacting with other processing units, thereby simplifying the data read and write process of the processing unit, saving processing resources of the processing unit, and speeding up the data Read and write efficiency.
  • the processing unit can uniformly manage the resources of each storage unit in the storage system, which is beneficial to improving the overall performance of the system.
  • the processing unit 212 may scan through the device to discover a device directly or indirectly connected to the processing unit 212, that is, a device that the processing unit 212 can access.
  • the device that the processing unit 212 can access may include at least one switching unit and at least one storage unit and other processing units.
  • the storage unit may serve as a PCIe endpoint device
  • the switching unit may serve as a bus and a bridge for connecting the processing unit and the storage unit.
  • the uplink port and the downlink port in the switching unit may serve as a bus in the same switching unit.
  • the connection link between the uplink port and the downlink port may serve as a bridge, but the embodiment of the present invention is not limited thereto.
  • the processing unit 212 may record the information of each device scanned, such as the device number of each device, in the process of scanning the device, and record the connection relationship between the scanned devices, and The recorded connection relationship between the scanned devices is generated, and the PCIe tree of the processing unit 212 is generated.
  • the processing unit 212 may perform a device scan using a Depth First Search (DFS) algorithm. Specifically, when scanning the device directly connected to the processing unit 212, the processing unit 212 may further scan the device directly connected to the directly connected device in the downlink direction until it is found that the currently scanned device is not in the downlink direction. There are other devices connected.
  • DFS Depth First Search
  • the processing unit 212 may return to the superior device of the device that is not connected to the other device, where the superior device of the specific device may refer to the device located in the uplink direction of the specific device and connected to the specific device, that is, a device connected to the upstream end of the specific device, and the processing unit 212 can scan the unscanned device connected to the upper device in a downlink direction, when the upper device is configured After the device connected to the standby device is scanned, the upper device returning to the upper device continues scanning until all devices directly or indirectly connected to the processing unit 212 in the storage system are scanned.
  • the superior device of the specific device may refer to the device located in the uplink direction of the specific device and connected to the specific device, that is, a device connected to the upstream end of the specific device, and the processing unit 212 can scan the unscanned device connected to the upper device in a downlink direction, when the upper device is configured After the device connected to the standby device is scanned, the upper device returning to the upper device continues scanning until all devices directly or indirectly connected to the
  • the processing unit 212 when the processing unit 212 scans the device connected to the processing unit 212, the device may be specifically the switching unit 214 or the storage unit 216 in the storage system 200, and may determine Whether it is the first scan to the device, that is, whether the device has been scanned.
  • the processing unit 212 may record the connection relationship between the device and the device and other devices scanned, and continue scanning in the downlink direction of the device.
  • the scanning may be stopped in the downlink direction and returned to the upstream direction of the device, that is, the device may be returned to the device.
  • the device connected to the upstream side scans. In this way, the processing unit 212 can be prevented from repeatedly performing infinite scanning in the annular connection structure.
  • the processing unit 212 may return to the upper switching unit connected to the currently scanned device, and scan the unswapped connection with the upper switching unit. To the device.
  • the processing unit 212 may return the upper switching unit connected to the uplink port of the upper switching unit to scan the unscanned device until the storage system All devices connected directly or indirectly to the processing unit 212 are scanned.
  • the repeatedly scanned device (that is, the device that has been scanned by the processing unit) may be specifically a switching unit directly connected to the processing unit, but the embodiment of the present invention is not limited thereto.
  • the processing unit 212 may further record the information of the repeated scan, for example, record information of the device that is repeatedly scanned, and the repeated scan to
  • the information of the device may include the device number of the repeatedly scanned device and/or the connection relationship between the repeatedly scanned device and other devices, and the like, but the embodiment of the present invention is not limited thereto.
  • the processing unit 212 can determine whether to scan the device connected thereto for the first time in various manners. As an alternative embodiment, the processing unit 212 can determine if the scanned device is assigned a device address. Alternatively, if the scanned device is not assigned a device address, the processing unit 212 may determine to scan the device for the first time. At this time, the processing unit 212 may allocate a device address to the scanned device, and add the allocated device address to the device address set corresponding to the processing unit 212, where the device of the processing unit 212 Address collection The device address assigned to each device in the at least one device scanned by the processing unit 212 is included, wherein the device address assigned to each device may be allocated by the processing unit 212, or may be other processing in the storage system. The unit is allocated, which is not limited by the embodiment of the present invention.
  • the processing unit 212 may further determine whether the device address of the device is allocated by the processing unit, and if yes, indicating that the processing unit has scanned the device, the processing unit 212 may return to the upstream direction of the device. scanning.
  • the processing unit 212 may add the device address of the device to the device address set of the processing unit 212 and follow the device's downlink. The direction continues scanning, but the embodiment of the invention is not limited thereto.
  • the processing unit 212 can determine whether the scanned device is assigned a device address in various manners.
  • the processing unit 212 may obtain an address of the device from a configuration space of the scanned device, and may determine, according to the obtained address, whether the scanned device is assigned a device address. Specifically, if the obtained address is the default address, it can be determined that the device is not assigned a device address.
  • the default address may be a fixed address, and the default address may be pre-configured by the device manufacturer or the storage system is preset by other means.
  • the default address may be a numeric string consisting of multiple 0s or 1.
  • a character string composed of a plurality of characters F is not limited in this embodiment of the present invention.
  • the processing unit 212 may further determine whether the acquired address is allocated by the processing unit 212. For example, the processing unit 212 may determine whether the acquired address is included in the allocated address set by matching the acquired address with at least one device address included in the allocated address set of the processing unit 212. If the acquired address is included in the set of allocated addresses, it indicates that the acquired address is allocated by the processing unit 212, that is, the processing unit 212 has scanned the device. At this point, the processing unit 212 can return to the upstream device connected to the currently scanned device to continue scanning. Optionally, if the acquired address is not included in the set of allocated addresses, it indicates that the obtained address is allocated by another processing unit in the storage system, and the processing unit 212 scans the device for the first time. However, embodiments of the invention are not limited thereto.
  • the processing unit 212 may be configured to scan a device connected to the scanned current device in a downlink direction, where the connected device may include a switching unit and a storage device. At least one of a unit and a processing unit. If the device connected to the current device is scanned, the processing unit 212 may obtain the address of the scanned device; if the address is not the default address, it may be determined whether the address exists in the allocated address set of the processing unit 212.
  • the set of allocated addresses of the processing unit 212 includes a device address assigned to each device in the at least one device that the processing unit 212 has scanned.
  • the processing unit 212 may return to the current device to continue scanning to determine whether the current device is also connected to other unscanned devices.
  • the processing unit 212 may continue to scan the device connected to the scanned device in the downlink direction. Specifically, if the currently scanned device is a switching unit, the processing unit 212 may continue to scan the device connected to the downlink port of the switching unit; if the currently scanned device is a PCIe endpoint device, such as a storage unit, the processing is performed. The unit 212 may determine that the PCIe endpoint device is not connected to other devices, and return to the current device to continue scanning to determine whether the current device is connected to other unscanned devices, and the embodiment of the present invention is not limited thereto.
  • the PCIe storage system may include four storage devices: a storage device A, a storage device B, a storage device C, and a storage device D.
  • Each storage device includes a CPU, a switch chip, and at least one slave device, wherein the at least one slave device can be specifically at least one storage unit.
  • the switch chip may be specifically a PCIe switch chip, and the switch chip may have multiple PCIe uplink ports and multiple PCIe downlink ports, where a first uplink port of the multiple PCIe uplink ports is connected to the CPU, and the multiple PCIe At least one of the first PCIe downlink ports of the downlink ports may be in a one-to-one correspondence with the at least one slave device.
  • the slave device can be a shared slave device, ie the slave device can be shared by a CPU in the storage device.
  • the second downlink port of the PCIe switch chip in the storage device A is connected to the second uplink port of the PCIe switch chip in the storage device B, and the second downlink port of the PCIe switch chip in the storage device B is in the storage device C.
  • the second uplink port of the PCIe switch chip is connected, the second downlink port of the PCIe switch chip in the storage device C is connected to the second uplink port of the PCIe switch chip in the storage device D, and the second PCIe switch chip in the storage device D is the second The downlink port is connected to the second uplink port of the PCIe switch chip in the storage device A.
  • the CPU of the storage device A When the CPU of the storage device A performs device scanning, the CPU of the storage device A can be regarded as the root node of the PCIe tree.
  • the CPU of the storage device A may first scan the switch chip of the storage device A directly connected to the CPU, and if it is determined that the switch chip of the storage device A has not been allocated yet After the device address is assigned to the switch chip of the storage device A, the device address is recorded in the device address set of the CPU of the storage device A, and the switch chip of the storage device A and the CPU can be recorded. The connection between the two. Then, the CPU of the storage device A can scan the devices connected to the other uplink ports and the downlink ports of the switch chip in the same manner as the switch chips of the scan storage device A in a preset order.
  • the device connected to the uplink port and the downlink port in the switch chip of the storage device B can be scanned. At this time, even the switch chip of the storage device A The connected device has not been scanned yet, and the device connected to the switch chip of the storage device A is no longer scanned. Similarly, when the uplink port and the downlink port connected to the switch chip of the storage device B are scanned, If the device connected to the downlink port is the switch chip of the storage device C, the device connected to the switch chip of the storage device B is not scanned, but the device connected to the switch chip of the storage device C is scanned.
  • the switch chip of the storage device A when scanning the device connected to the switch chip of the storage device D, if the device connected to the downlink port of the switch chip of the storage device D is the switch chip of the storage device A, the switch chip of the storage device A Has been scanned before, so the device address of the switch chip of storage device A already exists in the device address set. That is, when a duplicate device address occurs, the connection relationship between the switch chip of the storage device A and the switch chip of the storage device D can be recorded, and then the switch chip of the storage device D connected to the uplink port of the switch chip of the storage device A is returned.
  • the root node and the outermost leaf node may be the CPU itself, wherein the distance between the outermost leaf node and the root node is the most Far, correspondingly, the number of nodes that need to pass through in the process of reaching the root node is the largest.
  • FIG. 3 to FIG. 4 are intended to help those skilled in the art to better understand the embodiments of the present invention and not to limit the scope of the embodiments of the present invention.
  • Those skilled in the art will appreciate that various modifications or changes can be made in accordance with the examples of Figures 3 through 4, such modifications or variations. It also falls within the scope of the embodiments of the present invention.
  • the processing unit 212 may also perform a verification process to determine the established Is the PCIe tree correct?
  • the processing unit 212 may send a check message, where the destination address of the check message is the device address of the outermost leaf node device in the PCIe tree, that is, the switching unit connected to the processing unit 212
  • the uplink port in the uplink sends a check message to the outermost leaf node of the PCIe tree, where the outermost leaf node may be the node in the PCIe tree that is the farthest from the processing unit 212, and the outermost leaf node
  • the number of nodes that need to pass to reach the processing unit is the largest. As can be seen from the PCIe tree example shown in FIG.
  • the processing unit 212 if the processing unit 212 establishes the correct PCIe tree, the outermost leaf node device of the PCIe tree can be the processing unit itself. Therefore, if the processing unit 212 can receive the verification message sent by itself, it indicates that the PCIe tree is successfully established; otherwise, the processing unit 212 does not receive the verification message, indicating that the PCIe tree is incorrect.
  • the need to re-establish the PCIe tree or the modification of the established PCIe tree is not limited in this embodiment of the present invention.
  • the processing unit 212 may further set a timer when sending the check message, and determine whether the check message is received before the timer expires. If the check message is received before the timer expires, it indicates that the PCIe tree is correctly established, but the embodiment of the present invention is not limited thereto.
  • the processing unit 212 may search for the device repeatedly scanned during the process of establishing the PCIe tree. For example, the processing unit 212 can determine the device to be scanned repeatedly by looking up the information of the repeated scan of the record. The processing unit 212 may allocate a new device address different from the device's current device address to the repeatedly scanned device, and may continue scanning with the repeatedly scanned device as a starting point to establish a new PCIe tree.
  • the new PCIe tree may be verified by the above process, and iteratively, until the processing unit determines that the established PCIe tree is correct, but the embodiment of the present invention is not limited thereto.
  • FIG. 5 shows a device scanning method 300 provided by an embodiment of the present invention.
  • the method 300 can be applied to the storage system 200 of the above embodiment.
  • the method 300 can be performed by a processing unit in a storage device.
  • the execution body of the method 300 is referred to as the current place.
  • the management unit which may be located in any storage device of the storage system, is not limited in this embodiment of the present invention.
  • the current processing unit scans the device that the current processing unit can access.
  • the device that the current processing unit can access may include a device directly or indirectly connected to the current processing unit, and specifically may include a switching unit directly or indirectly connected to the current processing unit and a device connected to the switching unit.
  • a device directly or indirectly connected to the current processing unit, and specifically may include a switching unit directly or indirectly connected to the current processing unit and a device connected to the switching unit.
  • a storage unit and/or a processing unit may be included in the device that the current processing unit can access.
  • the current processing unit can scan according to the DFS.
  • the current processing unit scans the device directly or indirectly connected to the current processing unit, if it is determined that the device is scanned for the first time, the device connected to the scanned device may continue to be scanned in the downlink direction.
  • the device that is connected to the uplink of the repeatedly scanned device may continue to perform. Scan, that is, you can return to the upstream branch of the device that is repeatedly scanned for scanning. Specifically, if it is determined that the scanning is repeated to the switching unit, it may be returned to the device connected to the uplink port of the repeatedly scanned switching unit for scanning; or if it is determined to repeatedly scan to the storage unit or the processing unit, it may return to The repeatedly scanned storage unit or the switching unit connected to the processing unit performs scanning.
  • the current processing unit determines that the currently scanned device has been scanned by the current processing unit, it may be determined that the currently scanned device is an exchange unit directly connected to the current processing unit.
  • the current processing unit may return to the upper switching unit connected to the uplink port of the currently scanned device, and scan and exchange with the superior.
  • the current processing unit may return to the upper switching unit connected to the uplink port of the upper switching unit to scan, to discover the device that is not scanned, Until all the devices connected to the switching unit in the storage system are scanned.
  • the current processing unit may further record the information of the repeated scan, for example, information of the device that is repeatedly scanned, for example, a device address of the device that is repeatedly scanned. Or the device number and the connection relationship between the repeatedly scanned device and other devices, but the embodiment of the present invention is not limited thereto.
  • the current processing unit may specifically determine whether the scanned device is assigned a device address; if it is determined that the scanned device is assigned a device address, determine whether the scanned device is assigned the device address.
  • the device address set corresponding to the current processing unit where the device address set corresponding to the current processing unit includes a device address allocated by each device in the at least one device that has been scanned by the current processing unit; if the scanned device is The assigned device address exists in the device address set corresponding to the current processing unit, and determines that the scanned device has been scanned by the current processing unit.
  • the current processing unit may allocate a device address for the scanned device, and add the allocated device address to the device address corresponding to the current processing unit. In the collection. At this time, optionally, the current processing unit may scan the device connected to the scanned device along the downlink direction.
  • the current processing unit may add the device address to which the scanned device is assigned to the device address.
  • the device address set corresponding to the current processing unit may scan the device connected to the scanned device along the downlink direction.
  • the method 300 may further include:
  • the destination address of the check message is a device address of the outermost leaf node device in the PCIe tree, where the root node of the PCIe tree is the current processing unit, and the outermost leaf node of the PCIe tree The node that is the farthest from the current processing unit in the PCIe tree;
  • the current processing unit receives the verification message, it is determined that the PCIe tree is successfully established.
  • the method 300 may further include:
  • the current processing unit does not receive the verification message, re-establish the PCIe tree of the current processing unit.
  • the current processing unit may determine the device that is repeatedly scanned during the scanning process for generating the PCIe tree; the device allocation for the repeated scanning is different from the Repeating the new device address of the current device address of the scanned device, and replacing the current device address with the new device address in the device address set corresponding to the current processing unit; scanning the device that is repeatedly scanned as a starting point The device connected to the repeatedly scanned device records the connection relationship between the scanned devices during the scanning process, and according to the record The recorded relationship between the scanned devices is generated, and a new PCIe tree of the current processing unit is generated.
  • the device scanning method when the current processing unit determines that the scanned device has been scanned by the current processing unit, stops scanning the device, and returns to the upstream device of the scanned device for scanning.
  • the uplink device of the scanned device may be specifically connected to the scanned device along the uplink direction, and the current processing unit may be prevented from scanning multiple times on the same device during the device scanning process. Loop and avoid creating a wrong PCIe tree due to repeated scans, improving the efficiency and accuracy of device scanning.
  • FIG. 6 is a schematic flowchart of a device scanning method 400 according to another embodiment of the present invention.
  • the processing unit scans the bus connected to the processing unit and assigns a bus number to the scanned bus.
  • a bus directly connected to the processing unit may be defined as a bus 0, and serially numbered for the scanned bus, but the embodiment of the present invention is not limited thereto.
  • the processing unit determines whether the downstream device of the current bus is connected to an unscanned bridge device.
  • the unscanned bridge device may be a unique bridge device connected in the downlink direction of the current bus, or may be a bridge device other than the bridge device connected in the downlink direction of the current bus that has been scanned.
  • the processing unit may perform S480 and S485. Otherwise, the processing unit can execute S430 and S440.
  • the processing unit may read the address of the currently scanned bridge device, and determine whether the read address is the default address in S440. If the read address is the default address, the processing unit may perform S450 and S410, that is, write the read address to the allocated address storage space, and continue scanning whether the downlink direction of the current bus is not scanned. Bridge equipment. Alternatively, if the read address is not the default address, the processing unit may perform S460.
  • the processing unit may determine whether the read address is assigned by the processing unit. Specifically, the processing unit may determine whether the read address exists in the allocated address storage space by comparing the read address with a device address stored in the allocated address storage space. If it exists, it indicates that the address read is allocated by the processing unit, otherwise, It indicates that the address read is not allocated by the processing unit. Optionally, the processing unit may determine, by other manners, whether the address that is read is allocated by the processing unit, which is not limited by the embodiment of the present invention.
  • the processing unit may perform S470 and S490, that is, interrupt further scanning of the current bridge device, and return to the current bridge device.
  • the upstream bus is scanned. If the upstream bus is bus 0, the processing unit can end the current device scanning process. If the upstream bus is not bus 0, the processing unit may perform S410.
  • the processing unit may perform S410, that is, continue to scan the downlink bus of the bridge device, but the embodiment of the present invention is not limited thereto.
  • FIG. 6 is intended to help those skilled in the art to better understand the embodiments of the present invention and not to limit the scope of the embodiments of the present invention.
  • a person skilled in the art will be able to make various modifications and changes in accordance with the example of FIG. 6 which are within the scope of the embodiments of the present invention.
  • system and “network” are used interchangeably herein.
  • the term “and/or” in this context is merely an association describing the associated object, indicating that there may be three relationships, for example, A and / or B, which may indicate that A exists separately, and both A and B exist, respectively. B these three situations.
  • the character "/” in this document generally means that the contextual object is an “or” relationship; the term “plurality” generally means at least two, that is, two or more.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Mathematical Physics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Small-Scale Networks (AREA)

Abstract

一种PCIe存储系统及设备扫描方法，能够降低系统复杂度和建造成本。该系统包括：呈环形连接的N个存储设备，每个存储设备包括至少一个处理单元、交换单元和至少一个存储单元，该交换单元包括至少两个上行端口和至少两个下行端口，其中，该至少两个上行端口包括至少一个第一上行端口和第二上行端口，该至少一个处理单元连接至该至少一个第一上行端口；该至少两个下行端口包括至少一个第一下行端口和第二下行端口，该至少一个存储单元连接至该至少一个第一下行端口；其中，第i个存储设备的第二下行端口与第i+1个存储设备的第二上行端口连接，第N个存储设备中的第二下行端口与第1个存储设备的第二上行端口连接。

Description

存储系统及设备扫描方法 技术领域
本发明实施例涉及存储领域，并且更具体地，涉及存储系统及设备扫描方法。
背景技术
现有技术中的存储系统，例如外围部件互连高速(Peripheral Component Interconnect express，PCIe)存储系统，可以包括多个存储设备，并且该多个存储设备可以两两互连。例如，如图1所示，存储系统100中包括N个存储设备110，该N个存储设备110中的任意两个存储设备110之间存在直连链路，并且可以通过两者之间的直连链路相互通信。因此，存储系统100中的N个存储设备之间存在N×(N-1)/2个直连链路。如果存储系统中包括的存储设备数量较多，即N的数值较大，存储系统的连接链路的数量较多，链路结构复杂。
发明内容
本发明实施例提供一种存储系统以及设备扫描方法，能够降低系统复杂度和建造成本。
第一方面，提供了一种存储系统，包括：呈环形连接的N个存储设备，每个存储设备包括至少一个处理单元、交换单元和至少一个存储单元，该交换单元包括至少两个上行端口和至少两个下行端口，其中，该至少两个上行端口包括至少一个第一上行端口和第二上行端口，该至少一个处理单元连接至该至少一个第一上行端口；该至少两个下行端口包括至少一个第一下行端口和第二下行端口，该至少一个存储单元连接至该至少一个第一下行端口；其中，在该N个存储设备中，第i个存储设备的第二下行端口与第i+1个存储设备的第二上行端口连接，第N个存储设备中的第二下行端口与第1个存储设备的第二上行端口连接，i＝1，…，N-1，N≥2。
可选地，该存储系统具体为PCIe存储系统。此时，该至少两个上行端口可以具体为PCIe上行端口，该至少两个下行端口可以具体为PCIe下行端口。
因此，本发明实施例提供的存储系统，通过将N个存储设备首尾相接，构成环形连接结构，以使得该N个存储设备中的任意两个存储设备之间可以相互通信，与现有技术中通过存储设备两两直连相比，能够降低系统的物理链路的复杂度和建造成本。
在第一方面的第一种可能的实现方式中，该至少两个下行端口中的任一下行端口与该至少两个上行端口中的任一上行端口之间存在连接链路。
此时，该至少两个下行端口均为共享下行端口，相应地，存储设备中的至少一个存储单元为共享存储单元，即能够被存储设备中的所有处理单元共享。
这样，通过将交换单元中的下行端口设置为共享端口，使得存储单元可以被系统中的所有处理单元共享。此外，处理单元可以对系统中的任意存储单元直接进行读写操作，无需通过与其它处理单元进行交互，从而简化处理单元的数据读写流程，节约处理单元的处理资源，并加快数据读写效率。此外，处理单元可以对存储系统中的各个存储单元的资源统一进行管理，有利于提高系统整体性能。
可选地，该至少两个上行端口之间存在连接链路。
结合上述可能的实现方式，在第一方面的第二种可能的实现方式中，处理单元用于：扫描该处理单元能够访问的设备；在扫描过程中记录该处理单元扫描到的设备之间的连接关系，并且根据所记录的连接关系，生成该处理单元的PCIe树。
结合上述可能的实现方式，在第一方面的第三种可能的实现方式中，在扫描能够访问的设备时，处理单元具体用于：根据深度优先搜索算法DFS，扫描与该处理单元连接的设备。
可选地，处理单元具体用于：确定扫描到的设备是否曾被该处理单元扫描过；若确定扫描到的设备曾经被该处理单元扫描过，返回至与该扫描到的设备的上行端连接的设备进行扫描。
具体地，与该扫描到的设备的上行端连接的设备可以具体为沿着上行方向与该扫描到的设备连接的设备。
可选地，若确定重复扫描到某个设备，则处理单元可以返回至该设备的上行设备继续进行扫描。
可选地，处理单元具体用于：若确定扫描到的设备曾经被处理单元扫描 过，确定该扫描到的设备为与该处理单元直接相连的交换单元。
可选地，处理单元具体用于：若确定扫描到的设备曾经被处理单元扫描过，返回至与该扫描到的设备的上行端口连接的上级交换单元，并扫描与该上级交换单元连接的未被扫描到的设备。
可选地，处理单元还用于：当该上级交换单元所连接的设备被扫描完后，则返回该上级交换单元的上行端口连接的上级交换单元扫描未被扫描到的设备，直到与该处理单元连接的所有交换单元所连接的设备全部被扫描完。
可选地，若确定首次扫描到某个设备，则处理单元可以为该设备分配设备地址，并沿着下行方向扫描与该设备连接的设备。可选地，该处理单元还可以将为该设备分配的设备地址添加至该处理单元的分配地址集合中。
可选地，上行方向可以具体为靠近该处理单元的方向，下行方向可以具体为远离该处理单元的方向。
结合上述可能的实现方式，在第一方面的第四种可能的实现方式中，处理单元具体用于：确定所扫描到的设备是否被分配了设备地址；若确定该扫描到的设备被分配了设备地址，判断该扫描到的设备被分配的设备地址是否存在于该处理单元对应的设备地址集合中，其中，处理单元对应的设备地址集合包括该处理单元已扫描到的至少一个设备中每个设备被分配的设备地址；若该扫描到的设备被分配的设备地址存在于该处理单元对应的设备地址集合中，确定扫描到的设备曾经被该处理单元扫描过。
结合上述可能的实现方式，在第一方面的第五种可能的实现方式中，处理单元还用于：若确定该扫描到的设备没有被分配设备地址，为该扫描到的设备分配设备地址，并将该分配的设备地址添加到该处理单元对应的设备地址集合中。
此时，可选地，该处理单元可以继续扫描与该扫描的设备连接的设备。
可选地，处理单元还用于：若确定该扫描到的设备被分配的设备地址不存在于该处理单元对应的设备地址集合中，将该扫描到的设备被分配的设备地址添加到该处理单元对应的设备地址集合中。
结合第一方面的第五种可能的实现方式，在第一方面的第七种可能的实现方式中，处理单元还用于：在生成该处理单元的PCIe树之后，发送校验消息，该校验消息的目的地址为该PCIe树中最外层叶节点设备的设备地址，其中，该PCIe树的根节点为该处理单元，该PCIe树的最外层叶节点为在 PCIe树中与该处理单元距离最远的节点；若该处理单元接收到该校验消息，确定该PCIe树建立成功。
结合第一方面的第五种可能的实现方式，在第一方面的第八种可能的实现方式中，处理单元还用于：若该处理单元未接收到该校验消息，确定在用于生成该PCIe树的扫描过程中重复扫描到的设备；为该重复扫描到的设备分配不同于该重复扫描到的设备当前的设备地址的新设备地址，并在该处理单元对应的设备地址集合中将该当前的设备地址替换为该新设备地址；以该重复扫描到的设备为起点，扫描与该重复扫描到的设备连接的设备，在扫描过程中记录扫描到的设备之间的连接关系，并且根据所记录的连接关系，生成该处理单元的新的PCIe树。
第二方面，提供了一种设备扫描方法，可以应用于如第一方面或第一方面的任意可能实现方式中的PCIe存储系统，该方法包括：该呈环形连接的N个存储设备中的当前处理单元扫描该当前处理单元能够访问的设备；在扫描过程中记录该当前处理单元扫描到的设备之间的连接关系，并且根据所记录的扫描到的设备之间的连接关系，生成该当前处理单元的PCIe树。
在第二方面的第一种可能的实现方式中，该扫描该当前处理单元能够访问的设备，包括：根据深度优先搜索算法DFS，扫描与该当前处理单元直接或间接连接的设备。
可选地，该扫描该当前处理单元能够访问的设备，包括：若确定扫描到的设备曾经被该当前处理单元扫描过，返回至与该扫描到的设备的上行端连接的设备进行扫描。
具体地，若确定扫描到的设备曾经被该当前处理单元扫描过，则确定该扫描到的设备为与该当前处理单元直接相连的交换单元，返回至与该当前处理单元直接相连的交换单元的上行端口连接的上级交换单元，并扫描与该上级交换单元连接的未被扫描到的设备，当该上级交换单元所连接的设备被扫描完后，则返回该上级交换单元的上行端口连接的上级交换单元扫描未被扫描的设备，直到该存储系统中所有交换单元所连接的设备全部被扫描完。
结合上述可能的实现方式，在第二方面的第二种可能的实现方式中，该确定扫描到的设备曾经被该当前处理单元扫描过，包括：确定所扫描到的设备是否被分配了设备地址；若确定该扫描到的设备被分配了设备地址，判断该扫描到的设备被分配的设备地址是否存在于该当前处理单元对应的设备 地址集合中，其中，该当前处理单元对应的设备地址集合包括该当前处理单元已扫描到的至少一个设备中每个设备被分配的设备地址；若该扫描到的设备被分配的设备地址存在于该当前处理单元对应的设备地址集合中，确定扫描到的设备曾经被该当前处理单元扫描过。
结合上述可能的实现方式，在第二方面的第三种可能的实现方式中，该扫描该当前处理单元能够访问的设备，还包括：若确定该扫描到的设备没有被分配设备地址，为该扫描到的设备分配设备地址，并将该分配的设备地址添加到该当前处理单元对应的设备地址集合中；和/或若确定该扫描到的设备被分配的设备地址不存在于该当前处理单元对应的设备地址集合中，将该扫描到的设备被分配的设备地址添加到该当前处理单元对应的设备地址集合中。
结合上述可能的实现方式，在第二方面的第四种可能的实现方式中，在生成该PCIe树之后，该方法还包括：发送校验消息，该校验消息的目的地址为该PCIe树中最外层叶节点设备的设备地址，其中，该PCIe树的根节点为该当前处理单元，该PCIe树的最外层叶节点为在PCIe树中与该当前处理单元距离最远的节点；若该当前处理单元接收到该校验消息，确定该PCIe树建立成功。
结合上述可能的实现方式，在第二方面的第五种可能的实现方式中，该方法还包括：若该当前处理单元未接收到该校验消息，确定在用于生成该PCIe树的扫描过程中重复扫描到的设备；为该重复扫描到的设备分配不同于该重复扫描到的设备当前的设备地址的新设备地址，并在该当前处理单元对应的设备地址集合中将该当前的设备地址替换为该新设备地址；以该重复扫描到的设备为起点，扫描与该重复扫描到的设备连接的设备，在扫描过程中记录扫描到的设备之间的连接关系，并且根据所记录的连接关系，生成该当前处理单元的新的PCIe树。
本发明实施例提供的设备扫描方法，通过发送校验消息来验证PCIe树的建立是否正确，并且在确定当前PCIe树存在错误时重新建立PCIe树，能够避免错误PCIe树的建立，从而提高系统性能。
此外，在本发明实施例中，通过以上次扫描过程中重复扫描到的设备(即中断扫描处的设备)为起点进行扫描，避免需要从该第一处理设备开始进行重复扫描，从而提高PCIe树的建立速度和效率，进一步提高系统性能。
第三方面，提供了一种计算机可读介质，用于存储计算机程序，该计算机程序包括用于执行第二方面或第二方面的任意可能的实现方式中的方法的指令。
附图说明
为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面所描述的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是现有技术中的存储系统的架构示意图。
图2是本发明实施例的存储系统的架构示意图。
图3是本发明实施例的存储系统示例的示意图。
图4是图3所示的存储系统示例中存储设备A的CPU建立的PCIe树的示意图。
图5是本发明实施例的设备扫描方法的示意性流程图。
图6是本发明另一实施例的设备扫描方法的示意性流程图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都应属于本发明保护的范围。
图2示出了本发明实施例提供的存储系统200的架构示意图。该存储系统200包括：呈环形连接的N个存储设备210，N≥2，其中，每个存储设备210可以包括至少一个处理单元212、交换单元214和至少一个存储单元216。
如图2所示，该交换单元214可以包括至少两个上行端口(图中表示为U)和至少两个下行端口(图中表示为D)，其中，该至少两个上行端口可以包括至少一个第一上行端口和至少一个第二上行端口，该至少一个第一上行端口可以与该至少一个处理单元212连接；该至少两个PCIe下行端口可以包括至少一个第一下行端口和第二下行端口，该至少一个第一下行端口可以与该至少一个存储单元216连接。
在该N个存储设备的环形连接结构中，第i个存储设备210的交换单元的第二下行端口可以与第i+1个存储设备210的交换单元的第二上行端口连接，并且第N个存储设备210中的交换单元的第二下行端口可以与第1个存储设备210的交换单元的第二上行端口连接，其中，i＝1，…，N-1。
可选地，该存储系统200可以具体为PCIe存储系统。此时，该PCIe存储系统可以包括N个PCIe存储设备210，PCIe存储设备210中的交换单元214可以具体为PCIe交换单元，该PCIe交换单元可以具体通过PCIe上行端口和PCIe下行端口分别与处理单元212和存储单元216连接。可选地，该处理单元212可以作为PCIe主设备，而存储单元216可以作为PCIe从设备，但本发明实施例不限于此。
该交换单元214可以包括交换芯片，可以具有多个上行端口和多个下行端口，其中，该多个上行端口可以包括至少一个第一上行端口以及一个或多个第二上行端口，其中，第一上行端口可以为用于与同一个存储设备中的处理单元连接的上行端口，第二上行端口可以为用于与另一个存储设备中的交换单元214的下行端口连接的上行端口。此外，该多个下行端口可以包括至少一个第一下行端口以及一个或多个第二下行端口，其中，该第一下行端口可以为用于与同一个存储设备中的存储单元连接的下行端口，该第二下行端口可以为用于与另一个存储设备中的交换单元的上行端口连接的下行端口，但本发明实施例不限于此。
该处理单元212可以包括中央处理单元(Central Processing Unit，CPU)，可以用于与存储设备中的交换单元的第一上行端口连接。在存储设备中，交换单元214的至少一个第一上行端口的数量可以等于该至少一个处理单元212的数量，此时，该至少一个第一上行端口可以与该至少一个处理单元212一一对应地连接；或者，交换单元214的至少一个第一上行端口的数量可以不等于该至少一个处理单元212的数量，例如，交换单元214的至少一个第一上行端口的数量大于该至少一个处理单元212的数量，此时，该至少一个第一上行端口中的部分第一上行端口可以与该至少一个处理单元212一一对应的连接，而另一部分第一上行端口可以闲置，但本发明实施例对此不做限定。
该存储单元216可以用于与同一个存储设备中的交换单元214的第一下行端口连接。可选地，在存储设备210内，交换单元214的至少一个第一下 行端口的数量可以等于该至少一个存储单元216的数量，此时，该至少一个第一下行端口可以与该至少一个存储单元216一一对应地连接；或者，该交换单元214的至少一个第一下行端口的数量也可以不等于该至少一个存储单元的数量，例如，该交换单元214的至少一个第一下行端口的数量可以大于该至少一个存储单元216的数量，此时，该至少一个第一下行端口中的部分第一下行端口可以与该至少一个存储单元216一一对应的连接，而另一部分第一下行端口可以闲置，但本发明实施例不限于此。
在本发明实施例中，可选地，该N个存储设备中的不同存储设备可以包括不同数量的处理单元212和/或存储单元216，并且不同存储设备中的交换单元可以具有不同数量的上行端口和/或下行端口，本发明实施例对此不做限定。
这样，在本发明实施例提供的存储系统中，N个存储设备通过交换单元首尾相接，构成环形连接结构。该N个存储设备中的任意两个存储设备之间可以通过该环形连接结构相互通信，与现有技术的存储系统中多个存储设备两两直连相比，能够减少物理链路的数量，降低系统的复杂度和建造成本。
可选地，在本发明实施例中，存储设备内的至少一个处理单元212之间可以存在连接链路，并且交换单元中的至少两个上行端口之间也可以存在连接链路。作为一个可选实施例，若该存储设备具体为PCIe存储设备，则该交换单元中的上行端口和下行端口可以具体为PCIe端口，此时，该处理单元可以具体作为PCIe主设备，该存储单元可以具体作为PCIe从设备。此外，可选地，该至少两个上行端口之间可以通过非透明桥连接，并且该至少一个处理单元之间可以通过非透明桥连接，但本发明实施例不限于此。
可选地，该下行端口可以为能够被共享的共享下行端口或为不能够被共享的普通下行端口，其中，交换单元的普通下行端口只能与该交换单元中的一个上行端口之间存在连接链路或连接通道，相应地，存储设备内仅有一个处理单元能访问该下行端口，并通过该下行端口访问与该下行端口连接的存储单元，其中，该处理单元具体可以为与该下行端口连接的上行端口所对应的处理单元；而交换单元的共享下行端口可以与该交换单元中的所有上行端口之间均存在连接链路或连接通道，相应地，存储设备内的所有处理单元均可以访问该共享下行端口，并通过该共享下行端口访问与其连接的存储单元，使得该存储单元能够被该存储设备中的所有处理单元共享，但本发明实 施例对此不做限定。
作为一个可选实施例，交换单元中的至少两个下行端口可以均为共享下行端口。相应地，该至少两个下行端口中的任一下行端口与该至少两个上行端口中的任一上行端口之间可以存在连接链路。这样，存储设备内的任意处理单元均可以通过该共享下行端口访问该存储设备中的任意存储单元，并且可以通过环形连接结构访问其它存储设备中的任意存储单元，从而使得存储单元可以被存储系统中的任意处理单元访问，并且处理单元可以对任意存储单元直接进行读写操作，无需通过与其它处理单元进行交互，从而简化处理单元的数据读写流程，节约处理单元的处理资源，并加快数据读写效率。此外，处理单元可以对存储系统中的各个存储单元的资源统一进行管理，有利于提高系统整体性能。
在本发明实施例中，可选地，处理单元212可以通过设备扫描，以发现与该处理单元212直接或间接连接的设备，即发现该处理单元212能够访问的设备。其中，该处理单元212能够访问的设备可以包括至少一个交换单元和至少一个存储单元及其他处理单元。具体地，存储单元可以作为PCIe端点设备，该交换单元可以作为总线和桥，用于连接处理单元和存储单元，例如，交换单元中的上行端口和下行端口可以作为总线，同一个交换单元中的上行端口与下行端口之间的连接链路可以作为桥，但本发明实施例不限于此。
具体地，处理单元212可以在设备扫描的过程中记录扫描到的每个设备的信息，例如每个设备的设备号等配置信息，并记录扫描到的各个设备之间的连接关系，以及根据所记录的扫描到的各个设备之间的连接关系，生成处理单元212的PCIe树。
可选地，处理单元212可以利用深度优先搜索(Depth First Search，DFS)算法进行设备扫描。具体地，处理单元212在扫描到与该处理单元212直接连接的设备时，可以进一步沿着下行方向扫描与该直接连接的设备直接连接的设备，直到发现当前扫描到的设备沿着下行方向未连接有其它设备。此时，该处理单元212可以返回至与该未连接其它设备的设备的上级设备，其中，特定设备的上级设备可以指位于该特定设备的上行方向并且与该特定设备连接的设备，也就是与该特定设备的上行端连接的设备，并且该处理单元212可以沿着下行方向扫描与该上级设备连接的未被扫描到的设备，当该上级设 备所连接的设备被扫描完后，则返回该上级设备的上级设备继续进行扫描，直到该存储系统中所有与该处理单元212直接或间接连接的设备全部被扫描完。
在本发明实施例中，可选地，当处理单元212扫描到与该处理单元212连接的设备时，其中，该设备可以具体为该存储系统200中的交换单元214或存储单元216，可以确定是否为首次扫描到该设备，即确定是否曾经扫描到过该设备。可选地，若确定是首次扫描到该设备，则该处理单元212可以记录该设备以及该设备与扫描到的其它设备之间的连接关系，并沿着该设备的下行方向继续进行扫描。可选地，若确定不是首次扫描到该设备，即曾经扫描到过该设备，则可以停止扫描该设备的下行方向，并返回至与该设备的上行方向进行扫描，即可以返回至该设备的上行端连接的设备进行扫描。这样，可以防止该处理单元212在该环形连接结构中进行重复地无限扫描。
具体地，如果处理单元212确定当前扫描到的设备曾经被该处理单元扫描过，则可以返回至与该当前扫描到的设备连接的上级交换单元，并扫描与该上级交换单元连接的未被扫描到的设备。可选地，当该上级交换单元所连接的设备被扫描完后，则该处理单元212可以返回与该上级交换单元的上行端口连接的上级交换单元扫描未被扫描的设备，直到该存储系统中所有与该处理单元212直接或间接连接的设备被扫描完。
可选地，该重复扫描到的设备(即曾经被该处理单元扫描过的设备)可以具体为与该处理单元直接连接的交换单元，但本发明实施例不限于此。
可选地，如果确定重复扫描到该设备，即确定不是首次扫描到该设备，则该处理单元212还可以记录该重复扫描的信息，例如记录该重复扫描到的设备的信息，该重复扫描到的设备的信息可以包括该重复扫描到的设备的设备号和/或该重复扫描到的设备与其它设备之间的连接关系，等等，但本发明实施例不限于此。
可选地，该处理单元212可以通过多种方式确定是否首次扫描到与其连接的设备。作为一个可选实施例，该处理单元212可以确定扫描到的设备是否被分配了设备地址。可选地，如果该扫描到的设备未被分配设备地址，则该处理单元212可以确定首次扫描到该设备。此时，可选地，该处理单元212可以为该扫描到的设备分配设备地址，并将该分配的设备地址添加至该处理单元212对应的设备地址集合中，其中，该处理单元212的设备地址集合可 以包括该处理单元212扫描到的至少一个设备中每个设备被分配的设备地址，其中，每个设备被分配的设备地址可以是本处理单元212分配的，也可以是存储系统中的其它处理单元分配的，本发明实施例对此不做限定。
可选地，如果确定该扫描到的设备被分配了设备地址，则表明该处理单元212或存储系统中的其它处理单元212曾经扫描到过该设备。此时，该处理单元212可以进一步确定该设备的设备地址是否是本处理单元分配的，如果是，则表明该处理单元曾经扫描到该设备，该处理单元212可以返回至该设备的上行方向进行扫描。可选地，如果确定该设备的设备地址是其它处理单元212分配的，则该处理单元212可以将该设备的设备地址添加至该处理单元212的设备地址集合中，并沿着该设备的下行方向继续扫描，但本发明实施例不限于此。
在本发明实施例中，该处理单元212可以通过多种方式确定扫描到的设备是否被分配了设备地址。可选地，该处理单元212可以从扫描到的设备的配置空间，获取该设备的地址，并且可以根据获取到的地址，确定扫描到的设备是否被分配了设备地址。具体地，如果获取到的地址为默认地址，则可以确定该设备未被分配设备地址。其中，该默认地址可以为固定地址，并且该默认地址可以是设备厂商预先配置的或存储系统通过其它方式预先设置的，例如该默认地址可以为由多个0、1组成的数字串或为由多个字符F组成的字符串，本发明实施例对此不做限定。
可选地，如果获取到的该地址不为默认地址，表明该设备被分配了设备地址。此时，该处理单元212可以进一步确定获取到的该地址是否为本处理单元212分配的。例如，该处理单元212可以通过将获取到的地址与该处理单元212的分配地址集合中包括的至少一个设备地址进行匹配，确定该分配地址集合中是否包括该获取到的地址。如果该分配地址集合中包括该获取到的地址，则表明该获取到的地址是由本处理单元212分配的，即该处理单元212曾经扫描到该设备。此时，该处理单元212可以返回至与当前扫描到的设备连接的上行设备继续进行扫描。可选地，如果该分配地址集合中不包括该获取到的地址，则表明获取到的地址是由存储系统中的其它处理单元分配的，而本处理单元212是第一次扫描到该设备，但本发明实施例不限于此。
作为一个可选实施例，处理单元212可以用于扫描沿着下行方向与扫描到的当前设备所连接的设备，其中，该连接的设备可以包括交换单元、存储 单元和处理单元中的至少一种。如果扫描到与该当前设备连接的设备，则该处理单元212可以获取扫描到的设备的地址；若该地址不为默认地址，则可以确定该处理单元212的分配地址集合中是否存在该地址，其中，该处理单元212的分配地址集合包括该处理单元212已扫描到的至少一个设备中每个设备被分配的设备地址。可选地，如果该分配地址集合中存在该地址，则该处理单元212可以返回至当前设备继续进行扫描，以确定该当前设备是否还连接有其它未扫描到的设备。可选地，作为另一实施例，如果该分配地址集合中不存在该地址，则该处理单元212可以继续沿着下行方向扫描与该扫描到的设备连接的设备。具体地，如果当前扫描到的设备为交换单元，则该处理单元212可以继续扫描与该交换单元的下行端口连接的设备；如果当前扫描到的设备为PCIe端点设备，例如存储单元，则该处理单元212可以可以确定该PCIe端点设备未连接其它设备，并返回至当前设备继续进行扫描，以确定该当前设备是否连接有其它未扫描到的设备，本发明实施例不限于此。
作为一个示例，在图3所示的PCIe存储系统中，PCIe存储系统可以包括4个存储设备：存储设备A、存储设备B、存储设备C和存储设备D。每个存储设备包括一个CPU、交换芯片和至少一个从设备，其中，该至少一个从设备可以具体为至少一个存储单元。该交换芯片可以具体为PCIe交换芯片，并且该交换芯片可以具有多个PCIe上行端口和多个PCIe下行端口，其中，该多个PCIe上行端口中的第一上行端口与CPU连接，该多个PCIe下行端口中的至少一个第一PCIe下行端口可以与该至少一个从设备一一对应的连接。该从设备可以为共享从设备，即该从设备能够被存储设备中的CPU共享。此外，存储设备A中的PCIe交换芯片的第二下行端口与存储设备B中的PCIe交换芯片的第二上行端口连接，存储设备B中的PCIe交换芯片的第二下行端口与存储设备C中的PCIe交换芯片的第二上行端口连接，存储设备C中的PCIe交换芯片的第二下行端口与存储设备D中的PCIe交换芯片的第二上行端口连接，存储设备D中的PCIe交换芯片的第二下行端口与存储设备A中的PCIe交换芯片的第二上行端口连接。
在存储设备A的CPU进行设备扫描时，可以将存储设备A的CPU作为PCIe树的根节点。该存储设备A的CPU可以首先扫描到与该CPU直接连接的存储设备A的交换芯片，若确定存储设备A的交换芯片还未被分配 设备地址，则在为存储设备A的交换芯片分配设备地址后，将所分配的设备地址记录至该存储设备A的CPU的设备地址集合中，并且可以记录存储设备A的交换芯片与该CPU之间的连接关系。然后，该存储设备A的CPU可以按照预设的顺序以与扫描存储设备A的交换芯片同样的方式扫描该交换芯片的其他上行端口及下行端口所连接的设备。当扫描到与下行端口所连接的存储设备B的交换芯片时，则可以对存储设备B的交换芯片中的上行端口及下行端口所连接的设备进行扫描，此时，即使存储设备A的交换芯片所连接的设备还没有被扫描完，也不再对存储设备A的交换芯片所连接的设备进行扫描，同样，在扫描存储设备B的交换芯片中的上行端口及下行端口所连接的设备时，若扫描到下行端口所连接的设备为存储设备C的交换芯片时，则不再扫描存储设备B的交换芯片所连接的设备，而是对存储设备C的交换芯片所连接的设备进行扫描，以此类推，在对存储设备D的交换芯片所连接的设备进行扫描时，若扫描到存储设备D的交换芯片的下行端口所连接的设备为存储设备A的交换芯片，由于存储设备A的交换芯片之前已经被扫描过，所以存储设备A的交换芯片的设备地址已经存在于所述设备地址集合中，即出现了重复的设备地址，则可以记录存储设备A的交换芯片与存储设备D的交换芯片之间的连接关系，然后返回存储设备A的交换芯片的上行端口连接的存储设备D的交换芯片，对存储设备D的交换芯片所连接的之前没有扫描到设备进行扫描，以此类推，直到返回至存储设备A的交换芯片，对存储设备A的交换芯片所连接的之前没有扫描到设备进行扫描，在对存储设备A的交换芯片所连接的设备扫描完成之后，即完成了对存储设备A的CPU所能访问的设备的扫描，并根据上述设备扫描过程中记录的所扫描到的设备的连接关系生成存储设备A的CPU的PCIe树(如图4所示的)。另外，为了后续校验所述PCIe树是否正确建立，可以在所述PCIe树的末端的存储设备A的交换芯片上建立与存储设备A的CPU的连接关系。
如图4所示，在存储设备A的CPU所对应的PCIe树中，根节点和最外层叶节点可以均为该CPU本身，其中，该最外层叶节点与根节点之间的距离最远，相应地在到达根节点的过程中需要经过的节点数目最多。
应理解，图3至图4的例子是为了帮助本领域技术人员更好地理解本发明实施例，而非要限制本发明实施例的范围。本领域技术人员根据所给出的图3至图4的例子，显然可以进行各种等价的修改或变化，这样的修改或变 化也落入本发明实施例的范围内。
在本发明实施例中，可选地，在处理单元212通过上述设备扫描过程生成了该处理单元212对应的PCIe树之后，还可以对生成的该PCIe树进行校验处理，以确定建立的该PCIe树是否正确。
作为一个可选实施例，该处理单元212可以发送校验消息，该校验消息的目的地址为该PCIe树中最外层叶节点设备的设备地址，即该处理单元212通过与其连接的交换单元中的上行端口向该PCIe树的最外层叶节点发送校验消息，其中，该最外层叶节点可以为该PCIe树中与该处理单元212距离最远的节点，该最外层叶节点到达该处理单元需要经过的节点数目最多。从图4所示的PCIe树示例可以看出，如果该处理单元212建立了正确的PCIe树，则该PCIe树的最外层叶节点设备可以为该处理单元本身。因此，如果该处理单元212能够接收到自身发送的该校验消息，则表明该PCIe树建立成功；否则，该处理单元212未接收到该校验消息，则表明该PCIe树错误，该处理单元212需要重新建立PCIe树或对已建立的该PCIe树进行修正，本发明实施例对此不做限定。
可选地，该处理单元212在发送该校验消息时，还可以设置定时器，并确定在该定时器超时前是否收到该校验消息。如果在在定时器超时前接收到该校验消息，则表明该PCIe树建立正确，但本发明实施例不限于此。
作为一个可选实施例，如果该处理单元212未接收到或未在定时器超时前接收到该校验消息，则该处理单元212可以查找在建立该PCIe树的过程中重复扫描到的设备，例如，该处理单元212可以通过查找记录的重复扫描的信息，确定该重复扫描到的设备。该处理单元212可以为该重复扫描到的设备分配不同于该设备的当前设备地址的新设备地址，并且可以以该重复扫描到的设备为起点继续扫描，以建立新的PCIe树。当该处理单元建立了新的PCIe树之后，还可以采用上述流程对新的PCIe树进行验证，循环往复，直到该处理单元确定其建立的PCIe树正确，但本发明实施例不限于此。
上文中结合图2至图4，详细描述了本发明实施例提供的存储系统，下面将结合图5至图6，描述本发明实施例提供的存储系统中的设备扫描方法。
图5示出了本发明实施例提供的设备扫描方法300。该方法300可以应用于上述实施例中的存储系统200。可选地，该方法300可以由存储设备中的处理单元执行。为了便于理解，下面将该方法300的执行主体称为当前处 理单元，其中，该当前处理单元可以位于存储系统的任一存储设备中，本发明实施例对此不做限定。
S310，当前处理单元扫描该当前处理单元能够访问的设备。
可选地，该当前处理单元能够访问的设备可以包括与该当前处理单元直接或间接连接的设备，具体可以包括与该当前处理单元直接或间接连接的交换单元以及与该交换单元连接的设备，例如存储单元和/或处理单元。
可选地，该当前处理单元可以根据DFS进行扫描。
可选地，当该当前处理单元扫描到与该当前处理单元直接或间接连接的设备时，若确定首次扫描到该设备，则可以沿着下行方向继续扫描与该扫描到的设备连接的设备。
可选地，若确定该当前处理单元重复扫描到该设备，即该扫描到的设备曾经被该当前处理单元扫描过，则可以返回至与该重复扫描到的设备的上行端连接的设备继续进行扫描，即可以返回至该重复扫描到的设备的上行分支进行扫描。具体地，若确定重复扫描到交换单元，则可以返回至与该重复扫描到的交换单元的上行端口连接的设备进行扫描；或若确定重复扫描到与存储单元或处理单元，则可以返回至与该重复扫描到的存储单元或处理单元连接的交换单元进行扫描。
可选地，如果该当前处理单元确定当前扫描到的设备曾经被该当前处理单元扫描过，则可以确定该当前扫描到的设备为与该当前处理单元直接相连的交换单元。
可选地，如果该当前处理单元确定当前扫描到的设备曾经被该当前处理单元扫描过，则可以返回至与该当前扫描到的设备的上行端口连接的上级交换单元，并扫描与该上级交换单元连接的未被扫描到的设备。
可选地，如果确定该上级交换单元所连接的设备被扫描完，则该当前处理单元可以返回与该上级交换单元的上行端口连接的上级交换单元进行扫描，以发现未被扫描到的设备，直到该存储系统中所有交换单元所连接的设备全部被扫描完。
可选地，若确定该当前处理单元重复扫描到设备，则该当前处理单元还可以记录该重复扫描的信息，例如，该重复扫描到的设备的信息，例如该重复扫描到的设备的设备地址或设备号以及该重复扫描到的设备与其它设备之间的连接关系，但本发明实施例不限于此。
可选地，该当前处理单元可以具体确定所扫描到的设备是否被分配了设备地址；若确定该扫描到的设备被分配了设备地址，判断该扫描到的设备被分配的设备地址是否存在于该当前处理单元对应的设备地址集合中，其中，该当前处理单元对应的设备地址集合包括该当前处理单元已扫描到的至少一个设备中每个设备被分配的设备地址；若该扫描到的设备被分配的设备地址存在于该当前处理单元对应的设备地址集合中，确定该扫描到的设备曾经被该当前处理单元扫描过。
可选地，若确定该扫描到的设备没有被分配设备地址，则该当前处理单元可以为该扫描到的设备分配设备地址，并将该分配的设备地址添加到该当前处理单元对应的设备地址集合中。此时，可选地，该当前处理单元可以沿着下行方向，扫描与该扫描到的设备连接的设备。
可选地，若确定该扫描到的设备被分配的设备地址不存在于该当前处理单元对应的设备地址集合中，则该当前处理单元可以将该扫描到的设备被分配的设备地址添加到该当前处理单元对应的设备地址集合中。此时，可选地，该当前处理单元可以沿着下行方向，扫描与该扫描到的设备连接的设备。
S320，在扫描过程中记录该当前处理单元扫描到的设备之间的连接关系，并且根据所记录的连接关系，生成该当前处理单元的PCIe树。
作为另一个可选实施例，在S320之后，该方法300还可以包括：
发送校验消息，该校验消息的目的地址为该PCIe树中最外层叶节点设备的设备地址，其中，该PCIe树的根节点为该当前处理单元，该PCIe树的最外层叶节点为在PCIe树中与该当前处理单元距离最远的节点；
若该当前处理单元接收到该校验消息，确定该PCIe树建立成功。
作为另一个可选实施例，该方法300还可以包括：
若该当前处理单元未接收到该校验消息，重新建立该当前处理单元的PCIe树。
可选地，在重新建立该当前处理单元的PCIe树时，该当前处理单元可以确定在用于生成该PCIe树的扫描过程中重复扫描到的设备；为该重复扫描到的设备分配不同于该重复扫描到的设备当前的设备地址的新设备地址，并在该当前处理单元对应的设备地址集合中将该当前的设备地址替换为该新设备地址；以该重复扫描到的设备为起点，扫描与该重复扫描到的设备连接的设备，在扫描过程中记录扫描到的设备之间的连接关系，并且根据所记 录的扫描到的设备之间的连接关系，生成该当前处理单元的新的PCIe树。
因此，根据本发明实施例的设备扫描方法，该当前处理单元在确定扫描到的设备曾经被该当前处理单元扫描过时，则停止扫描该设备，并且返回至该扫描到的设备的上行设备进行扫描，其中，该扫描到的设备的上行设备可以具体为沿着上行方向与该扫描到的设备连接的设备，能够避免该当前处理单元在设备扫描过程中对同一设备进行多次扫描而陷入扫描死循环，以及避免由于重复扫描而建立错误的PCIe树，从而提高设备扫描的效率和准确度。
下面将结合具体例子对本发明实施例提供的设备扫描方法做更详细的说明。应注意，这些例子只是为了帮助本领域技术人员更好地理解本发明实施例，而非限制本发明实施例的范围。
图6是本发明另一实施例提供的设备扫描方法400的示意性流程图。
在S410中，处理单元扫描该处理单元连接的总线，并为扫描到的总线分配总线号。
具体地，可以将与该处理单元直接连接的总线定义为总线0，并且为扫描到的总线依次编号，但本发明实施例不限于此。
在S420中，处理单元确定当前总线的下行方向是否连接有未扫描到的桥设备。其中，该未扫描到的桥设备可以为该当前总线的下行方向连接的唯一桥设备，也可以为除了已经扫描到的该当前总线的下行方向连接的桥设备之外的其它桥设备。
可选地，如果当前总线的下行方向不存在未扫描到的桥设备，则该处理单元可以执行S480和S485。反之，则该处理单元可以执行S430和S440。
在S430中，该处理单元可以读取当前扫描到的桥设备的地址，并且在S440中确定读取到的该地址是否为默认地址。如果读取到的该地址为默认地址，则该处理单元可以执行S450和S410，即将读取到的该地址写入分配地址存储空间，并且继续扫描该当前总线的下行方向是否连接有未扫描到的桥设备。可选地，如果读取到的该地址不为默认地址，则该处理单元可以执行S460。
在S460中，该处理单元可以确定读取到的该地址是否是该处理单元分配的。具体地，该处理单元可以通过将该读取到的地址与分配地址存储空间中存储的设备地址进行比较，以确定该分配地址存储空间中是否存在该读取到的地址。如果存在，则表明读取到的该地址是由该处理单元分配的，反之， 则表明读取到的该地址不是由该处理单元分配的。可选地，该处理单元还可以通过其它方式确定读取到的该地址是否是由该处理单元分配的，本发明实施例对此不做限定。
可选地，如果该处理单元确定读取到的该地址是由本处理单元分配的，则该处理单元可以执行S470和S490，即中断对当前桥设备的进一步扫描，并且返回至该当前桥设备的上行总线进行扫描。如果该上行总线为总线0，则该处理单元可以结束本次设备扫描过程。如果该上行总线不为总线0，则该处理单元可以执行S410。
可选地，如果该处理单元确定读取到的该地址不是由本处理单元分配的，则该处理单元可以执行S410，即继续扫描该桥设备的下行总线，但本发明实施例不限于此。
应注意，图6的例子是为了帮助本领域技术人员更好地理解本发明实施例，而非要限制本发明实施例的范围。本领域技术人员根据所给出的图6的例子，显然可以进行各种等价的修改或变化，这样的修改或变化也落入本发明实施例的范围内。
应理解，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
还应理解，上文对实施例的描述着重于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，这里不再赘述。
还应理解，本文中术语“系统”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系；术语“多个”一般表示至少两个，即两个或三个以上。
本领域普通技术人员可以意识到，结合本文中所公开的实施例中描述的各方法步骤和单元，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各实施例的步骤及组成。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域普通技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应 认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本发明实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围 之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (11)

  1. 一种存储系统，其特征在于，所述系统包括：
    N个存储设备，每个存储设备包括至少一个处理单元、交换单元和至少一个存储单元，所述交换单元包括至少两个上行端口和至少两个下行端口，其中，
    所述至少两个上行端口包括至少一个第一上行端口和第二上行端口，所述至少一个处理单元连接至所述至少一个第一上行端口；
    所述至少两个下行端口包括至少一个第一下行端口和第二下行端口，所述至少一个存储单元连接至所述至少一个第一下行端口；
    所述N个存储设备呈环形连接，其中，
    在所述N个存储设备中，第i个存储设备的第二下行端口与第i+1个存储设备的第二上行端口连接，第N个存储设备中的第二下行端口与第1个存储设备的第二上行端口连接，i＝1，…，N-1，N≥2。
  2. 根据权利要求1所述的存储系统，其特征在于，所述至少两个下行端口中的任一下行端口与所述至少两个上行端口中的任一上行端口之间存在连接链路。
  3. 根据权利要求1或2所述的存储系统，其特征在于，所述至少两个上行端口中的每个上行端口具体为外围部件互连高速PCIe上行端口，所述至少两个下行端口中的每个下行端口具体为PCIe下行端口。
  4. 根据权利要求3所述的存储系统，其特征在于，所述呈环形连接的N个存储设备中每个存储设备的至少一个处理单元中的每个处理单元用于：
    扫描所述每个处理单元能够访问的设备；
    在扫描过程中记录所述每个处理单元扫描到的设备之间的连接关系，并且根据所记录的所述连接关系，生成所述每个处理单元的PCIe树。
  5. 根据权利要求4所述的存储系统，其特征在于，在扫描能够访问的设备时，当前进行设备扫描的当前处理单元具体用于：
    根据深度优先搜索算法DFS，扫描与所述当前处理单元直接或者间接连接的设备；
    若确定扫描到的设备曾经被所述当前处理单元扫描过，则确定所述扫描到的设备为与所述当前处理单元直接相连的交换单元，返回至与所述当前处理单元直接相连的交换单元的上行端口连接的上级交换单元，并扫描与所述 上级交换单元连接的未被扫描到的设备，当所述上级交换单元所连接的设备被扫描完后，则返回所述上级交换单元的上行端口连接的上级交换单元扫描未被扫描的设备，直到所述存储系统中所有交换单元所连接的设备全部被扫描完。
  6. 根据权利要求5所述的存储系统，其特征在于，每个处理单元对应设备地址集合，所述设备地址集合用于记录每个处理单元所扫描到的设备的设备地址；
    在确定扫描到的交换单元曾经被所述当前处理单元扫描过时，所述当前处理单元具体用于：
    确定所扫描到的设备是否被分配了设备地址；
    若确定所述扫描到的设备被分配了设备地址，判断所述扫描到的设备被分配的设备地址是否存在于所述当前处理单元对应的设备地址集合中；
    若所述扫描到的设备被分配的设备地址存在于所述当前处理单元对应的设备地址集合中，确定扫描到的设备曾经被所述当前处理单元扫描过。
  7. 根据权利要求6所述的存储系统，其特征在于，所述当前处理单元还用于：
    若确定所述扫描到的设备没有被分配设备地址，为所述扫描到的设备分配设备地址，并将所述分配的设备地址添加到所述每个处理单元对应的设备地址集合中；和/或
    若确定所述扫描到的设备被分配的设备地址不存在于所述当前处理单元对应的设备地址集合中，将所述扫描到的设备被分配的设备地址添加到所述当前处理单元对应的设备地址集合中。
  8. 一种设备扫描方法，其特征在于，应用于如权利要求1至3中任一项所述的存储系统，所述方法包括：
    所述N个存储设备中当前进行设备扫描的当前处理单元扫描所述当前处理单元能够访问的设备；
    在扫描过程中记录所述当前处理单元扫描到的设备之间的连接关系，并且根据所记录的所述连接关系，生成所述当前处理单元的PCIe树。
  9. 根据权利要求8所述的设备扫描方法，其特征在于，所述扫描所述当前处理单元能够访问的设备，包括：
    根据深度优先搜索算法DFS，扫描与所述当前处理单元直接或间接连接 的设备；
    若确定扫描到的设备曾经被所述当前处理单元扫描过，确定所述扫描到的设备为与所述当前处理单元直接相连的交换单元，返回至与所述当前处理单元直接相连的交换单元的上行端口连接的上级交换单元，并扫描与所述上级交换单元连接的未被扫描到的设备，当所述上级交换单元所连接的设备被扫描完后，则返回所述上级交换单元的上行端口连接的上级交换单元扫描未被扫描的设备，直到所述存储系统中所有交换单元所连接的设备全部被扫描完。
  10. 根据权利要求9所述的设备扫描方法，其特征在于，所述确定扫描到的设备曾经被所述当前处理单元扫描过，包括：
    确定所扫描到的设备是否被分配了设备地址；
    若确定所述扫描到的设备被分配了设备地址，判断所述扫描到的设备被分配的设备地址是否存在于所述当前处理单元对应的设备地址集合中，其中，所述当前处理单元对应的设备地址集合包括所述当前处理单元已扫描到的至少一个设备中每个设备被分配的设备地址；
    若所述扫描到的设备被分配的设备地址存在于所述当前处理单元对应的设备地址集合中，确定扫描到的设备曾经被所述当前处理单元扫描过。
  11. 根据权利要求10所述的设备扫描方法，其特征在于，所述所述扫描所述当前处理单元能够访问的设备，还包括：
    若确定所述扫描到的设备没有被分配设备地址，为所述扫描到的设备分配设备地址，并将所述分配的设备地址添加到所述当前处理单元对应的设备地址集合中；和/或
    若确定所述扫描到的设备被分配的设备地址不存在于所述当前处理单元对应的设备地址集合中，将所述扫描到的设备被分配的设备地址添加到所述当前处理单元对应的设备地址集合中。
PCT/CN2016/083701 2016-05-27 2016-05-27 存储系统及设备扫描方法 WO2017201742A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP16890129.6A EP3291096B1 (en) 2016-05-27 2016-05-27 Storage system and device scanning method
CN201680003259.9A CN107851058B (zh) 2016-05-27 2016-05-27 存储系统及设备扫描方法
PCT/CN2016/083701 WO2017201742A1 (zh) 2016-05-27 2016-05-27 存储系统及设备扫描方法
US15/797,687 US10437473B2 (en) 2016-05-27 2017-10-30 Storage system and method for scanning for devices

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/083701 WO2017201742A1 (zh) 2016-05-27 2016-05-27 存储系统及设备扫描方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/797,687 Continuation US10437473B2 (en) 2016-05-27 2017-10-30 Storage system and method for scanning for devices

Publications (1)

Publication Number Publication Date
WO2017201742A1 true WO2017201742A1 (zh) 2017-11-30

Family

ID=60412041

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/083701 WO2017201742A1 (zh) 2016-05-27 2016-05-27 存储系统及设备扫描方法

Country Status (4)

Country Link
US (1) US10437473B2 (zh)
EP (1) EP3291096B1 (zh)
CN (1) CN107851058B (zh)
WO (1) WO2017201742A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108052421A (zh) * 2017-12-13 2018-05-18 郑州云海信息技术有限公司 一种基于存储多重冗余架构的整机系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1832489A (zh) * 2006-04-19 2006-09-13 杭州华为三康技术有限公司 一种对目的磁盘进行访问的方法和扩展磁盘容量的系统
CN102508797A (zh) * 2011-10-27 2012-06-20 忆正存储技术（武汉）有限公司 闪存控制扩展模块、控制器、存储系统及其数据传输方法
WO2013103510A1 (en) * 2012-01-06 2013-07-11 Ge Intelligent Platforms, Inc. Ring topology for compute devices
CN103543961A (zh) * 2013-10-12 2014-01-29 浙江宇视科技有限公司 一种基于PCIe的存储扩展系统及存储扩展方法
CN103634277A (zh) * 2012-08-23 2014-03-12 深圳市腾讯计算机系统有限公司 一种共享内存的方法、服务器及系统

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11510929A (ja) * 1995-08-11 1999-09-21 シーメンス ニクスドルフ インフオルマチオーンスジステーメ アクチエンゲゼルシヤフト 周辺記憶装置を接続するための装置
US7835973B2 (en) * 1999-10-20 2010-11-16 Accenture Global Services Limited Spot market clearing
US7287678B2 (en) * 2001-11-16 2007-10-30 Vamco International Inc. Method and apparatus for determining and setting material release mechanism timing for a material feed mechanism
JP4503072B2 (ja) * 2004-05-14 2010-07-14 シーメンス メディカル ソリューションズ ユーエスエー インコーポレイテッド ３次元陽電子放出断層撮影におけるオンラインデータ取得装置
US20060041715A1 (en) * 2004-05-28 2006-02-23 Chrysos George Z Multiprocessor chip having bidirectional ring interconnect
US20080082969A1 (en) * 2006-04-04 2008-04-03 The Board Of Trustees Of The University Of Illinois Software Testing Technique Supporting Dynamic Data Structures
US20080209436A1 (en) * 2006-10-25 2008-08-28 Gul Agha Automated testing of programs using race-detection and flipping
US7991839B2 (en) * 2006-12-19 2011-08-02 International Business Machines Corporation Communication between host systems using a socket connection and shared memories
US20090077297A1 (en) * 2007-09-14 2009-03-19 Hongxiao Zhao Method and system for dynamically reconfiguring PCIe-cardbus controllers
WO2010044409A1 (ja) * 2008-10-15 2010-04-22 日本電気株式会社 マルチルートｐｃｉエクスプレススイッチ、その起動方法、及び、マルチルートｐｃｉマネージャプログラム
US7992044B2 (en) * 2008-12-05 2011-08-02 Oracle America, Inc. Method and system for platform independent fault management
CN101587453B (zh) * 2009-06-18 2012-01-04 成都市华为赛门铁克科技有限公司 数据备份处理方法、数据存储节点设备及数据存储装置
GB2473675B (en) * 2009-09-22 2011-12-28 Virtensys Ltd Switching method
CN101917492B (zh) * 2010-08-06 2013-06-05 北京乾唐视联网络科技有限公司 一种新型网的通信方法及系统
TWI439837B (zh) * 2011-08-26 2014-06-01 Richtek Technology Corp 穩壓電路控制器
US9305312B2 (en) * 2011-10-25 2016-04-05 Theodosios Kountotsis Express easy-pass checkout at grocery stores and retail establishments for preferred members
US8943258B2 (en) * 2011-11-04 2015-01-27 Lsi Corporation Server direct attached storage shared through virtual SAS expanders
US20130179621A1 (en) * 2012-01-06 2013-07-11 Glenn Willis Smith Extensible daisy-chain topology for compute devices
US9575806B2 (en) * 2012-06-29 2017-02-21 Intel Corporation Monitoring accesses of a thread to multiple memory controllers and selecting a thread processor for the thread based on the monitoring
KR102007368B1 (ko) * 2012-12-17 2019-08-05 한국전자통신연구원 Ｐｃｉ 익스프레스 스위치 및 이를 이용한 컴퓨터 시스템
CN103246545B (zh) * 2013-05-07 2014-05-14 中国人民解放军国防科学技术大学 面向多虚拟域可定制的PCIe外设设备树生成方法
US9519606B2 (en) * 2013-07-22 2016-12-13 GigaIO Networks, Inc. Network switch
WO2015045030A1 (ja) * 2013-09-25 2015-04-02 株式会社日立製作所 ドライブの増設可能な情報システムおよび該情報システムのバス番号割り当て方法
US9026687B1 (en) * 2013-12-26 2015-05-05 Lsi Corporation Host based enumeration and configuration for computer expansion bus controllers
US20150261709A1 (en) * 2014-03-14 2015-09-17 Emilio Billi Peripheral component interconnect express (pcie) distributed non- transparent bridging designed for scalability,networking and io sharing enabling the creation of complex architectures.
US9678910B2 (en) * 2014-04-25 2017-06-13 Liqid Inc. Power handling in a scalable storage system
US9419918B2 (en) * 2014-11-07 2016-08-16 Futurewei Technologies, Inc. Non-transparent bridge method and apparatus for configuring high-dimensional PCI-express networks
US10191882B2 (en) * 2015-06-29 2019-01-29 Futurewei Technologies, Inc. Method and system for aggregation-friendly address assignment to PCIe devices
US10061707B2 (en) * 2015-12-26 2018-08-28 Intel Corporation Speculative enumeration of bus-device-function address space
US10482049B2 (en) * 2017-02-03 2019-11-19 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Configuring NVMe devices for redundancy and scaling

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1832489A (zh) * 2006-04-19 2006-09-13 杭州华为三康技术有限公司 一种对目的磁盘进行访问的方法和扩展磁盘容量的系统
CN102508797A (zh) * 2011-10-27 2012-06-20 忆正存储技术（武汉）有限公司 闪存控制扩展模块、控制器、存储系统及其数据传输方法
WO2013103510A1 (en) * 2012-01-06 2013-07-11 Ge Intelligent Platforms, Inc. Ring topology for compute devices
CN103634277A (zh) * 2012-08-23 2014-03-12 深圳市腾讯计算机系统有限公司 一种共享内存的方法、服务器及系统
CN103543961A (zh) * 2013-10-12 2014-01-29 浙江宇视科技有限公司 一种基于PCIe的存储扩展系统及存储扩展方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3291096A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108052421A (zh) * 2017-12-13 2018-05-18 郑州云海信息技术有限公司 一种基于存储多重冗余架构的整机系统

Also Published As

Publication number Publication date
CN107851058A (zh) 2018-03-27
EP3291096A4 (en) 2018-05-30
CN107851058B (zh) 2021-02-12
EP3291096B1 (en) 2020-01-15
US10437473B2 (en) 2019-10-08
US20180046380A1 (en) 2018-02-15
EP3291096A1 (en) 2018-03-07

Similar Documents

Publication Publication Date Title
US20220183084A1 (en) Method for startup configuration of a bluetooth mesh network
US9477280B1 (en) Specification for automatic power management of network-on-chip and system-on-chip
CN107251486B (zh) 一种扩展联动的方法、装置及系统
CN106936739B (zh) 一种报文转发方法及装置
CN106031116A (zh) 一种ns与vnf的关联方法、装置及系统
US10846186B2 (en) Central processing unit CPU hot-remove method and apparatus, and central processing unit CPU hot-add method and apparatus
US9350655B2 (en) Vertical converged framework
US20200099585A1 (en) Software defined networking orchestration method and sdn controller
US20200334384A1 (en) Method of dynamically configuring fpga and network security device
CN105450588A (zh) 一种基于rdma的数据传输方法及rdma网卡
JP2020524869A (ja) 仮想マシン管理
CN104461698A (zh) 虚拟磁盘动态挂载的方法、管理装置及分布式存储系统
CN106484327B (zh) 一种数据处理方法及相关存储设备
CN107181636B (zh) 一种负载均衡系统中的健康检查方法及装置
CN112148219A (zh) 一种ceph类型的分布式存储集群设计方法及装置
EP3285173A1 (en) Cpu interconnecting apparatus, system and control method, control apparatus therefor
WO2017201742A1 (zh) 存储系统及设备扫描方法
US8194678B2 (en) Providing point to point communications among compute nodes in a global combining network of a parallel computer
CN109729731B (zh) 一种加速处理方法及设备
CN108076163B (zh) 移动自组网ip地址分配方法及装置
WO2017206562A1 (zh) 一种数据表的处理方法、装置及系统
US10700890B1 (en) Spanning tree protocol port-based link selection system
CN111107120B (zh) 一种Redis集群的构建方法及系统
CN107615872B (zh) 一种释放连接的方法、装置及系统
WO2022178675A1 (zh) 一种互联系统、数据传输方法以及芯片

Legal Events

Date Code Title Description
REEP Request for entry into the european phase

Ref document number: 2016890129

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE