JP6826300B1 - Information information system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent data communication from being erroneously performed to a platform on which a sudden restart is performed when a sudden restart is performed on one of the platforms, and to prevent a communication error on the other platform. Suppress the occurrence. An information processing system according to a first aspect of the present invention is communicably connected to a plurality of platforms and each of the plurality of platforms, and relays communication between the plurality of platforms using a transfer bus. It is provided with a relay device to be used. The relay device includes a driver status register having a storage area for each platform for storing a communication status indicating whether or not the platform can communicate. The platform updates the communication status stored in the storage area of the platform itself, and refers to the communication status stored in the storage area, and is capable of communicating among the plurality of platforms. It is equipped with a communication driver that communicates with. [Selection diagram] FIG. 11

Description

The present invention relates to an information processing system.

It is provided with a plurality of platforms and a relay device that is communicably connected to each of the plurality of platforms and relays communication between the plurality of platforms using a transfer bus such as a PCIe (Peripheral Component Interconnect Express) bus. Information processing systems are being developed.

Japanese Unexamined Patent Publication No. 2018-5569

By the way, in the above-mentioned information processing system, when communication is being performed between a plurality of platforms, if a communication error occurs due to a sudden restart due to a kernel panic or the like on one of the platforms, the communication state of the communication partner Can not be acquired, and communication between platforms will be started assuming that there is a communication partner, so communication errors may occur one after another even on other platforms where communication errors have not occurred. ..

In one aspect, the invention prevents accidental data communication to a platform that has undergone a sudden reboot in the event of a sudden reboot, and the other. The purpose is to suppress the occurrence of communication errors on the platform of.

The information processing system according to the first aspect of the present invention includes a plurality of platforms and a relay device that is communicably connected to each of the plurality of platforms and relays communication between the plurality of platforms using a transfer bus. , Equipped with. The relay device is provided in common with the plurality of platforms, and includes a driver status register for each platform, which has a storage area for storing communication status indicating whether or not the platform can communicate. The platform updates the communication status stored in the storage area of the platform itself, and among the storage areas of the driver status register , the communication stored in the storage area of another platform. A communication driver that communicates with the platform capable of communicating among the plurality of platforms by referring to the status is provided.

According to the first aspect of the present invention, when a sudden restart is performed on any platform, it is possible to prevent data communication from being erroneously performed on the platform on which the sudden restart is performed. , It is possible to suppress the occurrence of transfer bus errors.

FIG. 1 is a diagram showing an example of the overall configuration of the information processing system according to the present embodiment. FIG. 2 is a diagram showing an example of a software configuration of a platform in the information processing system according to the present embodiment. FIG. 3 is a diagram schematically showing an example of the hardware configuration of the PCIe bridge controller in the information processing system according to the present embodiment. FIG. 4 is a diagram showing a layer configuration of PCIe as an example of the present embodiment. FIG. 5 is a diagram illustrating the appearance of another processor from the processor in the information processing system according to the present embodiment. FIG. 6 is a diagram illustrating the appearance of another processor from the processor in the information processing system according to the present embodiment. FIG. 7 is a diagram for explaining an example of a data transfer method between platforms via a PCIe bridge controller in the information processing system according to the present embodiment. FIG. 8 is a diagram for explaining an example of a data transfer method between platforms via a PCIe bridge controller in the information processing system according to the present embodiment. FIG. 9 is a diagram for explaining an example of a register configuration of a PCIe bridge controller in the information processing system according to the present embodiment. FIG. 10 is a diagram showing an example of the configuration of a common register included in the register of the PCIe bridge controller in the information processing system according to the present embodiment. FIG. 11 is a sequence diagram showing an example of the flow of communication processing between a plurality of platforms in the information processing system according to the present embodiment.

Hereinafter, an example of the information processing system according to the present embodiment will be described with reference to the attached drawings.

FIG. 1 is a diagram showing an example of the overall configuration of the information processing system according to the present embodiment. As shown in FIG. 1, the information processing system 1 according to the present embodiment includes a plurality of platforms 2-1 to 2-8 and a PCIe bridge controller 3. Each of the plurality of platforms 2-1 to 2-8 is connected to the PCIe bridge controller 3.

In the following description, it is not necessary to distinguish between a plurality of platforms 2-1 to 2-8, and when an arbitrary platform is indicated, it is described as platform 2. Further, here, an example in which the information processing system 1 has eight platforms 2-1 to 2-8 will be described, but the present invention is not limited to this as long as it has a plurality of platforms 2.

Platform 2-1 includes processor 21-1. Similarly, platforms 2-2-2-8 also include processors 21-2-21-8, respectively.

Processors 21-1 to 21-8 may be provided by different manufacturers (vendors). For example, the processors 21-1,21-2,21-3,21-4,21-5,21-6,21-7,21-8 are companies A, B, C, and D, respectively. It is assumed that it is provided by company E, company F, company G, and company H.

In addition, hereinafter, the processors 21-1,21-2,21-3,21-4,21-5,21-6,21-7,21-8 are used as processors A, B, C, D, E, respectively. It may be called F, G, H. Further, different platforms may be connected to the EP mounted on the PCIe bridge controller 3. Further, a plurality of EPs may be connected to one platform 2, and the platform 2 side may communicate with the PCIe bridge controller 3 using a plurality of RCs.

In the following description, as the code indicating the processor, when it is necessary to specify one of a plurality of processors, the reference numerals 21-1 to 21-8, the reference numerals A to H, and the like are used, but when referring to an arbitrary processor, the reference numerals are used. Reference numeral 21 is used.

Platforms 2-1 to 2-8 are computer environments that perform arithmetic processing such as AI inference processing and image processing, and include a processor 21, a storage 23 and a memory (physical memory) 22 shown in FIG. 7.

In the platform 2, various functions are realized by the processor 21 executing a program stored in the memory 22 and the storage 23.

The storage 23 is a storage device such as a hard disk drive (HDD: Hard Disk Drive), an SSD (Solid State Drive), and a storage class memory (SCM: Storage Class Memory), and stores various data.

The memory 22 is a storage memory including a ROM (Read Only Memory) and a RAM (Random Access Memory). Various software programs and data for this program are written in the ROM of the memory 22. The software program on the memory 22 is appropriately read and executed by the processor 21. Further, the RAM of the memory 22 is used as a primary storage memory or a working memory.

The processor 21 controls the entire platform 2. The processor 21 may be a multiprocessor. The processor 21 includes, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be any one. Further, the processor 21 may be a combination of two or more types of elements of the CPU, MPU, DSP, ASIC, PLD, and FPGA.

FIG. 2 is a diagram showing an example of a software configuration of a platform in the information processing system according to the present embodiment. Although FIG. 2 shows only the software configuration of platforms 2-1 to 2-3, it is assumed that other platforms 2-4 to 2-8 also have the same software configuration.

As shown in FIG. 2, the platform 2-1 uses Windows (registered trademark) as an OS, and various programs are executed on this OS. Platforms 2-2 and 2-3 each use Linux (registered trademark) as an OS, and distributed processing programs (distributed processing A and B) are executed on this OS.

Each platform 2 is provided with a bridge driver 20, which communicates with the PCIe bridge controller 3 and other platforms 2 via the bridge driver 20. The communication method by the bridge driver 20 will be described later.

Each platform 2 includes a processor 21 and a memory (physical memory) 22 (see FIG. 7 and the like), and the processor 21 executes each function by executing an OS, various programs, drivers, etc. stored in the memory 22. Realize.

The processors 21 provided in each platform 2 may be provided by different vendors. In the example shown in FIG. 1, a platform having a plurality of RCs on at least a part of platforms 2 (for example, platform 2-7) (for example, an Intel x86 processor) may be used.

Further, each platform 2 is configured to be able to operate independently so as not to affect other driver configurations.

In the platform 2, a part of the storage area of the memory 22 is used as a communication buffer 221 in which the data transferred between the platforms 2 (between the processors 21) is temporarily stored, as will be described later with reference to FIG.

The PCIe bridge controller 3 realizes communication of data and the like between a plurality of platforms 2-1 to 2-8.

FIG. 3 is a diagram schematically showing an example of the hardware configuration of the PCIe bridge controller in the information processing system according to the present embodiment.

As shown in FIG. 3, the PCIe bridge controller 3 is a relay device that is communicably connected to each of the plurality of platforms 10 and relays communication between the plurality of platforms 10 using a PCIe bus (an example of a transfer bus). This is an example. In this embodiment, the PCIe bridge controller 3 has 8 channels of EP in one chip. As shown in FIG. 3, the PCIe bridge controller 3 includes a CPU 31, a memory 32, an interconnect 33, and a plurality of slots 34-1 to 3-4-8.

Devices configured to meet PCIe standards are connected to slots 34-1 to 3-4-8, respectively. In the information processing system 1 according to the present embodiment, the platform 2 is connected to each of the slots 34-1 to 3-4-8.

In the following description, as the code indicating a slot, reference numeral 34-1 to 3-4-8 is used when it is necessary to specify one of a plurality of slots, but reference numeral 34 is used when indicating an arbitrary slot. Is used.

Further, one platform 2 may be connected to one slot 34, or one platform 2 may be connected to a plurality of slots 34, and various modifications can be made.

By assigning one platform 2 to the plurality of slots 34, it is possible to make the platform 2 perform communication using a wide communication band.

Each slot 34 is connected to the interconnect 33 via an internal bus. Further, the CPU 31 and the memory 32 are connected to the interconnect 33. As a result, each slot 34, the CPU 31, and the memory 32 are connected to each other via the interconnect 33 so as to be able to communicate with each other.

The memory 32 is, for example, a storage memory (physical memory) including a ROM and a RAM. A software program related to data communication control and data for this program are written in the ROM of the memory 32. The software program on the memory 32 is appropriately read and executed by the CPU 31. Further, the RAM of the memory 32 is used as a primary storage memory or a working memory.

Further, in the PCIe bridge controller 3, a register 35 (see FIG. 7) is provided corresponding to each slot 34, and a storage area is provided for each slot in the BAR (Base Address Register) space of the register 35. That is, a storage area corresponding to each of slots # 0 to # 7 is provided in the BAR space of the register 35.

In the PCIe bridge controller 3, as will be described later, data transfer between the platforms 2 is performed using the storage area for each slot 34 in the BAR space.

The CPU 31 controls the entire PCIe bridge controller 3. The CPU 31 may be a multiprocessor. Instead of the CPU 31, any one of MPU, DSP, ASIC, PLD, and FPGA may be used. Further, the CPU 31 may be a combination of two or more types of elements of the CPU, MPU, DSP, ASIC, PLD, and FPGA.

Then, the CPU 31 executes the software program stored in the memory 32 to realize data transfer between the platforms 2 (between the processors 21) in the PCIe bridge controller 3.

The PCIe bridge controller 3 uses PCIe to speed up data transfer between platforms 2, and as shown in FIG. 1, each processor provided in each platform 2 operates as an RC, and between EPs operating as devices. Realize data transfer.

Specifically, in the information processing system 1, the processor of each platform 2 is operated as a PCIe RC as a data transfer interface. Further, for each platform 2 (processor 21), the PCIe bridge controller 3, that is, the slot 34 to which each platform 2 is connected is operated as an EP.

As a method of connecting the PCIe bridge controller 3 to the processor 21 as an EP, various known methods can be used.

For example, the PCIe bridge controller 3 connects to the processor 21 as an EP by notifying the processor 21 of a signal indicating that it functions as an EP when connecting to the platform 2.

In the PCIe bridge controller 3, data is tunneled by EPtoEP (End Point to End Point), and the data is transferred to a plurality of RCs. Communication between platforms 2 is logically connected when a PCIe transaction occurs, and when data transfer is not concentrated on one processor 21, data can be transferred in parallel between the respective platforms 2.

FIG. 4 is a diagram showing a layer configuration of PCIe as an example of the present embodiment. FIG. 4 shows an example of communication between the processor A of the platform 2-1 and the processor B of the platform 2-2, but it is assumed that the same communication is performed between the processors 21 of the other platforms 2. ..

In the source platform 2-1 the data generated in the RC processor A is sequentially transferred to the software, transaction layer, data link layer, and physical layer (PHY), and the PCIe bridge controller in the physical layer. Transferred to 3 physical layers.

In the PCIe bridge controller 3, the physical layer, the data link layer, the transaction layer, and the software are sequentially transferred, and the data is transferred by tunneling to the EP corresponding to the RC of the destination platform 2.

That is, in the PCIe bridge controller 3, data is tunneled between EPs (in other words, data received by one EP from one platform 2 is tunneled to another EP), so that one platform 2 is used. Data is transferred from the RC included in the above to the RC included in the other platform 2.

In the destination platform 2-2, the data transferred from the PCIe bridge controller 3 is sequentially transferred to the physical layer (PHY), the data link layer, the transaction layer, and the software, and is transferred to the destination platform 2-2. Is transferred to the processor B of.

In the information processing system 1, the communication between the processors 21 (between the platforms 2) is logically connected when a PCIe transaction occurs.

When data transfer from a plurality of other processors 21 is not concentrated on a specific processor 21 connected to one of the eight slots of the PCIe bridge controller 3, any plurality of different sets of each processor 21 Data may be transferred in parallel between them.

For example, when each of the processor B of the platform 2-2 and the processor C of the platform 2-3 tries to communicate with the processor A of the platform 2-1, the PCIe bridge controller 3 is the processor B and C. Process communication serially.

However, when different processors such as processor A-processor B, processor C-processor D, and processor E-processor F communicate with each other and communication is not concentrated on a specific processor, the PCIe bridge controller 3 may be used. Communication between each processor 21 is processed in parallel.

FIG. 5 is a diagram illustrating the appearance of another processor from the processor in the information processing system according to the present embodiment. Further, FIG. 6 is a diagram illustrating how the other processor looks from the processor in the information processing system according to the present embodiment.

Even when communication is being performed between the processors 21, only the PCIe bridge controller 3 can be seen from the OS (for example, Windows device manager) on each processor 21, and the other processors 21 to be connected are directly managed. No need. That is, the device driver of the PCIe bridge controller 3 may manage the processor 21 connected to the tip of the PCIe bridge controller 3.

Therefore, it is not necessary to prepare a device driver for operating the processors 21 of the source and the receiver, and communication between the processors 21 is performed only by performing communication processing with the PCIe bridge controller 3 by the driver of the PCIe bridge controller 3. Can be done.

FIG. 7 is a diagram for explaining an example of a data transfer method between platforms via a PCIe bridge controller in the information processing system according to the present embodiment. FIG. 7 describes a case where data is transferred from platform 2-1 connected to slot # 0 to platform 2-5 connected to slot # 4, but data transfer between other platforms 2 is also performed in the same manner. It shall be.

In platform 2-1 of the data transmission source, data transmitted by software or the like (hereinafter, may be referred to as transmission data) is loaded into the communication buffer 221 from the storage 23 provided in platform 2-1 (reference numeral P1). reference).

The software specifies the position information (for example, Offset / Length) of the area in which the transmission data is stored in the communication buffer 221 and the information of the transmission destination (for example, Slot / Offset), and is passed to the bridge driver 20. (See reference numeral P2).

In the source EP, the bridge driver 20 passes transmission data to the address of slot # 4 in the BAR space (reference numeral P3). As a result, in the PCIe bridge controller 3, transmission data is transmitted from the source port to the slot (destination slot) corresponding to the destination platform 2 by EPtoEP (see reference numeral P4). In the destination slot, the transmission data is stored in the storage area corresponding to slot # 4 in the BAR space of the register 35.

In the destination slot, for example, the bridge driver 20 transfers transmission data from the storage area corresponding to slot # 4 in the BAR space of register 35 to the communication buffer 221 and the transmission data is specified by offset in this communication buffer 221. It is stored in a predetermined area (see reference numeral P5).

In the destination platform 2, for example, the program reads the transmission data stored in the communication buffer 221 and moves it to the memory (local memory) 22 or the storage 23 (see reference numerals P6 and P7).

As described above, the transmission data is transferred from the source platform 2-1 to the destination platform 2-5.

FIG. 8 is a diagram for explaining an example of a data transfer method between platforms via a PCIe bridge controller in the information processing system according to the present embodiment. FIG. 8 shows a case where data is transferred from platform 2-1 connected to slot # 0 to platform 2-5 connected to slot # 4, but data is similarly transferred between other platforms 2. It shall be transferred.

The transmission source platform 2-1 stores the transmission data transmitted by software or the like in the memory area 36 of the platform 2-1 from the storage 23 or the like provided in the platform 2-1 (step S701). The memory area 36 may be a part of a communication buffer in which the transmitted data to be transferred is temporarily stored. The memory area 36 is an area provided on each of the platforms 2 in the same size as the memory 22 and the like. The memory area 36 is divided according to the number of slots 305. The divided storage area of the memory area 36 is associated with any slot 305. For example, the storage area indicated by slot # 0 in the memory area 36 is associated with the platform 2-1 connected to slot # 0, and the storage area indicated by slot # 4 in the memory area 36 is stored in slot # 4. It is associated with the connected platform 2-5. The platform 2-1 stores transmission data in an area (here, slot # 4) allocated to the transmission destination slot 305 in the memory area 36.

The source platform 2-1 having a function as a route complex has slot information indicating the destination slot 305 and the inside of the divided area in the destination memory area 36 based on the storage area of the memory area 36 of the platform 2. Acquires or generates the address information indicating the address in (step S702).

The source platform 2-1 passes the transfer data including the slot information, the address information, and the transmission data to the PCIe bridge controller 3 having the functions of a plurality of endpoints (step S703). As a result, the PCIe bridge controller 3 transfers the transfer data to the destination platform 2-4 by connecting the source slot 305 and the destination slot 305 by EPtoEP based on the slot information (step S704). ). The destination platform 2 stores transmission data (or transfer data) in the area indicated by the address information in the storage area corresponding to the communication buffer 221 of the destination platform 2 based on the slot information and the address information (step). S705).

On the destination platform 2-5, the program reads the transmission data stored in the communication buffer 221 and moves it to another area of the memory (local memory) 22 or the storage 23 (step S706, step S707).

As described above, data (transfer data) is transferred from the source platform 2-1 to the destination platform 2-5.

By the way, when communication is being performed between a plurality of platforms 2, if a communication error occurs due to a sudden restart due to a kernel panic or the like on one of the platforms 2, the other platform 2 will perform the communication. It becomes impossible to acquire the communication status of the platform 2 in which the error has occurred. Therefore, the other platform 2 determines that the communication partner platform 2 (the platform 2 in which the communication error has occurred) can communicate, and starts communication with the communication partner platform 2. Therefore, the other platform 2 A communication error may occur even in 2.

Therefore, in the present embodiment, the PCIe bridge controller 3 includes a register that stores a communication status indicating whether or not each platform 2 can communicate. Then, the platform 2 communicates with another platform 2 by referring to the communication status stored in the register included in the PCIe bridge controller 3.

This makes it possible to prevent communication with the platform on which a communication error has occurred due to a sudden restart due to a kernel panic or the like. As a result, it is possible to suppress the occurrence of a communication error on any of the platforms 2 in the information processing system 1 and the occurrence of a communication error on the other platform 2.

FIG. 9 is a diagram for explaining an example of a register configuration of a PCIe bridge controller in the information processing system according to the present embodiment. FIG. 9 shows an example of the configuration of the storage area (hereinafter referred to as a common register) corresponding to slot # 0 in the BAR space of the register 35 included in the PCIe bridge controller 3, but the other slots # 1 to # 7 are shown. The common register corresponding to each has the same configuration.

As shown in FIG. 9, the BAR space of each slot 34 of the register 35 included in the PCIe bridge controller 3 has a common BAR space among the registers 35 of the plurality of slots # 0 to # 8 (hereinafter, referred to as a common area). 900 is included.

Then, the common area 900 has a driver status register 901 as shown in FIG. Here, the driver status register 901 has a common register 902 to 902-7 which is a storage area provided for each slot 34 (in other words, the platform 2). Hereinafter, as the code indicating the common register, reference numerals 902 to 902-7 are used when it is necessary to specify one of a plurality of common registers, but reference numeral 902 is used when indicating an arbitrary common register. Is used.

Here, the common register 902 is a storage area for storing the communication status. Further, here, the communication status indicates whether or not the platform 2 corresponding to the common register 902 can communicate.

In the present embodiment, as shown in FIG. 9, the common register 902 of each slot 34 of the driver status register 901 is assigned a 32-bit register for each slot 34. Then, as shown in FIG. 9, the 24th bit among the bits of the common register 902 represents the communication status of the slot 34 (platform 2) to which the common register 902 is assigned.

Specifically, when the platform 2 corresponding to the common register 902 cannot communicate, the 24th bit (communication status) becomes 0. Further, when the platform 2 corresponding to the common register 902 can communicate, the 24th bit (communication status) is 1.

The bridge driver 20 (an example of a communication driver) of the platform 2 has a communication status stored in the common register 902 corresponding to the platform 2 itself among the common registers 902 to 902-7 (in the present embodiment, the common register 902). Updates the 24th bit) of the bits held by.

In the present embodiment, the bridge driver 20 of the platform 2 updates the communication status when the bridge driver 20 itself is unloaded. Specifically, the bridge driver 20 of the platform 2 has the bridge driver 20 itself unloaded (bridge driver) from the state in which the bridge driver 20 itself is loaded (the bridge driver 20 itself is valid). When transitioning to the (invalid state of 20 itself), the communication status stored in the common register 902 corresponding to the platform 2 itself is updated to 0.

Further, in the present embodiment, the bridge driver 20 of the platform 2 updates the communication status when the bridge driver 20 itself is loaded. Specifically, the bridge driver 20 of the platform 2 is loaded from the state in which the bridge driver 20 itself is unloaded (the state in which the bridge driver 20 itself is invalid) to the state in which the bridge driver 20 itself is loaded (bridge driver). When transitioning to the valid state of 20 itself), the communication status stored in the common register 902 corresponding to the platform 2 itself is updated to 1.

Further, the bridge driver 20 of the platform 2 refers to the communication status stored in the common registers 902 to 902-7 (in the present embodiment, the 24th bit of each of the common registers 902 to 902-7). Then, it communicates with the communicable platform 2 among the platforms 2-1 to 2-7. On the other hand, the bridge driver 20 of the platform 2 refers to the communication status stored in the common registers 902 to 902-7 (in this embodiment, the 24th bit of each of the common registers 902 to 902-7). Therefore, of the platforms 2-1 to 2-7, communication is not performed with the platform 2 which cannot communicate.

As a result, it is possible to prevent communication with the platform 2 in which a communication error has occurred due to a sudden restart due to a kernel panic or the like. As a result, it is possible to suppress the occurrence of a communication error on any of the platforms 2 in the information processing system 1 and the occurrence of a communication error on the other platform 2. Further, when the bridge driver 20 detects a restart of the platform 2 (for example, a sudden restart due to a kernel panic or the like) based on the communication status stored in the PCIe status register 903 described later, the bridge driver 20 is set in the driver status register 901. Among the stored communication statuses, the communication status of the platform 2 that has detected the restart (or the entire common register 902 of the platform 2 that has detected the restart) may be cleared. Then, the bridge driver 20 may not update the driver status register 901 by the CPU 31 of the PCIe bridge controller 3 during normal operation or the like in which no communication error has occurred.

Further, as shown in FIG. 9, among the bits of the common register 902, the 0th to 15th bits are bits indicating the platform 2 with which the slot 34 (platform 2) corresponding to the common register 902 can communicate. is there.

For example, the first bit of the common register 902-0 is a bit indicating whether or not communication is possible with the platform 2-2 connected to slot # 1. If it is 0, it indicates that communication is not possible. If it is 1, it means that communication is possible. Further, the second bit of the common register 902-0 is a bit indicating whether or not communication is possible with the platform 2-3 connected to slot # 2, and if it is 0, it indicates that communication is not possible. If it is 1, it means that communication is possible. Similarly, the third to fifteenth bits of the common register 902-0 also indicate whether or not communication is possible with the platform 2 connected to the slot 34 that matches the number of the bit.

Further, the common area 900 has a PCIe status register 903 as shown in FIG. Here, the PCIe status register 903 is a register different from the driver status register 901 and is a register that stores the communication status of the plurality of platforms 2-1 to 2-8.

The firmware executed by the CPU 31 (an example of the control unit) of the PCIe bridge controller 3 (hereinafter referred to as bridge firmware) is the communication state (hereinafter referred to as link state) with the platform 2 by the physical layer of the PCIe bridge controller 3. Detect changes. Then, the bridge firmware updates the communication status stored in the PCIe status register 903 based on the detection result of the change in the link state.

In this embodiment, the bridge firmware periodically detects a change in the link state. For example, when an interrupt is issued from the physical layer of the PCIe bridge controller 3, the bridge firmware cannot communicate from the state in which the link state can communicate with the platform 2 (hereinafter referred to as the connect state CTD). It detects whether the state (hereinafter referred to as the not-connect state NOC) has changed, or whether the communication state with the platform 2 has changed from the not-connect state NOC to the connect state CTD.

Then, when the bridge firmware detects that the link state has changed from the connect state CTD to the not connect state NOC, the link state has changed to the not connect state NOC among the communication status stored in the PCIe status register 903. The communication status of platform 2 is updated to the communication status indicating that communication is not possible. Further, when the bridge firmware detects that the link state has changed from the not-connect state NOC to the connect state CTD, the platform in which the link state has changed to the connect state CTD among the communication status stored in the PCIe status register 903. The communication status of 2 is updated to the communication status indicating that communication is possible.

If the common area 900 has a PCIe status register 903 in addition to the driver status register 901, the bridge driver 20 on platform 2-1 will refer to the communication status stored in the common register 902 and the PCIe status register 903. Communicate with platform 2.

As a result, it is possible to prevent communication with the platform 2 when the communication status stored in the common register 902 corresponding to the platform 2 itself cannot be updated due to a communication error or the like on the platform 2. As a result, it is possible to more effectively suppress the occurrence of a communication error on any of the platforms 2 in the information processing system 1 due to the occurrence of a communication error on the other platform 2.

In the present embodiment, an example in which the common area 900 has both the driver status register 901 and the PCIe status register 903 will be described. However, the common area 900 has at least one of the driver status register 901 and the PCIe status register 903. Just do it. When the common area 900 has only one of the driver status register 901 and the PCIe status register 903, the bridge driver 20 of the platform 2 has the communication status stored in one of the driver status register 901 and the PCIe status register 903. It is assumed that communication with the platform 2 is performed with reference to.

FIG. 10 is a diagram showing an example of the configuration of a common register included in the register of the PCIe bridge controller in the information processing system according to the present embodiment.

In this embodiment, the common register 902 is a 32-bit register as shown in FIG. Then, as shown in FIG. 10, in the common register 902, whether the 25th to 31st bits rsv are reserved for communication with the other platform 2 with respect to the platform 2 corresponding to the common register 902. It is a bit indicating whether or not.

Further, as shown in FIG. 10, in the common register 902, the 24th bit active is a bit indicating the communication status of the platform 2 connected to the slot 34 assigned to the common register 902. The bridge driver 20 of the platform 2 updates the 24th bit active to 0 when the bridge driver 20 transitions from a valid state to an invalid state. On the other hand, the bridge driver 20 of the platform 2 updates the 24th bit active to 1 when the bridge driver 20 transitions from the invalid state to the valid state.

Further, as shown in FIG. 10, in the common register 902, the 16th to 23rd bits mоde are bits indicating the operating state of the platform 2 corresponding to the common register 902.

For example, the bridge driver 20 of the platform 2 updates the bits mоde from the 16th to the 23rd to 0x00, which represents the initialization, when the initialization is performed on the platform 2.

Further, for example, when the initialization is completed on the platform 2, the bridge driver 20 of the platform 2 updates the bits mоde from the 16th to the 23rd to 0x01 indicating the completion of the initialization.

Further, for example, when the bridge driver 20 of the platform 2 is preparing for communication in which the platform 2 is changing from the invalid state to the valid state, the 16th to 23rd bits mоde are being prepared for communication. Update to 0x20 to represent.

Further, for example, the bridge driver 20 of the platform 2 updates the 16th to 23rd bits mоde to 0x21 indicating the valid state when the platform 2 is changed to the valid state.

Further, for example, when the platform 2 bridge driver 20 is preparing for the transition from the valid state to the invalid state, the 16th to 23rd bits mоde are updated to 0x30 indicating that the transition is being prepared. To do.

Further, for example, when the bridge driver 20 of the platform 2 is waiting for the transition completion when the platform 2 is transitioning from the valid state to the invalid state, the 16th to 23rd bits mоde represent the transition completion waiting. Update to 0x31.

Further, for example, the bridge driver 20 of the platform 2 updates the bits mоde from the 16th to the 23rd to 0x32 indicating the invalid state when the platform 2 is in the invalid state.

Further, as shown in FIG. 10, in the common register 902, the 0th to 15th bits active_nоde are bits indicating the platform 2 with which the platform 2 corresponding to the common register 902 can communicate. Here, the bit numbers match the numbers in slot 34 to which platform 2 is connected.

For example, the 0th bit corresponds to slot # 0 and indicates whether or not the platform 2 connected to the slot # 0 can communicate. Similarly, the first to fifteenth bits also match the numbers of the slots 34, indicating whether or not the platform 2 connected to each slot 34 can communicate.

In the present embodiment, the bridge driver 20 of the platform 2 refers to the 24th bit active of the common register 902 corresponding to each of the plurality of platforms 2, and has a number (0th to 0th to) that matches the number of the plurality of slots 34. The 15th) bit active_nоde is updated.

Then, when the bridge driver 20 of the platform 2 terminates communication with another platform 2 (in other words, when the bridge driver 20 itself is unloaded), the bridge driver 20 of the platform 2 is the platform 2 itself. It is checked whether or not the 24th bit active of the common register 902 of is 0.

Next, when the 24th bit active of the common register 902 of the platform 2 itself is 0, the bridge driver 20 of the platform 2 has all the bits active_nоde from the 0th to the 15th bits of the common register 902 of the platform 2 itself. Check if it is 0 or not. Then, when all the bits active_nоde from the 0th to the 15th common registers 902 of the platform 2 itself are 0, the bridge driver 20 of the platform 2 terminates the communication with the other platform 2.

FIG. 11 is a sequence diagram showing an example of the flow of communication processing between a plurality of platforms in the information processing system according to the present embodiment. Next, an example of the flow of communication processing between the plurality of platforms 2 in the information processing system 1 according to the present embodiment will be described with reference to FIG. Here, an example of the flow of communication processing between the processor A of the platform 2-1 and the processor E of the platform 2-5 will be described, but the communication processing is also performed between the other platforms 2 in the same manner. And.

The bridge firmware of the processor A of the platform 2-1 and the PCIe bridge controller 3 executes a link-up process for changing to a link state (connect state CTD) capable of communicating with each other (step S1101). In addition, the bridge firmware of the processor E of the platform 2-5 and the PCIe bridge controller 3 also executes the link-up process in the same manner.

The processor A of the source platform 2-1 communicates with another platform 2 at the time of initializing the platform 2 and uploading the network I / F (not shown) until the PCIe bridge controller 3 can be accessed. Wait without doing. When the PCIe bridge controller 3 becomes accessible, the processor A of the source platform 2-1 acquires the address of the BAR space (common area 900) of the register 35 of the PCIe bridge controller 3 (step S1102).

Next, the processor A of the platform 2-1 executes a confirmation process of whether or not the microcomputer in the PCIe bridge controller 3 is physically connected to the RC. In other words, the processor A of the platform 2-1 executes a confirmation process of whether or not the platform 2 facing the PCIe bridge controller 3 exists (step S1103).

At that time, the processor A of the platform 2-1 requests the PCIe bridge controller 3 to transmit the existence / absence report (step S1104). Here, the presence / absence report is a report showing the detection result of the link state with the platform 2 by the physical layer of the PCIe bridge controller 3. In other words, the presence / absence report is a report indicating whether or not the platform 2 facing the PCIe bridge controller 3 exists.

When the platform 2-1 requests transmission of the presence / absence report, the bridge firmware of the PCIe bridge controller 3 generates a presence / absence report (step S1105) and transmits the generated presence / absence report to the platform 201 (step S1106).

In the present embodiment, the bridge firmware of the PCIe bridge controller 3 acquires the PRSNT2 # signal output from the GPIO (General Purpose Input Output) terminal for each slot 34 of the PCIe bridge controller 3. Here, the PRSNT # signal is a signal indicating whether or not the RC of the platform 2 is physically connected to the microcomputer included in the PCIe bridge controller 3. In the present embodiment, the PRSNT # signal is in the case of a connected state CTD in which the RC of the platform 2 and the microcomputer of the PCIe bridge controller 3 are physically connected (that is, when the opposite platform 2 exists). It becomes Low, and in the case of a not-connect state NOC in which the RC of the platform 2 and the microcomputer of the PCIe bridge controller 3 are not physically connected (that is, when the opposite platform 2 does not exist), it becomes High. The bridge firmware of the PCIe bridge controller 3 updates the communication status of platforms 2-1 and 2-6 stored in the PCIe status register 903 based on the PRSNT2 # signal. Then, the bridge firmware of the PCIe bridge controller 3 refers to the communication status stored in the PCIe status register 903 to determine whether or not the opposite platform 2 exists, and based on the determination result, reports the existence or nonexistence. Generate.

Next, the processor A of the platform 2-1 confirms whether or not another platform 2-5 facing the PCIe bridge controller 3 has been linked up (step S step S1107). Specifically, when accessing the other platform 2-5 in an unaccessed state without accessing the other platform 2-5, the processor A periodically reports the existence / absence from the PCIe bridge controller 3. To the other platform 2-5 until it is confirmed that the other platform 2-5 is in normal operation (that is, there is a platform 2-5 facing the PCIe bridge controller 3). Suppress access to. Processor A periodically checks the presence / absence report to confirm that the other Platform 2-5 is not in normal operation (ie, there is no Platform 2-5 facing the PCIe Bridge Controller 3). Does not communicate with other platforms 2-5.

On the other hand, when it is confirmed that the other platform 2 is operating normally, the processor A of the platform 2-1 accesses the PCIe / Wrapper register (not shown) and is the port (EP) of the PCIe bridge controller 3. Determine if the link has been uploaded to the corresponding port.

For example, the processor A refers to the LINK_MONITOR register via the UART (Universal Asynchronous Receiver Transmitter) and determines whether or not the corresponding port, which is the port to which the other platforms 2-4 are connected, has been linked up. When the LINK_MONITOR register is "0", the bridge driver 20 determines that the link is down, and when the LINK_MONITOR register is "1", the bridge driver 20 determines that the link is up.

Then, when it is determined that the corresponding port has been linked up, the processor A of the platform 2-1 refers to the communication status of the destination platform 2-5 stored in the address space acquired in step S1102. By communicating with the PCIe bridge controller 3, communication with the processor E of the destination platform 2-5 is realized (step S1108).

As described above, according to the information processing system 1 according to the present embodiment, it is possible to prevent communication with the platform 2 in which a communication error has occurred due to a sudden restart due to a kernel panic or the like. It is possible to suppress the occurrence of a communication error on the other platform 2 due to the occurrence of a communication error on any of the platforms 2 in the information processing system 1.

In the above-described embodiment, PCIe has been described as an example of the I / O interface of each part, but the interface is not limited to PCIe. For example, the interface of each part may be a technology capable of transferring data between a device (peripheral controller) and a processor by a data transfer bus. The data transfer bus may be a general-purpose bus capable of transferring data at high speed in a local environment (for example, one system or one device) provided in one housing or the like. The interface may be either a parallel interface or a serial interface.

In the case of serial transfer, the I / O interface may be configured so that point-to-point connection is possible and data can be transferred on a packet basis. The I / O interface may have a plurality of lanes in the case of serial transfer. The layer structure of the I / O interface may include a transaction layer that generates and decodes packets, a data link layer that performs error detection, and a physical layer that converts serial and parallel. The I / O interface also includes a root complex at the top of the hierarchy with one or more ports, endpoints that are I / O devices, switches to increase ports, bridges that translate protocols, and so on. It's fine. The interface may transmit the data to be transmitted and the clock signal multiplexed by a multiplexer. In this case, the receiving side may separate the data and the clock signal by a demultiplexer.

1 Information processing system 2-1 to 2-8, 2 Platform 3 PCIe bridge controller 20 Bridge driver 21-11 to 21-8, 21 Processor 22 Memory 221 Communication buffer 23 Storage 31 CPU
32 Memory 33 Interconnect 34-1-3-4-8, 34 Slot 35 Register 900 Common Area 901 Driver Status Register 902-0-902-7,902 Common Register

Claims (3)

An information processing system including a plurality of platforms and a relay device that is communicably connected to each of the plurality of platforms and relays communication between the plurality of platforms using a transfer bus.
The relay device
A driver status register, which is provided in common to the plurality of platforms and has a storage area for storing communication status indicating whether or not the platform can communicate, is provided for each platform.
The platform
The communication status stored in the storage area of the platform itself is updated, and among the storage areas of the driver status register , the communication status stored in the storage area of another platform is referred to. A communication driver that communicates with the platform capable of communicating among the plurality of platforms.
Information processing system equipped with. The transfer bus is a bus PCI e (Peripheral Component Interconnect Express) ,
The relay device
A PCIe status register that is different from the driver status register and stores the communication status of the plurality of platforms, and a PCIe status register.
A control unit that detects a change in the communication state with the platform by the physical layer of the relay device and updates the communication status stored in the PCIe status register based on the detection result.
The information processing system according to claim 1, wherein the communication driver refers to the PCIe status register and the communication status stored in the storage area to communicate with the platform. The information processing system according to claim 1 or 2 , wherein the communication driver updates the communication status to a value indicating that the platform cannot communicate when the communication driver itself is unloaded.

Images