JP2014197402A - Information processor, control method and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the influence range of an error when an abnormality related to data transfer between nodes occurs.SOLUTION: An abnormality detection part 37c disposed in at least one node is configured to detect an abnormality on a data transfer path of data transfer using a shared memory region included in a storage device disposed in one node or the other node. An error information generation part is configured to generate error information on the basis of the abnormality detected by the abnormality detection part, and to generate interruption to a processor in an own node. The processor is configured to execute recovery processing on the basis of the error information. The error information generation part is configured to, when the node whose abnormality has been detected by the abnormality detection part is a remote node, and cache data of the remote node are not written back to the shared memory region of a home node due to the abnormality of the remote node, write data indicating the abnormality in a directory. The processor is configured to, when executing recovery processing, delete the data indicating the abnormality from the directory.

Description

  The present invention relates to an information processing apparatus, a control method, and a control program.

  Conventionally, an SMP (Symmetric MultiProcessor) technique in which a plurality of arithmetic processing devices share a main storage device is known. As an example of an information processing system to which such SMP technology is applied, a plurality of nodes each having an arithmetic processing unit and a main storage device are connected by the same bus, and each arithmetic processing unit is connected to each main processing unit via the bus. There is an information processing system that shares a storage device.

  In such an information processing system, for example, a snoop method is used to hold the coherency of data cached by the arithmetic processing unit of each node.

  Further, in a system that uses a shared memory as a means for communicating data between nodes, there is a technique for converting data to be transmitted into data indicating abnormality and transmitting the converted data when a node abnormality is detected. In this technique, a node that has received data indicating abnormality discards the received data.

  Further, there is a technique for changing a communication path and continuing processing in a system in which a plurality of nodes are connected by a crossbar switch when packet communication stays. In this technique, a request transmitted by a node is transmitted from the crossbar switch to its own node and other nodes. In this technique, the node that transmitted the request measures the time from when the request is transmitted until it is received, and detects a timeout, thereby determining that packet communication has been retained.

  Also, in a system in which a plurality of nodes are connected by a crossbar switch, when the data transmitted from the node is interrupted, if the interrupted time is a predetermined time or more, dummy data including data indicating abnormality is received on the receiving node There is technology to send to.

JP 2004-013723 A JP 2002-366451 A JP-A-11-168502

  However, the above-described technique has a problem that the influence range of errors cannot be suppressed when an abnormality relating to data transfer between nodes occurs.

  For example, in the above information processing system that uses the snoop method to maintain coherency of cached data, the following can be considered. That is, when a failure occurs in a certain node (node goes down) and communication abnormality occurs between the nodes, it is conceivable to bring down all the nodes in order to maintain the coherency of the cached data. In this case, the error influence range extends to all nodes.

  An object of one aspect of the present invention is to suppress an error influence range when an abnormality relating to data transfer between nodes occurs.

  In one aspect, there is provided an information processing apparatus having a plurality of nodes each including a storage device that can be partially set as a shared memory area, and an interconnect that connects the plurality of nodes. A detector, an error information generator, and a processor; The abnormality detection unit detects an abnormality in data transfer between a plurality of nodes or an abnormality in another node. The error information generation unit generates error information based on the abnormality detected by the abnormality detection unit, and generates an interrupt to the processor in the node that issued the data transfer request. When receiving the interrupt, the processor performs recovery processing based on the error information generated by the error information generation unit. The error information generation unit detects an abnormality when the node where the abnormality is detected by the abnormality detection unit is a remote node, and the cache data of the remote node is not written back to the shared memory area of the home node due to the abnormality of the remote node. Write the indicated data to the directory. The processor deletes data indicating an abnormality from the directory when executing the recovery process.

  According to one embodiment, when an abnormality relating to data transfer between nodes occurs, the influence range of an error can be suppressed.

FIG. 1 is a schematic diagram illustrating an example of an information processing system according to the first embodiment. FIG. 2 is a diagram for explaining the functional configuration of the building block according to the first embodiment. FIG. 3 is a diagram illustrating an example of a memory map when another node attaches to a node to which a shared memory is allocated. FIG. 4 is a diagram for explaining a functional configuration of the CPU according to the first embodiment. FIG. 5 is a diagram for explaining an example of the data configuration of the node map according to the first embodiment. FIG. 6 is a diagram for explaining an example of the data structure of a directory. FIG. 7 is a diagram for explaining a packet transmitted by the CPU according to the first embodiment. FIG. 8 is a diagram illustrating an example of a transmission packet. FIG. 9 is a diagram illustrating an example of another configuration of the abnormality detection unit. FIG. 10 is a diagram illustrating an example of a data configuration of “TLP header”. FIG. 11 is a diagram for explaining a specific example of the operation of the PCIe control unit that has received the “poisoned TLP” packet. FIG. 12 is a schematic diagram illustrating an example of a process in which the CPU according to the first embodiment transmits a request. FIG. 13 is a diagram for explaining an example of processing executed when the CPU according to the first embodiment receives a packet. FIG. 14 is a diagram for explaining an example of processing in which the I / O device according to the first embodiment transmits a request. FIG. 15 is a diagram for explaining an example of processing in which the I / O device according to the first embodiment receives a response. FIG. 16 is a diagram for explaining an example of processing executed when a data transfer abnormality occurs between a node and a node having a memory to be accessed by the node. FIG. 17 is a diagram for explaining an example of processing executed when a data transfer abnormality occurs between a node and a node having a memory to be accessed by the node. FIG. 18 is a flowchart for explaining the flow of processing for controlling a shared area. FIG. 19 is a flowchart for explaining shared memory allocation processing. FIG. 20 is a flowchart for explaining the shared memory attach process. FIG. 21 is a flowchart for explaining processing in which an application uses a shared memory. FIG. 22 is a flowchart for explaining a shared memory detach process between nodes. FIG. 23 is a flowchart for explaining the inter-node shared memory release process. FIG. 24 is a flowchart for explaining the flow of processing for issuing a request. FIG. 25 is a flowchart for explaining the flow of processing executed when a request is received. FIG. 26 is a flowchart for explaining the flow of processing executed when the CPU receives a response. FIG. 27 is a flowchart for explaining the flow of processing executed when the CPU transmits a request. FIG. 28 is a flowchart for explaining a flow of processing executed when the PCIe control unit transmits a read request. FIG. 29 is a flowchart for explaining the flow of processing executed when the PCIe control unit transmits a write request. FIG. 30 is a flowchart for explaining the flow of trap processing executed by the OS when a trap occurs. FIG. 31 is a diagram illustrating an example of the data structure of the handler table. FIG. 32 is a flowchart for explaining the flow of processing executed by the signal handler notified of the signal. FIG. 33 is a flowchart for explaining another processing flow executed by the signal handler notified of the signal. FIG. 34 is a schematic diagram of an information processing system for explaining an example of a method for detecting an abnormality of a node. FIG. 35 is a flowchart for explaining the flow of processing when a method different from the abnormality detection method of the first embodiment is used. FIG. 36 is a flowchart for explaining the flow of processing when the cluster management manager detects an abnormality. FIG. 37 is a diagram for explaining an example of the information processing system. FIG. 38 is a diagram for explaining an example of a partition. FIG. 39A is a diagram for describing an example of a node map stored by the CPU of partition #A. FIG. 39B is a diagram for describing an example of a node map indicating the partition #A. FIG. 39C is a diagram for describing an example of a node map indicating the partition #B.

  The information processing apparatus, control method, and control program according to the present application will be described below with reference to the accompanying drawings.

  In the following first embodiment, an example of an information processing system having a plurality of nodes will be described with reference to FIG. FIG. 1 is a schematic diagram illustrating an example of an information processing system according to the first embodiment. In the example illustrated in FIG. 1, the information processing system 1 includes an XB (crossbar switch) 2 and a plurality of building blocks 10 to 10 e. XB2 is a crossbar switch that connects the building blocks 10 to 10e to each other. Moreover, XB2 has a service processor (not shown) which becomes a master of each service processor included in each of the building blocks 10 to 10e described later. In the case of a small-scale configuration in which a small number of nodes are connected, building blocks may be directly connected without using XB2.

  The building block 10 includes a plurality of CPUs (Central Processing Units) 21 to 21c and a plurality of memories 22 to 22c. Further, the other building blocks 10a to 10e are assumed to have the same configuration as the building block 10, and the following description is omitted. In the example shown in FIG. 1, the description of the CPUs 21b and 21c and the memories 22b and 22c is omitted. In each building block, an I / O (Input Output) device (not shown) is provided. In this embodiment, cache coherence control between CPUs is realized by a directory system, and a home CPU (to be described later) having data on a memory manages the corresponding directory.

  Each of the building blocks 10 to 10e operates the OS independently. That is, each of the CPUs 21 to 21c executes the OS independently. The OS executed by each building block 10 to 10e operates in a different partition for each building block. Here, the partition indicates a group of building blocks in which the same OS operates and operates as one system as viewed from the operating OS.

  For example, the building blocks 10 to 10a operate as the partition #A, and the building blocks 10b to 10d operate as the partition #B. In such a case, the OS that the building block 10 operates identifies that the building blocks 10 and 10a are operating as one system, and the OS that the building block 10b operates is that the building blocks 10b to 10d are one. Identified as operating as one system.

  Next, a configuration example of the building block will be described with reference to FIG. FIG. 2 is a diagram for explaining the functional configuration of the building block according to the first embodiment. In the example illustrated in FIG. 2, the building block 10 includes a node 20, a service processor 24, XB connection units 27 and 27 a, and a PCIe (Peripheral Component Interconnect Express) connection unit 28.

  The node 20 includes a plurality of CPUs 21 to 21 c, a plurality of memories 22 to 22 c, and a communication unit 23.

  The service processor 24 includes a control unit 25 and a communication unit 26. In the example shown in FIG. 2, the CPUs 21 to 21 c are connected to each other and to the communication unit 23. The memories 22 to 22c are connected to the CPUs 21 to 21c. The service processor 24 is connected to an administrator terminal of a server via a network line such as a LAN (Local Area Network) not shown, and receives various instructions from the administrator terminal to change various settings in the node or the building block 10. Control.

Further, each of the CPUs 21 to 21c is connected to the XB connection unit 27 or the XB connection unit 27a. The XB connection units 27 and 27a may be the same XB connection unit. Further, each of the CPUs 21 to 21 c is connected to the PCIe connection unit 28. The communication unit 23 is connected to a communication unit 26 included in the service processor 24. The control unit 25, a communication unit 26, a communication unit 23, each CPU21~21c, for ^example, are connected by ^{I 2 C (Inter-Integrated Circuit} ).

  The CPUs 21 to 21c are arithmetic processing devices that execute applications. The CPUs 21 to 21c are connected to memories 22 to 22c, respectively. Further, when the running application requests allocation of the shared memory, the CPUs 21 to 21 c communicate with each other and allocate the shared memory used by the application. In addition, each of the CPUs 21 to 21c uses a part of the memory included in each of the memories 22 to 22c and the other building blocks 10a to 10e as a shared memory.

  FIG. 3 is a diagram illustrating an example of a memory map when another node attaches to a node to which a shared memory entity is assigned. In the example of FIG. 3, when a shared memory is allocated to a node that owns a memory entity (referred to as a home node), the home node divides it into a certain area size. This division unit is referred to as a segment, but it is not essential to divide it into segments. When another node requests allocation of the shared memory owned by the home node, that is, by attaching, the shared memory of the home node can be used. A memory area used by the remote node is referred to as a shared memory image area. This shared memory image area may be attached by a single remote node or a plurality of remote nodes.

  Returning to FIG. 2, each of the CPUs 21 to 21 c has a node map in which a physical address is associated with a CPU ID (identification) that is an identifier of a CPU connected to a memory to which the physical address is allocated. This CPUID is uniquely determined by the system 1 and does not overlap.

  Each CPU 21-21c communicates with other CPUs using a node map. For example, when the CPU ID associated with the physical address to be accessed indicates a CPU different from the CPUs 21 to 21c, the CPU 21 receives another CPU via the XB connection unit 27 or the XB connection unit 27a and XB2. Send a memory access request to the node. Further, when the CPU 21 receives a request for the memory connected to itself from another node, the CPU 21 reads the data to be requested from the memory 22 connected to itself and transmits it to the request source. The other CPUs 21a to 21c perform the same processing.

  In addition, each of the CPUs 21 to 21c performs a function of executing processing similar to that of a conventional CPU, such as performing address conversion using a TLB (Translation Lookaside Buffer) and executing trap processing when a TLB miss occurs. Have.

  In addition, each of the CPUs 21 to 21c detects an abnormality (error) in data transfer between nodes. An example of a method for detecting an abnormality in data transfer between nodes will be described. For example, each of the CPUs 21 to 21c measures the time after transmitting the request. Subsequently, before sending a request (request) and before receiving a response (response), if the time after sending the request exceeds a predetermined time, a time-out has occurred, so each of the CPUs 21 to 21c. Detects an abnormality in data transfer between nodes. Each of the CPUs 21 to 21c also detects an abnormality when a negative response is returned from another node in response to the request.

  And when abnormality is detected, each CPU21-21c performs various processes. For example, when dirty cache data written back to the shared memory of each of the memories 22 to 22c is not written back by the node that caused the abnormality (for example, a down node), each of the CPUs 21 to 21c Performs the following process. That is, each of the CPUs 21 to 21c has a value indicating an abnormality in which the dirty cache data written back to the shared memory of each of the memories 22 to 22c is not written back by the down node in the directory indicating the cache state. Write. In addition, when dirty cache data that is written back to the shared memory of each of the memories 22 to 22 c is not written back by the node that has detected the abnormality and caused the occurrence of the abnormality, each of the CPUs 21 to 21 c The following processing can also be performed. That is, each of the CPUs 21 to 21c can write data indicating an error state in the shared memory area of each of the memories 22 to 22c where the cache data is written back by the down node. These processes can indicate that the data in the shared memory that has not been written back is not normal.

  In a case where a write back request does not reach the home CPU described later, the local CPU described later recognizes a transaction failure by detecting a timeout described later from the home CPU. In this case, the local CPU discards the corresponding data. Since the local CPU keeps the data taken out to the cache on the directory managed by the home CPU, a “MoveOut” request is generated from the home CPU. This “MoveOut” request causes a cache miss in the local CPU, but returns an error response to this “MoveOut” request, abnormally updates the state of the directory in the home CPU, that is, indicates the error state described above. Write data.

  Further, when the physical address (PA) of the shared memory of the down node is written in the error occurrence address register, a trap (interrupt) is generated. That is, when a physical address is written in the error occurrence address register, each of the CPUs 21 to 21c performs a trap process. In this trap processing, for example, a signal is transmitted to the signal handler.

  Here, the signal handler is activated when a signal is received. Various processes are performed in the process by the signal handler. For example, in the processing by the signal handler, when the “shared memory entity” exists in the down node, the “shared memory image” is detached, that is, the shared memory of the down node is deallocated. Further, in the processing by the signal handler, recovery processing is performed on the shared memory of the memory of the node that has detected that another node is down. As an example of the recovery process, there is a process of clearing a value indicating an abnormality in which the cache data written back to the shared memory is not written back from the directory. Another example of the recovery process is a process of clearing data indicating an error state from the shared memory.

  The memories 22 to 22c are memories shared by all the CPUs included in the information processing system 1. In the information processing system 1, the service processors of the building blocks 10 to 10e allocate physical addresses mapped to the same physical address space to the memories of all the building blocks 10 to 10e. That is, a physical address having a non-overlapping value is assigned to at least a memory used as a shared memory among all the memories included in the information processing system 1.

  In addition, in the memories 22 to 22c, a part of the storage area is a shared area shared by all the CPUs of the information processing system 1, and the other parts of the CPUs 21 to 21c that access the CPU 21 to 21c receive kernel data and user data. A local area to be stored and an I / O area used by an I / O device unrelated to exchanges with other nodes via a shared memory are used.

  The control unit 25 controls the building block 10. For example, the control unit 25 executes power management of the building block 10, monitoring and control of abnormality in the building block 10, and the like. Moreover, the control part 25 is connected with the service processor which the other building blocks 10a-10e have by the network not shown, and performs the control linked between each building block 10a-10e. Further, the control unit 25 can communicate with the OS executed by each of the CPUs 21 to 21c.

  The control unit 25 accesses the CPUs 21 to 21 c via the communication unit 26 and the communication unit 23. And the control part 25 performs update, control, etc. of the node map which each building block 10-10e has.

  The communication unit 23 transmits a control signal from the control unit 25 to the CPUs 21 to 21c via the communication unit 26 included in the service processor 24. In addition, the communication unit 26 transmits a control signal from the control unit 25 to the communication unit 23 included in the node 20. The XB connection units 27 and 27a connect the CPUs 21 to 21c to the XB 2 and relay communication between the CPUs of the building blocks 10 to 10e. The PCIe connection unit 28 relays access to the I / O device by the CPUs 21 to 21c.

  Next, the functional configuration of each of the CPUs 21 to 21c will be described with reference to FIG. FIG. 4 is a diagram for explaining a functional configuration of the CPU according to the first embodiment. Note that the CPUs 21a to 21c have the same functions as the CPU 21, and a description thereof will be omitted. In the example illustrated in FIG. 4, the description of the connection units 23 and 26 that connect the service processor 24 and the CPU 21 is omitted.

  In the example illustrated in FIG. 4, the CPU 21 includes an arithmetic processing unit 30, a router 40, a memory access unit 41, and a PCIe control unit 42. The arithmetic processing unit 30 includes an arithmetic unit 31, an L1 (Level 1) cache 32, an L2 (Level 2) cache 33, a node map 34, an address conversion unit 35, a cache directory management unit 36, a packet control unit 37, and an error occurrence. An address register 96 and a trap generator 97 are provided. Note that the router 40, the memory access unit 41, the PCIe control unit 42, and the like may not be included in the same CPU 21.

  The packet control unit 37 includes a packet generation unit 37a, a packet reception unit 37b, and an abnormality detection unit 37c. The PCIe control unit 42 includes a request generation unit 42a, a PCIe bus control unit 42b, and an abnormality detection unit 42c.

  First, the node map 34 included in the arithmetic processing unit 30 will be described. The node map 34 is a table in which a physical address and a CPU ID of a CPU connected to a memory having a storage area indicated by the physical address are registered in association with each other. Hereinafter, examples of information registered in the node map 34 will be described with reference to the drawings.

  FIG. 5 is a diagram for explaining an example of the data configuration of the node map according to the first embodiment. In the example illustrated in FIG. 5, the node map 34 has an entry that associates the registered contents of the items “address”, “valid”, “node ID”, and “CPUID”. Here, information indicating an address area including a plurality of consecutive physical addresses is stored in the “address” item of each entry.

  For example, the information processing system 1 divides a physical address space allocated to all memories into equal-sized address areas, and assigns identifiers such as # 0, # 1, and # 2 to each address area. Then, the information processing system 1 registers an identifier indicating each address area in the “address” of each entry included in the node map 34. The example of FIG. 5 shows a case where the identifier of # 0 is registered in the “address” item of the first entry. The example of FIG. 5 shows a case where the identifier of # 1 is registered in the “address” item of the second entry. The example of FIG. 5 shows a case where the identifier of # 2 is registered in the “address” item of the third entry.

  In the “valid” item of each entry, a valid bit indicating whether or not the storage area indicated by the physical address can be accessed is registered. For example, when the storage area indicated by the physical address is a shared area shared by the CPUs, a valid bit (for example, “1”) indicating that access can be performed is registered. The example of FIG. 5 shows a case where the valid bit “1” is registered in the “valid” item of the first entry. The example of FIG. 5 shows a case where the valid bit “1” is registered in the “valid” item of the second entry. The example of FIG. 5 shows a case where a valid bit “0” indicating that the storage area indicated by the physical address cannot be accessed is registered in the “valid” item of the third entry.

  In addition, an identifier indicating a node in which a memory to which a physical address is allocated exists is registered in the “node ID” item of each entry. The example of FIG. 5 shows a case where the identifier “1” indicating the node is registered in the “node ID” item of the first entry. The example of FIG. 5 shows a case where the identifier “1” indicating the node is registered in the “node ID” item of the second entry.

  In addition, an identifier indicating the CPU connected to the memory to which the physical address is allocated is registered in the “CPUID” item of each entry. That is, the node map 34 indicates which CPU is connected to which physical address to be accessed is a physical address of a memory. The example of FIG. 5 illustrates a case where the identifier “4” indicating the CPU is registered in the item “CPUID” of the first entry. The example of FIG. 5 shows a case where the identifier “5” indicating the CPU is registered in the “CPUID” item of the second entry.

  The node map 34 may register information in any format other than the present embodiment as long as it can indicate which CPU the physical address to be accessed is connected to.

  Returning to FIG. 4, the calculation unit 31 is a core of a calculation device that executes calculation processing and executes an OS (Operating System) and applications. In addition, when performing data reading (reading) or writing (writing), the arithmetic unit 31 addresses a logical address (Virtual Address; VA) of a storage area in which data to be read or written is stored. The data is output to the conversion unit 35.

  The L1 cache 32 is a cache memory that temporarily stores data frequently used in the calculation unit 31. Similar to the L1 cache 32, the L2 cache 33 temporarily stores frequently used data. However, the L2 cache 33 is a cache memory having a larger storage capacity than the L1 cache 32 and a low speed for reading and writing data. Here, the directory information is stored in the cache directory management unit 36 and is information indicating the CPU that caches the data stored in each storage area of the memory 22 and the update status of the cached data. In the following description, “directory information” may be simply referred to as “directory”. This directory-based cache memory management method is a technique often used in a ccNUMA (Cache Coherent Non-Uniform Memory) system, but both the ccNUMA technique and the directory technique are well-known techniques and will not be described in detail here. In FIG. 4, the directory 36 a is built in the cache directory management unit 36, but directory information can also be recorded in a part of the storage area of the memory 22.

  The address conversion unit 35 has a TLB 35a. An entry that associates a logical address with a physical address is registered in the TLB 35a. The address conversion unit 35 converts the logical address output from the calculation unit 31 into a physical address using the TLB 35a. For example, the address conversion unit 35 searches the physical address corresponding to the logical address acquired from the calculation unit 31 from the TLB 35a, and when the physical address is obtained as a result of the search, the obtained physical address is managed by the cache directory management. To the unit 36. Note that the address conversion unit 35 executes trap processing when a TLB miss occurs. Here, the system software such as the OS registers the TLB missed physical address and logical address pair in the TLB 35a. However, for a physical address for which registration of such a set is prohibited, even if a TLB miss occurs, a set of a physical address and a logical address is not registered in the TLB 35a by system software such as the OS.

  Here, the OS, the address conversion unit 35, and the like execute the following processing when an allocation to the shared memory is requested from an application executed by the calculation unit 31. That is, when a TLB miss occurs, system software such as the OS registers an entry in the TLB 35a. If no TLB miss occurs, the entry has already been registered in the TLB 35a, so the address conversion unit 35 performs conversion from a logical address to a physical address.

  Further, the address conversion unit 35 and the OS execute the following process when an application or OS requests local area allocation. That is, when a TLB miss occurs, system software such as the OS registers an entry in the TLB 35a that associates a logical address for accessing the local area dedicated to the CPU 21 with a physical address in a range allocated to the local area. .

  Further, the OS or the like deletes the entry including the physical address of the shared memory of the node where the abnormality has occurred from the TLB 35a.

  The cache directory management unit 36 has a directory 36a. The cache directory management unit 36 manages cache data and directories. The cache directory management unit 36 acquires a physical address obtained by converting the logical address output from the calculation unit 31 from the address conversion unit 35.

  When the cache directory management unit 36 acquires a physical address from the address conversion unit 35, the cache directory management unit 36 performs the following processing. That is, the cache directory management unit 36 uses the directory 36 a to determine whether or not the data stored at the acquired physical address is cached in the L1 cache 32 and the L2 cache 33.

  When the cache directory management unit 36 determines that the data stored at the acquired physical address is cached, the cache directory management unit 36 outputs the cached data to the calculation unit 31. Further, when the data stored at the acquired physical address is not cached in the L1 cache 32 and the L2 cache 33, the cache directory management unit 36 executes the following processing. First, the cache directory management unit 36 refers to the node map 34 and identifies an entry in a range including the acquired physical address. Then, the cache directory management unit 36 determines whether or not the CPUID of the identified entry is the CPUID of the CPU 21. Thereafter, when the CPUID of the identified entry is the CPUID of the CPU 21, the cache directory management unit 36 outputs a physical address to the memory access unit 41.

  Further, when the CPUID of the identified entry is not the CPUID of the CPU 21, the cache directory management unit 36 executes the following process. That is, the cache directory management unit 36 acquires the CPU ID and node ID of the identified entry. Then, the cache directory management unit 36 outputs the acquired CPU ID and physical address to the packet control unit 37.

  In addition, when the cache directory management unit 36 acquires the data stored in the storage area indicated by the output physical address from the memory access unit 41 or the packet control unit 37, the acquired data is stored in the L1 cache 32 and the L2 cache. 33. Then, the cache directory management unit 36 outputs the data cached in the L1 cache 32 to the calculation unit 31.

  In addition, when the cache directory management unit 36 acquires a physical address from the packet control unit 37, that is, when a physical address that is a target of a memory access request from another CPU or I / O device is acquired, The following processing is executed. That is, the cache directory management unit 36 refers to the node map 34 to determine whether or not the acquired physical address is a physical address assigned to the local area.

  If the acquired physical address is a physical address assigned to the local area, the cache directory management unit 36 instructs the packet control unit 37 to transmit a negative response (access error) to the request source. .

  If the acquired physical address is a physical address allocated to the shared area, the cache directory management unit 36 acquires data stored in the storage area indicated by the acquired physical address, and packetizes the acquired data. It outputs to the control part 37 and instruct | indicates to transmit to a request origin.

  In addition, the cache directory management unit 36 executes a process for holding the coherency of the cached data using the directory method. For example, when the data stored in the memory 22 is transmitted to the request transmission source CPU, the cache directory management unit 36 determines whether or not the data is cached by a CPU other than the request transmission source CPU. judge.

  The cache directory management unit 36 acquires the data to be requested from the L1 cache 32, the L2 cache 33, and the memory 22 when other CPUs do not cache the data to be requested. Thereafter, the cache directory management unit 36 outputs the acquired data to the packet control unit 37.

  On the other hand, when another CPU caches the data to be requested, the cache directory management unit 36 executes a process for maintaining cache coherence using a technique such as an Illinois protocol. For example, the cache directory management unit 36 determines whether the state of the cached data is MESI (Modified / Exclusive / Shared / Invalid).

  Then, according to the determination result, the cache directory management unit 36 transmits and receives requests and orders (commands) for maintaining coherency with the cache directory management unit of other CPUs, and according to the state of the cached data. Execute the process. Here, “Modified” indicates a state in which any one CPU caches data and the cached data is updated. Note that when the state of the cached data is “Modified”, it is necessary to execute write back.

  “Exclusive” indicates a state in which any one CPU caches data and the cached data is not updated. “Shared” indicates that a plurality of CPUs cache data and the cached data is not updated. “Invalid” indicates that the cache status is not registered.

  As a detailed example, the cache directory management unit 36 instructs the packet generation unit 37a to transmit an order for instructing write-back to a CPU that caches data whose status is M (Modified). Then, the cache directory management unit 36 updates the status of the data, and executes processing according to the updated status. The types of requests and orders sent and received by the cache directory management unit 36 will be described later.

  In addition, when an abnormality in data transfer between nodes is detected, the cache directory management unit 36 stores the cache data written back to the shared memory of the memory 22 by the node that caused the abnormality. If not, the following processing is performed. That is, the cache directory management unit 36 writes a value indicating an abnormality in which the cache data written back to the shared memory of the memory 22 is not written back to a predetermined area of the directory 36a by the down node.

  FIG. 6 is a diagram for explaining an example of the data structure of a directory. As illustrated in FIG. 6, the directory 36 a includes a 4-bit “UE” item from the 0th bit to the 3rd bit. Further, the directory 36a has a 63-bit “PRC” item from the 4th bit to the 66th bit. Further, the directory 36a has a 2-bit “CKBIT” item from the 67th bit to the 68th bit. In the item “CKBIT”, data obtained by coding the cache state is registered. In the item “PRC”, data representing the position of the CPU holding the cache as a bitmap is registered. In the item “UE”, data representing a directory abnormality and the cause of the abnormality is registered.

  Here, a case where the cache directory management unit 36 includes the directory 36a illustrated in the example of FIG. 6 will be described. In this case, when the cache data written back to the shared memory of the memory 22 is not written back by the node that has detected the abnormality and caused the occurrence of the abnormality, the cache directory management unit 36 performs the following processing. Do. That is, the cache directory management unit 36 assigns a value of 4 bits or less indicating the abnormality and cause of the cache data written back to the shared memory of the memory 22 by the node that has gone down to “ Write in the item “UE”. This can indicate that the data in the shared memory that has not been written back is not normal.

  In addition, the cache directory management unit 36 detects an abnormality in data transfer between nodes, and cache data written back to the shared memory of the memory 22 by the node that caused the abnormality is not written back. The following processing can also be performed. That is, the cache directory management unit 36 can also write data indicating an error state in the shared memory area of the memory 22 where the cache data is written back by the down node. Here, an example of data indicating an error state written in the shared memory area will be described. For example, when the data stored in the memory 22 includes ECC (Error Check and Correct) data for each predetermined number of bits, error correction of 2 bits or more is performed depending on an ECC generation polynomial. Can do. In this case, a syndrome having a specific value indicating an error of n (n ≧ 2) bits or more that is unlikely to occur compared to other errors is used as data indicating an error state written in the shared memory area. be able to. Further, when the data is written in the data body, a value such that the syndrome becomes a specific value can be used as data indicating an error state written in the shared memory area. This can indicate that the data in the shared memory that has not been written back is not normal.

  In addition, the cache directory management unit 36 performs recovery processing for the shared memory in the memory 22. As an example of the recovery process, there is a process of clearing a value indicating an abnormality in which the cache data written back to the shared memory is not written back from the directory 36a. Another example of the recovery process is a process of clearing data indicating an error state from the shared memory. As will be described later, this recovery processing is performed according to an instruction from the OS or application software.

  When the packet generation unit 37a acquires the physical address and CPUID from the cache directory management unit 36, the packet generation unit 37a generates a packet storing the acquired physical address and CPUID, that is, a packet serving as a memory access request. . Then, the packet generator 37a transmits the generated packet to the router 40.

  FIG. 7 is a diagram for explaining a packet transmitted by the CPU according to the first embodiment. In the example illustrated in FIG. 7, the packet generation unit 37 a generates a request including a CPU ID, a physical address, and data indicating the content of the request, and outputs the generated request to the router 40. In such a case, the router 40 outputs the request generated by the packet generation unit 37a to the XB2 via the XB connection unit 27. Then, XB2 transfers the request to the CPU indicated by the CPUID included in the request.

  When the packet generation unit 37a receives an instruction to issue a request or order for maintaining coherency from the cache directory management unit 36, the packet generation unit 37a generates the instructed request or order. Then, the packet generation unit 37a transmits the generated request or order to the instructed CPU via the router 40, the XB connection unit 27, and XB2. Further, when acquiring data from the I / O device, the packet generation unit 37 a outputs an access request for the I / O to the router 40.

  In addition, when the packet generation unit 37a transmits a request, the packet generation unit 37a outputs data indicating that the request has been transmitted to the abnormality detection unit 37c. This data includes information about the request, such as the type of request sent and the physical address of the memory to be accessed.

  When the packet receiving unit 37b receives a packet output from another CPU or another I / O device other than its own node via the XB2, the XB connecting unit 27, and the router 40, the packet receiving unit 37b sets the physical address included in the received packet. get. Then, the packet receiving unit 37b outputs the acquired physical address to the cache directory management unit 36. Further, when receiving data transmitted by another CPU, the packet receiving unit 37b outputs the received data to the cache directory management unit 36.

  Further, when receiving a request or order for maintaining coherency, the packet receiving unit 37b outputs the received request or order to the cache directory management unit 36. Further, when receiving a response or data of an access request to the I / O device from the router 40, the packet receiving unit 37b outputs the received response or data to the cache directory management unit 36. In such a case, for example, the cache directory management unit 36 outputs the acquired data to the memory access unit 41 and performs processing stored in the memory 22.

  Further, when receiving a response corresponding to the request transmitted by the packet generation unit 37a, the packet reception unit 37b outputs data indicating that the response has been received to the abnormality detection unit 37c. This data includes information about the response such as the type of response received.

  The abnormality detection unit 37c includes a pointer 80, a timer 81, a transmission packet 82, a PA 83, a request 84, and a detection unit 85.

  Each time the abnormality detection unit 37c receives data indicating that a request has been transmitted from the packet generation unit 37a, the abnormality detection unit 37c activates a timer 81 for measuring time. The abnormality detection unit 37c activates the timer 81 each time data indicating that a response has been received from the packet reception unit 37b. In addition, when receiving data indicating that the request has been transmitted from the packet generation unit 37a, the abnormality detection unit 37c stores, in the PA 83, the physical address of the access target memory included in the transmitted request from the received data. In addition, when the abnormality detection unit 37c receives data indicating that the request has been transmitted from the packet generation unit 37a, the abnormality detection unit 37c stores the transmitted request in the request 84 from the received data.

  Further, when receiving data indicating that the request has been transmitted from the packet generation unit 37a, the abnormality detection unit 37c registers the identifier of the transmitted request in the transmission packet 82 from the received data.

  In addition, when the abnormality detection unit 37c receives data indicating that a response has been received from the packet reception unit 37b, the physical address of the memory to be accessed included in the request corresponding to the received response is received from the PA 83. delete. In addition, when the abnormality detection unit 37c receives data indicating that a response has been received from the packet reception unit 37b, the abnormality detection unit 37c deletes the request corresponding to the received response from the request 84 from the received data. In addition, when receiving data indicating that a response has been received from the packet receiving unit 37b, the abnormality detection unit 37c deletes the identifier of the request corresponding to the received response from the transmission packet 82 from the received data. FIG. 8 is a diagram illustrating an example of a transmission packet. For example, when the number of requests for which no response is returned is N, the transmission packet 82 shown in the example of FIG. 8 includes N in each entry from the packet 1 management entry to the packet N management entry. Each of the request identifiers is registered.

  The pointer 80 indicates the request with the oldest transmission time among the requests indicated by the identifiers registered in the transmission packet 82. For example, in the example of FIG. 8, the pointer 80 points to the request indicated by the identifier registered in the packet 1 management entry.

  The detection unit 85 determines whether or not the time of the timer 81 has passed a predetermined time. When the predetermined time has elapsed, the detection unit 85 has not received a response corresponding to the request pointed to by the pointer 80 within the predetermined time, so data is transmitted between the node 20 and the node having the memory to be accessed. Detects that a transfer error has occurred. Then, the detection unit 85 acquires the physical address of the memory to be accessed included in the request indicated by the pointer 80 from the PA 83 and writes the acquired physical address in the error occurrence address register 96.

  The trap generation unit 97 generates a trap when a physical address is written in the error occurrence address register 96. If the request / order is “store data in memory” or “write back by cache replacement”, a trap is not generated, and only the data indicating the above-described abnormality is written, and the data CPU It is good also as detecting an abnormality at the time of loading. Here, the above-mentioned “writing of data indicating abnormality” means that a value of 4 bits or less indicating the abnormality and the cause is written in the item “UE” of the directory 36a, or in the shared memory area of the memory 22. Refers to writing data indicating an error condition.

  As described above, since the time is measured by one timer in the abnormality detection unit 37c, compared to the case where a timer is provided for each request, a smaller number is obtained when there are a plurality of requests for which no response is returned. The time can be measured with the timer.

  The configuration of the abnormality detection unit 37c is not limited to the configuration described above. For example, in the transmission packet 82 of the abnormality detection unit 37c, in addition to the request identifier, a difference (interval) in the transmission time of the request may be registered, and the timer 81 may be restarted taking the interval into account. .

  FIG. 9 is a diagram illustrating an example of another configuration of the abnormality detection unit. In the example of FIG. 9, in addition to the packet K (K is a natural number) management entry described above, the time from when a request is transmitted to when another request is transmitted is registered in the transmission packet 82. Here, a case will be described in which request 2 is transmitted 100 ns after request 1 is transmitted, and request 3 is transmitted 300 ns after request 1 is transmitted. In this case, the identifiers of the requests 1 to 3 are registered in the packet 1 management entry to the packet 3 management entry. At this time, with reference to request 1, 0 ns is registered in interval 1, 100 ns is registered in interval 2, and 300 ns is registered in interval 3. Here, it is assumed that the request indicated by the pointer 80 is switched from the request 1 to the request 2 because the packet receiving unit 37b has received a response corresponding to the request 1. At this time, the timer 81 restarts, but the initial value of the time is not 0 ns, and time measurement starts from the interval 100 ns corresponding to the request 2. When the request indicated by the pointer 80 is switched from the request 2 to the request 3, similarly, the timer 81 starts measuring time from the interval 300ns corresponding to the request 3. When responses are received for all transmitted requests, the interval between requests to be transmitted next becomes 0 ns again.

  As described above, in the abnormality detection unit 37c of another example, the time is measured by one timer 81, and therefore the time can be accurately measured by a smaller number of timers. Moreover, in the abnormality detection part 37c of another example, since the timer 81 measures time in consideration of an interval for each request, it is possible to measure time with higher accuracy.

  Note that the abnormality detection unit 37c can also provide a timer for each request without providing the pointer 80, and each timer can measure the time since the request was transmitted.

  When the router 40 receives the packet output from the packet generation unit 37 a included in the packet control unit 37, the router 40 outputs the received request to the XB connection unit 27. Further, the router 40 outputs packets and data transmitted by other CPUs to the packet receiving unit 37b via the XB connecting unit 27. Further, the router 40 outputs the packet output from the packet control unit 37 to the I / O device or the like to the PCIe control unit 42. When the router 40 receives a request or the like from the I / O device from the PCIe control unit 42, the router 40 outputs the received request or the like to the packet control unit 37. Further, when receiving a request from the I / O device from the PCIe control unit 42, the router 40 outputs the received request to the XB connection unit 27. In addition, when receiving a response to the I / O device via the XB connection unit 27, the router 40 outputs the received response to the PCIe control unit 42b.

  The memory access unit 41 is a so-called MAC (Memory Access Controller) and controls access to the memory 22. For example, when the memory access unit 41 receives a physical address from the cache directory management unit 36, the memory access unit 41 acquires data stored in the area of the memory 22 indicated by the received physical address, and the acquired data is stored in the cache directory management unit. To 36. Note that the memory access unit 41 may make the shared area redundant by using a memory mirror function.

  When the request generation unit 42a acquires an access request to the I / O device via the router 40, the request generation unit 42a generates a request to be transmitted to the I / O device that is the target of the access request, and the generated request is controlled by PCIe bus To the unit 42b. In addition, when the request generation unit 42a acquires the physical address and CPUID from the I / O device, the request generation unit 42a generates a packet storing the acquired physical address and CPUID, that is, a packet serving as a memory access request. To do. An example of the type of request is a request for an I / O device to read a memory connected to another CPU. Further, when the request generation unit 42a acquires a physical address, CPUID, and write data from the I / O device, the request generation unit 42a stores a packet that stores the acquired physical address, CPUID, and write data, that is, a memory Generate a packet that is an access request. An example of the type of request is a request for the I / O device to write data to a memory connected to another CPU. Then, the request generation unit 42 a transmits the generated packet to the router 40.

  The request generation unit 42 a includes an error occurrence address register 98 and a trap generation unit 99.

  When the PCIe bus control unit 42b acquires the request generated by the request generation unit 42a, the PCIe bus control unit 42b transmits the request to the I / O device via the PCIe connection unit 28. When the PCIe bus control unit 42b acquires the physical address and CPUID from the I / O device via the PCIe connection unit 28, the PCIe bus control unit 42b sends the acquired physical address and CPUID to the request generation unit 42a. Send. When the PCIe bus control unit 42b acquires the physical address, CPUID, and write data from the I / O device via the PCIe connection unit 28, the PCIe bus control unit 42b sends the acquired physical address to the request generation unit 42a. , CPUID and write data are transmitted.

  In addition, the PCIe bus control unit 42b does not receive a response corresponding to a request for the I / O device to read a memory connected to another CPU within a predetermined time after the request is transmitted. If detected, the following processing is performed. That is, the PCIe bus control unit 42 b transmits a “poisoned TLP” packet to the PCIe connection unit 28. In this case, when receiving the “poisoned TLP” packet, the PCIe connection unit 28 transmits the received “poisoned TLP” packet to the I / O device. As a result, the I / O device can detect that an abnormality has occurred. Further, when the PCIe connection unit 28 detects that an abnormality has occurred, the PCIe connection unit 28 notifies the device driver software that an abnormality has occurred, so that a recovery process is performed. As an example of the recovery process, when an abnormality occurs in the LAN, the processing of transmitting / receiving data as usual is performed after the transmission / reception data being processed is temporarily discarded and the state of the LAN chip is initialized. Can be mentioned.

  The “poisoned TLP” will be described with reference to FIG. FIG. 10 is a diagram illustrating an example of a data configuration of “TLP header”. “Poisoned TLP” is a packet defined by the specification of “PCI express”. When the EP bit of “TLP header” shown in the example of FIG. 10 is on, it is recognized as “poisoned TLP”.

  FIG. 11 is a diagram for explaining a specific example of the operation of the PCIe control unit that has received the “poisoned TLP” packet. The example of FIG. 11 shows a hierarchical structure of device drivers when the PCIe control unit 42 corresponds to “Root Complex” and the PCIe connection unit 28 corresponds to a “PCI express” card. In this example, a SAS “PCI express” card is mounted as a “PCI express” card. The detection of events such as the occurrence of “poisoned TLP” on the “PCI express” bus and the countermeasure on the bus are common regardless of the type of the “PCI express” card. Therefore, the detection of such an event and the countermeasure on the bus are not performed by an individual device driver such as a SAS device driver, but by a “Root Complex” driver.

  On the other hand, when an event such as an error-related event occurs on the bus, a SAS device driver-specific recovery process is often performed on the SAS device driver operating on the bus where the error occurred. Here, as an example of the recovery process unique to the SAS device driver, an end process or a retry process of the transmission process being processed can be cited. When “poisoned TLP” occurs, the “Root Complex” driver cuts error events (reads detailed information, clears status bits, etc.) and then notifies the SAS device driver on the bus of the occurrence of the error. . With this notification, the SAS device driver starts recovery processing specific to the SAS device driver. In addition, the occurrence of an error is not notified to the SAS device driver, the occurrence of the error is notified to the application process using the I / O device, and the I / O device is restarted from the application process. Good.

  In addition, the PCIe bus control unit 42b does not receive a response corresponding to a request for the I / O device to write data to a memory connected to another CPU within a predetermined time after the request is transmitted. When an abnormality is detected, the following processing is performed. That is, after detecting the abnormality, the PCIe bus control unit 42b discards the “request for writing data to the memory” received from the PCIe connection unit 28 related to the abnormality. Then, as described later, an error occurrence address is set in the error occurrence address register 98 and a trap is generated using the trap generation unit 97. Since this trap notifies the device driver software that an abnormality has occurred, recovery processing is performed. As an example of the recovery process, when an abnormality occurs in the LAN, the processing of transmitting / receiving data as usual is performed after the transmission / reception data being processed is temporarily discarded and the state of the LAN chip is initialized. Can be mentioned.

  The abnormality detection unit 42c includes a pointer 90, a timer 91, a transmission packet 92, a PA 93, a request 94, and a detection unit 95. Each of the pointer 90, timer 91, transmission packet 92, PA93, request 94, and detection unit 95 is the same as the pointer 80, timer 81, transmission packet 82, PA83, request 84, and detection unit 85 described above. The error occurrence address register 98 and the trap generation unit 99 are the same as the error occurrence address register 96 and the trap generation unit 97 described above. That is, the abnormality detection unit 37c detects an abnormality in data transfer between nodes when a predetermined time elapses after a request is transmitted by the packet control unit 37 and before a response is received. Similar to the above-described abnormality detection unit 37c, the abnormality detection unit 42c performs an abnormality in data transfer between nodes for a read request and a write request transmitted from the request generation unit 42a to another CPU via the router 40. Is detected. If an abnormality is detected, like the detection unit 85 of the abnormality detection unit 37c, the detection unit 95 acquires the physical address of the access target memory included in the request indicated by the pointer 90 from the PA 93, The acquired physical address is written in the error occurrence address register 98.

  The trap generation unit 99 generates a trap when a physical address is written in the error generation address register 98.

  Next, an example of processing in which the CPU 21 transmits a request to another CPU will be described with reference to FIG. FIG. 12 is a schematic diagram illustrating an example of a process in which the CPU according to the first embodiment transmits a request. For example, as shown in FIG. 12A, the service processor 24 sets an entry in which the CPU ID of the CPU accessing the memory to which the physical address is allocated and the physical address are associated with each other in the node map 34. .

  In addition, the calculation unit 31 executes calculation processing and outputs a logical address to be accessed to the address conversion unit 35 as shown in FIG. Then, the address conversion unit 35 converts the logical address into a physical address, and outputs the converted physical address to the cache directory management unit 36 as shown in FIG.

  Here, when the cache directory management unit 36 acquires the physical address from the address conversion unit 35, the cache directory management unit 36 refers to the node map 34 as shown in FIG. 12D, and determines the CPU ID associated with the acquired physical address. get. Then, when the acquired CPUID is not the CPUID of the CPU 21, the cache directory management unit 36 outputs the acquired CPUID and physical address to the packet control unit 37 as shown in FIG.

  In such a case, the packet generation unit 37a generates a packet storing the physical address and CPUID acquired from the cache directory management unit 36, and the generated packet is transmitted to the router as shown in FIG. Output to 40. Further, as shown in FIG. 12G, the packet generation unit 37a outputs data indicating that the packet that is a request has been transmitted to the abnormality detection unit 37c. Subsequently, as illustrated in FIG. 12H, the router 40 outputs the packet acquired from the packet generation unit 37 a to the XB connection unit 27. Thereafter, as shown in (I) of FIG. 12, the XB connection unit 27 outputs the acquired packet to XB2. Then, the XB 2 transmits the packet to the CPU indicated by the CPU ID stored in the packet.

  Next, an example of processing executed when the CPU 21 receives a packet from another CPU will be described with reference to FIG. FIG. 13 is a diagram for explaining an example of processing executed when the CPU according to the first embodiment receives a packet. For example, as shown in FIG. 13J, the packet receiving unit 37b receives a packet in which the CPU ID of the CPU 21 and the physical address assigned to the memory 22 are stored and a response packet from another CPU.

  In such a case, when the received packet is a response packet, the packet receiving unit 37b receives data indicating that the response packet has been received, as shown in FIG. It outputs to the abnormality detection part 37c. Then, the packet receiving unit 37b acquires a physical address from the received packet, and outputs the acquired physical address to the cache directory management unit 36 as shown in (L) of FIG. Then, the cache directory management unit 36 determines whether the storage area indicated by the physical address is a shared area or a local area.

  When the access is to the shared area, the cache directory management unit 36 caches the data in the storage area indicated by the physical address in the L1 cache 32 and the L2 cache 33 as shown in (M) in FIG. Determine if it is.

  If the cache directory management unit 36 determines that the data is not cached, the cache directory management unit 36 outputs the physical address to the memory access unit 41 as indicated by (N) in FIG. Then, as indicated by (O) in FIG. 13, the memory access unit 41 acquires the data in the storage area indicated by the physical address from the memory 22 and outputs the data to the cache directory management unit 36.

  When the cache directory management unit 36 acquires data from the L1 cache 32, the L2 cache 33, or the memory access unit 41, the cache directory management unit 36 outputs the acquired data to the packet control unit 37 and transmits it to the requesting CPU. Instruct.

  Next, an example of processing in which the I / O device transmits a read or write request to a CPU other than the CPU 21 will be described with reference to FIG. FIG. 14 is a diagram for explaining an example of processing in which the I / O device according to the first embodiment transmits a request. For example, when the PCIe connection unit 28 acquires the physical address and CPUID from the I / O device, the acquired physical address and CPUID are output to the PCIe bus control unit 42b as shown in (P) of FIG. To do. When the PCIe connection unit 28 acquires the physical address, CPUID, and write data from the I / O device, the acquired physical address, CPUID, write data, and the like, as shown in FIG. Is output to the PCIe bus control unit 42b.

  When the PCIe bus control unit 42b acquires the physical address and the CPUID from the PCIe connection unit 28, the PCIe bus control unit 42b outputs the acquired physical address and the CPUID to the request generation unit 42a as shown in (Q) of FIG. To do. Further, when the PCIe bus control unit 42b acquires the physical address, CPUID, and write data from the PCIe connection unit 28, as shown in (Q) of FIG. 14, the acquired physical address, CPUID, and write data are acquired. Is transmitted to the request generation unit 42a.

  When the request generation unit 42a acquires the physical address and CPUID from the PCIe bus control unit 42b, the request generation unit 42a generates a packet serving as a read request including the acquired physical address and CPUID. Further, when the request generation unit 42a acquires the physical address, CPUID, and write data from the PCIe bus control unit 42b, the request generation unit 42a generates a packet that becomes a write request including the acquired physical address, CPUID, and write data. To do. Then, the request generation unit 42a outputs the generated packet to the router 40 as shown in (R) in FIG.

  Further, as shown in FIG. 14S, the request generation unit 42a outputs data indicating that the read request and the write request are transmitted to the abnormality detection unit 42c. Subsequently, as illustrated in (T) in FIG. 14, the router 40 outputs the request acquired from the request generation unit 42 a to the XB connection unit 27. After that, as shown in FIG. 14 (U), the XB connection unit 27 outputs the acquired request to XB2. Then, XB2 transmits the packet to the CPU indicated by the CPUID stored in the request.

  Next, an example of processing in which the I / O device receives a response from a CPU other than the CPU 21 will be described with reference to FIG. FIG. 15 is a diagram for explaining an example of processing in which the I / O device according to the first embodiment receives a response. For example, as shown in FIG. 15 (V), the XB connection unit 27 receives a response from the CPU other than the CPU 21 to the I / O device.

  When receiving the response, the XB connection unit 27 outputs the received response to the router 40 as shown in FIG. When receiving the response, the router 40 outputs the received response to the request generation unit 42a, as indicated by (X) in FIG. Upon receiving the response, the request generation unit 42a outputs data indicating that the response has been received to the abnormality detection unit 42c. Further, the request generation unit 42a outputs a response to the PCIe bus control unit 42b as shown in (Z) in FIG. When receiving the response, the PCIe bus control unit 42b outputs the received response to the PCIe connection unit 28 as shown in (AA) in FIG. As a result, a response is transmitted from the PCIe connection unit 28 to the I / O device.

  Next, an example of processing executed when a data transfer abnormality occurs between the node 20 and a node having a memory to be accessed by the node 20 will be described with reference to FIGS. 16 and 17. FIG. 16 is a diagram for explaining an example of processing executed when a data transfer abnormality occurs between a node and a node having a memory to be accessed by the node. For example, the abnormality detection unit 37c determines whether or not the time of the timer 81 has passed a predetermined time. When the predetermined time has elapsed, the abnormality detection unit 37c has not received a response corresponding to the request indicated by the pointer 80 within the predetermined time, and therefore, between the node 20 and the node having the memory to be accessed. Detects that a data transfer error has occurred. Then, the abnormality detection unit 37c acquires the physical address of the memory to be accessed included in the request indicated by the pointer 80 from the PA 83, and the acquired physical address is stored in the error occurrence address register as shown in (AB) in FIG. Write to 96. When the physical address is written in the error occurrence address register 96, the trap generation unit 96 generates a trap as shown in (AC) in FIG.

  FIG. 17 is a diagram for explaining an example of processing executed when a data transfer abnormality occurs between a node and a node having a memory to be accessed by the node. For example, the abnormality detection unit 42c determines whether or not the time of the timer 91 has passed a predetermined time. When the predetermined time has elapsed, the abnormality detection unit 42c has not received a response corresponding to the request pointed to by the pointer 90 within the predetermined time, and therefore, between the node 20 and the node having the memory to be accessed. Detects that a data transfer error has occurred. Then, the abnormality detection unit 42c acquires the physical address of the memory to be accessed included in the request indicated by the pointer 90 from the PA 93, and the acquired physical address is stored in the error occurrence address register as shown in (AD) in FIG. Write to 98. When the physical address is written in the error occurrence address register 98, the trap generation unit 98 generates a trap as shown in (AE) in FIG.

  The communication unit 23, the service processor 24, the XB connection unit 27, the XB connection unit 27a, and the PCIe connection unit 28 are electronic circuits. Here, as an example of the electronic circuit, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), a CPU or a micro processing unit (MPU), or the like can be applied. Further, an integrated circuit such as an ASIC or FPGA, an MPU, or the like can be applied instead of the CPUs 21 to 21c.

  The memories 22 to 22a are semiconductor memory elements such as a random access memory (RAM), a read only memory (ROM), and a flash memory. The L1 cache 32 and the L2 cache 33 are high-speed semiconductor memory elements such as SRAM (Static Random Access Memory).

  Next, a process in which each of the CPUs 21 to 21c holds cache coherence will be described. In the following description, it is assumed that each CPU of the information processing system 1 holds cache coherence using the Illinois protocol.

  In the following description, each memory included in the information processing system 1 is identified as a memory having a space that can be cached by all CPUs. In the following description, a CPU that is physically directly connected to a memory that stores data to be cached via a MAC in the CPU is a home CPU, and a CPU that requests a cache is a local CPU. Describe.

  A CPU that has already transmitted a request to the home CPU and has already cached data is referred to as a remote CPU. Note that the local CPU and the home CPU may be the same CPU, or the local CPU and the remote CPU may be the same CPU.

  The local CPU refers to its own node map and determines that the physical address to be accessed is allocated to the memory accessed by the home CPU. Then, the local CPU issues a request storing the physical address to the home CPU. There are multiple types of requests issued by the local CPU. For this reason, the cache directory management unit included in the home CPU executes cache coherence control according to the type of the acquired request.

  For example, the types of requests issued by the local CPU include shared fetch access, exclusive fetch access, cache invalidation request, and cache replacement request. The shared fetch access is, for example, a “MoveIn to Share” execution request, which is a request issued when data is read from the memory accessed by the home CPU.

  The exclusive fetch access is, for example, an execution request for “MoveIn Exclusive”, which is issued when data is loaded into the cache when data is stored in the memory accessed by the home CPU. The cache invalidation request is, for example, an execution request for “MoveOut” and is issued when a cache line invalidation request is issued to the home CPU. When the home CPU receives the cache invalidation request, the home CPU may issue a cache invalidation request to the remote CPU or may issue an order for setting the cache to “Invalidation”.

  The cache replacement request is, for example, a request to execute “WriteBack”, and is issued when the updated cache data, that is, cache data in the “Modified” state is written back to the memory accessed by the home CPU. The cache replacement request is, for example, an execution request for “FlushBack”, and is issued when discarding cache data that has not been updated, that is, a cache in the “Shared” or “Exclusive” state.

  When the home CPU receives the above-described request from the local CPU or the remote CPU, the home CPU issues an order to the local CPU or the remote CPU in order to process the request. Here, the home CPU issues a plurality of types of orders in order to execute cache coherence control according to the type of the acquired request. For example, the home CPU issues “MoveOut and Bypass to Share” which causes the local CPU to load data cached by the remote CPU.

  For example, the home CPU invalidates the caches of all remote CPUs other than the local CPU, and then issues “MoveOut and Bypass Exclusive” for the home CPU to transmit data to the local CPU. Further, the home CPU issues “MoveOut WITH Invalidation” requesting the remote CPU to invalidate the cache. When the home CPU issues “MoveOut WITH Invalidation”, the caches of all CPUs are in the “Invalidate” state for the target address. When the transaction is completed, the local CPU caches data.

  The home CPU issues a “MoveOut for Flush” requesting the remote CPU to invalidate the cache line. When the home CPU issues “MoveOut for Flush”, the target data is stored only in the memory of the home CPU. Further, the home CPU issues “Buffer Invalidation” requesting the remote CPU to discard the cache when the state of the target data is “Shared”.

  The home CPU issues the above-described order according to the type of request, and changes the state of the data cached by each CPU. When the local CPU or the remote CPU receives an order, the local CPU or the remote CPU executes the process indicated by the order and changes the state of the data cached by itself.

  Thereafter, the local CPU and the remote CPU transmit a completion response to the order and a completion response with data to the home CPU. The home CPU or remote CPU transmits a request response with data to the local CPU after executing the order process.

[Process flow]
Next, the flow of processing in which the information processing system 1 controls the shared area will be described with reference to FIG. FIG. 18 is a flowchart for explaining the flow of processing for controlling a shared area. First, the information processing system 1 executes a shared memory allocation process between nodes in response to an application request (step S101). Next, the information processing system 1 executes a process for attaching a shared memory shared between nodes (step S102).

  Thereafter, an application executed by each CPU included in the information processing system 1 uses each memory (step S103). Next, the information processing system 1 executes a shared memory detach process (step S104). Thereafter, the information processing system 1 executes a shared memory release process (step S105), and ends the process. Note that steps S101 and S105 may be executed only by the application on the home node of the shared memory, and the actual processing is nop, but the application on the node other than the home node of the shared memory. Can also be implemented.

  Next, the flow of processing for executing the shared memory allocation processing shown in step S101 in FIG. 18 will be described with reference to FIG. FIG. 19 is a flowchart for explaining shared memory allocation processing. In the example shown in FIG. 19, for example, an application executed by the CPU 21 requests the OS to execute a shared memory allocation process between nodes (step S201).

  Then, the OS executed by the CPU 21 performs memory allocation of the requested size from the area of the physical address for the shared area (step S202). Next, the shared memory management ID assigned by the OS is delivered to the application (step S203), and the shared memory assignment process is terminated.

  Next, the flow of the shared memory attach process between nodes shown in step S102 in FIG. 18 will be described with reference to FIG. FIG. 20 is a flowchart for explaining the shared memory attach process. First, the application delivers a management ID to the OS and requests a shared memory attach process between nodes (step S301). In such a case, the OS communicates with the OS executed in another node, and acquires a physical address corresponding to the management ID (step S302).

  Here, when the OS communicates with an OS running on another node, communication using a LAN or the like, communication between nodes via the service processor 24, or the like is used. In addition, the OS executed in each node may perform communication by setting a specific shared area as an area used for inter-node communication and storing or reading information in the set area.

  Next, the OS determines a logical address corresponding to the physical address and assigns it (step S303). For example, the OS executed by the CPU 21 sets the TLB 35 a of the physical address and the logical address in the address conversion unit 35.

  The logical addresses used by the CPUs 21 to 21c may be in an overlapping range or may be a different range for each CPU. The logical addresses used by the CPUs 21 to 21c may be specified by the application to the OS. Thereafter, the OS hands over the value of the logical address to the application (step S304) and ends the process.

  Next, the flow of processing in which the application shown in step S103 in FIG. 18 uses the shared memory between nodes will be described with reference to FIG. FIG. 21 is a flowchart for explaining processing in which an application uses a shared memory. For example, the application executed by the CPU 21 issues a logical address and accesses a storage area indicated by the logical address (step S401).

  Then, the CPU 21 determines whether or not a TLB miss has occurred (step S402). When a TLB miss occurs (Yes at Step S402), the CPU 21 executes trap processing and sets a pair of logical address and physical address in the TLB (Step S403).

  Next, the application issues a logical address again, and after performing conversion to a physical address by TLB, normally executes access to the shared memory (step S404), and ends the process. On the other hand, if no TLB miss has occurred (No at step S402), the shared memory is normally accessed (step S405), and the process ends.

  Next, the flow of the shared memory detach process between nodes shown in step S104 in FIG. 18 will be described with reference to FIG. FIG. 22 is a flowchart for explaining a shared memory detach process between nodes. For example, the application executed by the CPU 21 requests the detach process by designating the logical address of the inter-node shared memory or the management ID to the OS (step S501).

  Then, the OS executed by the CPU 21 flushes the cache (step S502). That is, when the OS that releases the shared memory and then allocates it again as shared memory, when the CPU that accesses the real memory of the shared memory reboots when the shared memory is not allocated, the OS and cache There is a risk that the state of the memory may conflict. For this reason, the OS flushes the cache to prevent a state where the state of the cache and the real memory are inconsistent.

  Then, the OS releases the allocation of the logical address in the range used by the application, that is, the inter-node shared memory, and deletes the entry of the TLB 35a related to the released logical address (step S503). Thereafter, even if a TLB miss occurs on the memory address on which the detachment is completed on this node (Yes in step S402), the OS assigns the physical address corresponding to the logical address on which the detachment is completed to the TLB 35a. Not set. Therefore, step S404 does not end normally and an access error occurs. After the completion of the detachment, the OS communicates between the nodes, contrary to step S302, and notifies that this application has completed access to the PA of the shared memory (step S504). If this shared memory has been released on the home node and this application is the last user of this shared memory, the home node is requested to perform the release process (step S505), and the process ends.

  Next, the flow of the inter-node shared memory release process shown at step S105 in FIG. 18 will be described using FIG. FIG. 23 is a flowchart for explaining the inter-node shared memory release process. For example, the application executed by the CPU 21 requests the OS to release the shared memory between nodes (step S601). Then, when all users of the designated shared area have detached, the OS releases the allocation (step S602) and ends the process. If the detach has not been completed, the allocation release process is not performed, and the process is completed (the actual allocation completion process is performed in S505).

  Next, the flow of processing in which the CPU 21 transmits a memory access request to another CPU will be described with reference to FIG. FIG. 24 is a flowchart for explaining the flow of processing for issuing a request. For example, the calculation unit 31 of the CPU 21 issues a logical address (step S701).

  Then, the address conversion unit 35 performs conversion from a logical address to a physical address (step S702). Next, the cache directory management unit 36 acquires a physical address and executes cache directory management (step S703). That is, the cache directory management unit 36 changes the cache state for the storage area indicated by the acquired physical address.

  Next, the cache directory management unit 36 refers to the node map 34 and determines whether or not the acquired physical address is a physical address allocated to the memory of another node (step S704). If the cache directory management unit 36 determines that the acquired physical address is not a physical address allocated to the memory of another node (No in step S704), the cache directory management unit 36 executes memory access using the acquired physical address. (Step S705). Then, the process ends.

  On the other hand, when the acquired physical address is a physical address assigned to the memory of another node (Yes at step S704), the cache directory management unit 36 acquires the CPU ID associated with the physical address from the node map 34. (Step S706). Then, the packet transmission unit generates a packet storing the CPU ID and the physical address, that is, a memory access request, sends the packet to XB 2 (step S707), and ends the process.

  Next, a flow of processing executed when the CPU 21 receives a memory access request from another CPU will be described with reference to FIG. FIG. 25 is a flowchart for explaining the flow of processing executed when a request is received. In the example illustrated in FIG. 25, the flow of processing executed when the CPU 21 receives “MoveIn to Share” or “MoveIn Exclusive” from another CPU will be described. For example, the CPU 21 receives a request from another CPU via XB2 (step S801).

  In such a case, the CPU 21 uses the node map 34 to determine whether or not the physical address to be requested is a local area (step S802). If the physical address to be requested is a local area (Yes at step S802), the CPU 21 returns a negative response to the requesting CPU (step S803) and ends the process.

  On the other hand, when the physical address to be requested is not a local area (No at Step S802), the CPU 21 executes cache directory management that holds coherence (Step S804). Further, the CPU 21 determines the status of the storage area indicated by the physical address (step S805).

  Then, the CPU 21 issues an order corresponding to the determined status to another CPU (step S806), and changes the status (step S807). Thereafter, the CPU 21 makes a response to transmit the data in the storage area indicated by the physical address to the requesting CPU (step S808), and ends the process.

  Next, a flow of processing executed when the CPU 21 receives a response will be described with reference to FIG. FIG. 26 is a flowchart for explaining the flow of processing executed when the CPU receives a response. For example, the CPU 21 receives a response (step S901). In such a case, the CPU 21 determines whether or not the response content is a normal response (step S902).

  When the content of the response is normal, that is, when the data to be requested is received (Yes at Step S902), the CPU 21 executes normal processing using the data (Step S903) and performs the processing. finish. On the other hand, when receiving a negative response (No at Step S902), the CPU 21 determines whether the reason for the negative response is an access error (Step S904).

  If the reason for the negative response is not an access error (No at Step S904), the CPU 21 executes normal error processing (Step S905) and ends the process. On the other hand, if the reason for the negative response is not an access error (Yes at Step S904), the CPU 21 sets the physical address where the error has occurred in the error register, executes the trap process (Step S906), and ends the process. To do.

  Next, the flow of processing executed when the CPU 21 transmits a request will be described with reference to FIG. FIG. 27 is a flowchart for explaining the flow of processing executed when the CPU transmits a request. For example, the CPU 21 stores the physical address of the access destination included in the request in the PA 83 and stores the request type in the request 84 (step S1001). And CPU21 transmits a request to CPU of another node (step S1002). Subsequently, the CPU 21 starts up the timer 81 (step S1003).

  Thereafter, the CPU 21 determines whether or not a response to the request has been received before a predetermined time has elapsed since the request was transmitted, that is, before timeout (step S1004). If a response is received before the timeout (Yes at Step S1004), the CPU 21 stops the timer 81 (Step S1005). And CPU21 processes a response (step S1006) and complete | finishes a process.

  On the other hand, if a response to the request has not been received before the timeout (No at step S1004), the CPU 21 performs the following process. That is, the CPU 21 specifies the physical address stored in the PA 83 corresponding to the request, sets the specified physical address in the error occurrence address register 96, and generates a trap (step S1007). Next, for the CPU connected to the memory having the storage area indicated by the identified physical address, the CPU 21 is the home CPU, and the request includes an instruction to write back dirty cache data to the memory 21. It is determined whether or not it is (step S1008).

  When the CPU 21 is the home CPU and the request includes an instruction to write back dirty cache data to the memory 21 (Yes in step S1008), the CPU 21 performs the following process. That is, the CPU 21 generates a value indicating an abnormality in which the cache data written back to the shared memory of the memory 22 has not been written back, and writes the generated value in a predetermined area of the directory 36a. Alternatively, the CPU 21 generates data indicating an error state in the shared memory area of the memory 22 where the cache data is written back by the down node, and writes the generated data (step S1009). Then, the process ends. In addition, when the CPU 21 is not the home CPU or the request does not include an instruction to write back dirty cache data to the memory 21 (No in step S1008), the process is also terminated.

  Next, a flow of processing executed when the PCIe control unit 42 transmits a read request will be described with reference to FIG. FIG. 28 is a flowchart for explaining a flow of processing executed when the PCIe control unit transmits a read request. For example, the PCIe control unit 42 stores the physical address of the access destination included in the request in the PA 83 and stores the request type in the request 84 (step S1101). Then, the PCIe control unit 42 transmits the request to the CPU of another node (step S1102). Subsequently, the PCIe control unit 42 starts the timer 91 (step S1103).

  Thereafter, the PCIe control unit 42 determines whether or not a response to the request has been received before the timeout (step S1104). If a response is received before the timeout (Yes at Step S1104), the PCIe control unit 42 stops the timer 91 (Step S1105). Then, the PCIe control unit 42 processes the response (step S1106) and ends the process.

  On the other hand, if the response to the request has not been received before the timeout (No at Step S1104), the PCIe control unit 42 performs the following process. That is, the PCIe control unit 42 identifies the physical address stored in the PA 83 corresponding to the request, sets the identified physical address in the error occurrence address register 98, and generates a trap (step S1107). Next, the PCIe control unit 42 generates a “poisoned TLP” packet, transmits the generated “poisoned TLP” packet to the PCIe connection unit 28 (step S1108), and ends the process.

  Next, a flow of processing executed when the PCIe control unit 42 transmits a write request will be described with reference to FIG. FIG. 29 is a flowchart for explaining the flow of processing executed when the PCIe control unit transmits a write request. For example, the PCIe control unit 42 stores the physical address of the access destination included in the request in the PA 83 and stores the request type in the request 84 (step S1201). Then, the PCIe control unit 42 transmits the request to the CPU of another node (step S1202). Subsequently, the PCIe control unit 42 starts the timer 91 (step S1203).

  Thereafter, the PCIe control unit 42 determines whether a response to the request has been received before the timeout (step S1204). If a response is received before the timeout (Yes at Step S1204), the PCIe control unit 42 stops the timer 91 (Step S1205). Then, the PCIe control unit 42 processes the response (step S1206) and ends the process.

  On the other hand, if the response to the request has not been received before the timeout (No at Step S1204), the PCIe control unit 42 performs the following process. That is, the PCIe control unit 42 identifies the physical address stored in the PA 83 corresponding to the request, sets the identified physical address in the error occurrence address register 98, generates a trap (step S1207), and ends the process. To do.

  Next, the flow of trap processing executed by the OS when a trap occurs will be described with reference to FIG. FIG. 30 is a flowchart for explaining the flow of trap processing executed by the OS when a trap occurs. For example, when a trap occurs, the OS executed by the CPU 21 activates an interrupt handler (step S1301). The interrupt handler specifies the type of trap (step S1302). The interrupt handler determines whether or not the trap type indicates that a read process is being performed due to a communication error with the CPU of another node detected by the CPU 21. When the trap type indicates that the read process is being performed due to a communication error with the CPU of another node detected by the CPU 21 (Yes in S1303), the interrupt handler performs the following process. That is, the interrupt handler sets the signal handler of the process indicated by the program counter when the interrupt occurs as the signal transmission destination (step S1304). Subsequently, the interrupt handler transmits a signal to the signal transmission destination (step S1305), and ends the process.

  If the trap type does not indicate that read processing is in progress due to a communication error with the CPU of another node detected by the CPU 21 (No in step S1303), the interrupt handler performs the following processing. . That is, the interrupt handler determines whether or not the trap type indicates that the write process is being performed due to a communication error with the CPU of another node detected by the CPU 21 (step S1306). If the trap type indicates that a write process is being performed due to a communication error with the CPU of another node detected by the CPU 21 (Yes in step S1306), the interrupt handler reads the error occurrence address register 96 and reads the physical address. Is acquired (step S1307). Then, the interrupt handler searches the handler table for a signal handler corresponding to the acquired physical address (step S1308).

  The handler table is created as follows. First, when a memory allocation of the requested size is performed from the physical address area for the shared area in response to an application request executed by the CPU 21, a signal handler is acquired, and the entry address of the acquired signal handler function is set. Register in the handler table. Then, when registering the entry address of the function of the signal handler in the handler table, the OS performs the following processing. That is, the OS executed by the CPU 21 registers the shared memory address and the process identifier in the handler table in association with the entry address of the signal handler function. In this way, the handler table is created.

  FIG. 31 is a diagram illustrating an example of the data structure of the handler table. The handler table shown in FIG. 31 includes an item “shared memory address”, an item “pid”, and an item “entry address of a function of a signal handler”. In the item “shared memory address”, the address of the shared memory corresponding to the signal handler in which the entry address is registered in the item “entry address of function of signal handler” is registered. In the “pid” item, an identifier of a process corresponding to the signal handler in which the entry address is registered in the “signal handler function entry address” item is registered. The entry address of the function of the signal handler is registered in the item “entry address of the function of the signal handler”.

  Returning to FIG. 30, as a result of the search, the interrupt handler determines whether or not a signal handler has been obtained (step S1309). If the signal handler can be obtained (Yes at step S1309), the interrupt handler sets the obtained signal handler as the signal transmission destination (step S1310), and proceeds to step S1305. On the other hand, if the signal handler cannot be obtained (No in step S1309), the interrupt handler interrupts all processes using the shared memory in the shared area indicated by the physical address acquired in step S1307 (step S1307). S1311), the process is terminated.

  Further, when the trap type indicates that a write process is in progress due to a communication error with the CPU of another node detected by the CPU 21 (No in step S1306), the interrupt handler makes the following determination. Can do. That is, the interrupt handler can determine that the trap type is a communication error with the CPU of another node detected by the PCIe control unit 42. Therefore, the interrupt handler starts an interrupt processing routine of the PCIe control unit 42 (step S1312). In the interrupt process executed by the CPU 21, the error occurrence address register 98 of the PCIe control unit 42 is read and a physical address is acquired (step S1313).

  Next, the interrupt handler determines whether or not the storage area indicated by the acquired physical address is a shared area of the shared memory (step S1314). If the storage area indicated by the acquired physical address is a shared area of the shared memory (Yes at step S1314), the process returns to step S1308. On the other hand, if the storage area indicated by the acquired physical address is not a shared area of the shared memory (No at step S1314), a predetermined corresponding process is performed (step S1315), and the process is terminated.

  Next, the flow of processing executed by the signal handler notified of the signal will be described with reference to FIG. FIG. 32 is a flowchart for explaining the flow of processing executed by the signal handler notified of the signal. For example, the interrupt handler notifies the signal handler of the signal and activates the signal handler (step S1401). The activated signal handler identifies the shared memory in which an abnormality has occurred from the physical address notified to the signal handler (step S1402). The signal handler detaches all the shared memories existing in the node where the abnormality has occurred from the information on the node where the abnormality notified to the signal handler has occurred (step S1403).

  Subsequently, the signal handler performs recovery processing for all the shared memories of the node 20 shared with the node where the abnormality has occurred (step S1404). Then, the signal handler performs application-specific recovery processing (step S1405).

  An example of application-specific recovery processing will be described. For example, when the CPU 21 executes an application that creates “checkpoint” on an external storage device such as a disk at regular intervals, a recovery process of reading the data indicated by “checkpoint” and restarting the process is included. . Further, when the CPU 21 executes an application that does not create “checkpoint”, a recovery process such as a reinitialization or a restart process is given.

  Further, the processing executed by the signal handler notified of the signal is not limited to the processing described above. Therefore, the flow of other processing executed by the signal handler notified of the signal will be described with reference to FIG. FIG. 33 is a flowchart for explaining another processing flow executed by the signal handler notified of the signal. For example, the interrupt handler notifies the signal handler of the signal and activates the signal handler (step S1501). The activated signal handler identifies the shared memory in which an abnormality has occurred from the physical address notified to the signal handler (step S1502). The signal handler determines whether or not the node 20 (self node) having the CPU 21 that executes the signal handler has attached the memory of the node where the abnormality has occurred as a shared memory (step S1503). If the own node has not attached the memory of the node in which the abnormality has occurred as a shared memory (No at step S1503), the process proceeds to step S1508.

  On the other hand, when the own node has attached the memory of the node where the abnormality has occurred as a shared memory (Yes in step S1503), the signal handler performs the following processing. That is, the signal handler detaches all the shared memories existing in the node where the abnormality has occurred from the information of the node where the abnormality notified to the signal handler has occurred (step S1504).

  Subsequently, the signal handler deletes the address of the shared memory existing in the node where the abnormality has occurred from the L1 cache and the L2 cache (step S1505). Then, the signal handler deletes the entry including the address of the shared memory existing in the node where the abnormality has occurred from the TLB 35a (step S1506). Then, the signal handler sets the physical address notified to the signal handler as a physical address that prohibits registration in the TLB 35a even if a TLB miss occurs (step S1507).

  Subsequently, the signal handler determines whether or not the node where the abnormality has occurred is attached as a shared memory to the memory of its own node (step S1508). If the node where the abnormality occurred is attached as the shared memory to the memory of the own node (Yes in step S1508), the signal handler applies to all the shared memories of the node 20 shared with the node where the abnormality occurred. Recovery processing is performed (step S1509). Then, the signal handler performs an application-specific recovery process (step S1510) and ends the process. In addition, when the node in which an abnormality has occurred is not attached as a shared memory to the memory of the own node (No in step S1508), the process is also terminated.

[Effect of Example 1]
As described above, the information processing system 1 includes a plurality of nodes each including the memories 22 to 22c that can be partially set as a shared memory area, and the XB 2 that connects the plurality of nodes. Each of the plurality of nodes includes abnormality detection units 37c and 42c that detect an abnormality in data transfer between the plurality of nodes or an abnormality in another node. Further, each of the plurality of nodes generates a value indicating an abnormality to be registered in the “poisoned TLP” packet or the item “UE” of the directory 36a based on the abnormality detected by the abnormality detection units 37c and 42c. Each of the plurality of nodes generates an interrupt to the processor in the node that issued the data transfer request. Further, the OS executed by each CPU of the plurality of nodes performs a recovery process when receiving an interrupt.

  That is, at least one of the plurality of nodes has an abnormality detection unit. The abnormality detection unit detects an abnormality in the data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in the storage device included in the one node or the other node. To detect. In addition, the abnormality detection unit is included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in the storage device included in the one node or the other node. Detect anomalies in any of the nodes that can be included. In addition, at least one of the plurality of nodes has an error information generation unit that generates error information based on the abnormality detected by the abnormality detection unit and generates an interrupt to the processor in the own node. In addition, at least one of the plurality of nodes executes recovery processing based on the error information in response to the interrupt.

  In this way, in the information processing system 1, even when the communication partner node is down during data transfer, the own node shared by the down node by the OS executed by the CPU 21 notified of the interrupt Recovery processing is performed on the shared memory. Therefore, even when the communication partner node goes down and the counterpart node caches the data in the shared memory, the own node does not have to go down. Therefore, according to the information processing system 1, even if a communication abnormality occurs between nodes during data transfer, it is not necessary for all the nodes to be down as in the prior art. Therefore, according to the information processing system 1, when an abnormality related to data transfer between nodes occurs, it is possible to suppress the error influence range.

  In addition, the information processing system 1 can detect a timeout after a request is transmitted by using only one timer 81 or 91 by switching the request whose time is to be measured using the pointers 80 and 90.

  Further, since the information processing system 1 determines whether the access target is a shared area or a local area based on the received physical address, the security level of kernel data and user data stored in the local area is kept high. be able to. In addition, since the information processing system 1 can cache all the memories, the latency in memory access can be easily hidden.

  In addition, the CPU 21 accesses the shared area of the memory accessed by other CPUs in the same manner as when accessing the memory 22. That is, the arithmetic unit 31 included in the CPU 21 only needs to output a logical address regardless of whether the storage area to be accessed exists on the memory 22 or on another memory.

  For this reason, the CPU 21 can easily access the shared area without executing processing such as exclusive control of I / O, programming, and the like, so that the memory access performance can be improved. In addition, the CPU 21 can appropriately use the shared memory without modifying the program to be executed or the OS. As a result, the prefetch process can be executed in the same manner as before, thereby improving the memory access performance. Can be made.

  If the CPU 21 determines that the target of memory access from another CPU is access to the local area, it returns a negative response. For this reason, the information processing system 1 can prevent errors as a result of preventing access to areas other than the shared area.

  Further, the cache directory management unit 36 uses the node map 34 to convert the physical address into a CPU ID stored in association with the node map 34. Therefore, the CPU 21 can identify the CPU that accesses the memory to which the physical address to be accessed is allocated.

  The CPU 21 controls cache coherence using a directory that manages a CPU that caches data stored in the memory 22. For this reason, the information processing system 1 can efficiently maintain cache coherence without increasing the traffic of the XB 2 even when the number of CPUs included in the information processing system 1 increases.

  Specifically, in the information processing system 1, the communication between the CPUs is limited between the remote CPU and the home CPU, or between the remote CPU and the home CPU and the local CPU that caches updated data. For this reason, the information processing system 1 can hold cache coherence efficiently.

  Although the embodiments of the present invention have been described so far, the embodiments may be implemented in various different forms other than the embodiments described above. Therefore, another embodiment included in the present invention will be described below as a second embodiment.

(1) About the method of detecting node abnormalities,
In the first embodiment described above, the information processing system 1 has exemplified the case of detecting a node abnormality by detecting a request timeout, but the disclosed system is not limited thereto. For example, the disclosed system can also detect a node abnormality by checking the status of service processors between service processors connected to each other at predetermined time intervals. Further, the disclosed system can detect an abnormality of a node by performing “alive check” between nodes via a LAN at a predetermined time interval. Thereby, the node abnormality can be detected asynchronously with the timing at which the request is transmitted.

  FIG. 34 is a schematic diagram of an information processing system for explaining an example of a method for detecting an abnormality of a node. The example of FIG. 34 shows a case where the building box is expressed as “BB”. In the example of FIG. 34, each BB CPU has an abnormality detection circuit 72, own node number information 73, a node down notification unit 61, an interrupt generation circuit 75, a node down reception unit 62, and a node down information register 74.

  The abnormality detection circuit 72 detects an abnormality of the own node. The own node number information 73 is information indicating the identification number of the own node. When the abnormality detection circuit 72 detects an abnormality of the own node, the node down notification unit 61 includes the type of abnormality and the identification number of the own node indicated by the own node number information 73 in the node down notification packet. Send to XB. An example of the type of abnormality includes node down, hang, and information indicating which CPU has an abnormality. Further, the information indicated by the own node number information 73 may be anything as long as the information can identify the node. For example, when the relationship between the node and the CPU mounted on the node is defined in advance, the node number is known from the CPU ID, so the information indicated by the own node number information 73 may be the CPU ID.

  Upon receiving the node down notification packet transmitted from XB, the node down receiving unit 62 sets the type of abnormality included in the node down notification packet and the identification number of the own node in the node down information register 74. When the type of abnormality and the identification number of the own node are set in the node down information register 74, the software can deal with the abnormality using the set information. Further, when receiving the node down notification packet, the node down reception unit 62 outputs a control signal for causing the interrupt generation circuit 75 to generate an interrupt. When receiving the control signal from the node down reception unit 62, the interrupt generation circuit 75 generates an interrupt as in the first embodiment. Therefore, the processing after the interrupt is generated by this interrupt is the same as that of the first embodiment.

  In the example of FIG. 34, the ASIC of each BB includes an abnormality detection circuit 70, own node number information 71, and a node down notification unit 60.

  The abnormality detection circuit 70 detects an abnormality of the own node. The own node number information 71 is information indicating the identification number of the own node. When the abnormality detection circuit 70 detects an abnormality of the own node, the node down notification unit 60 includes the type of abnormality and the identification number of the own node indicated by the own node number information 71 in the node down notification packet. Send to XB.

  The node down notification units 60 and 61 can transmit a plurality of node down notification packets to the XB and transmit the node down notification packets to a plurality of CPUs. Further, the node down notification units 60 and 61 can transmit one node down notification packet to XB, and XB can transmit the node down notification packet to a plurality of CPUs. The node down notification units 60 and 61 can also transmit a node down notification packet to one CPU for each node.

  The flow of processing when a method different from the abnormality detection method according to the first embodiment is used will be described with reference to FIG. FIG. 35 is a flowchart for explaining the flow of processing when a method different from the abnormality detection method of the first embodiment is used. For example, the CPU 21 determines whether or not a node abnormality has been detected using any of the abnormality detection methods described in the second embodiment (step S1601). When the node abnormality is not detected (No at Step S1601), the CPU 21 performs the determination at Step S1601 again. On the other hand, when a node abnormality is detected (Yes at step S1601), the CPU 21 determines whether or not the node where the abnormality is detected is a remote node (step S1602).

  When the node where the abnormality is detected is a remote node (Yes at Step S1602), the OS executed by the CPU 21 performs the following process. That is, the OS takes the consistency of the directory 36a (step S1603) when the information of the node where the abnormality has occurred remains in the directory 36a of the own node 20, and ends the processing. Here, an example of how to maintain directory consistency will be described. For example, the CPU 21 performs the following processing when the down node caches data but the cache state is “clean”. That is, the CPU 21 performs a recovery process for changing the information of the cache directory 36a, which is the “clean”, to a state where the “down node does not have a cache”. Further, the CPU 21 performs the following process when the down node caches data and the cache state is “dirty”. That is, the CPU 21 performs a recovery process for changing the cache line that is “dirty” to an error state.

  On the other hand, when the node where the abnormality is detected is not a remote node, that is, when it is a home node (No in step S1602), the OS executed by the CPU 21 performs the following process. That is, the OS deletes the node information of the cache when there is information on the node where the abnormality has occurred in the cache of the own node (step S1604). If the physical address of the node where the abnormality has occurred is registered in the TLB 35a of the own node, the OS deletes all entries including the physical address of the node where the abnormality has occurred (step S1605), finish.

  Next, when the information processing system 1 is an application that performs a cluster operation, the cluster management manager can detect a node down in the cluster software. Thus, the flow of processing when the cluster management manager detects an abnormality will be described with reference to FIG. FIG. 36 is a flowchart for explaining the flow of processing when the cluster management manager detects an abnormality. For example, the cluster management manager determines whether or not a node abnormality is detected (step S1701). If no node abnormality is detected (No at Step S1701), the cluster management manager performs the process at Step S1701 again.

  On the other hand, if a node abnormality is detected (Yes at step S1701), the cluster management manager requests the application running on the own node to start a reconfiguration process due to node down (step S1702). Subsequently, the application requests the OS to release the shared memory existing in the node where the abnormality has occurred (step S1703). Thereafter, the OS deletes the released shared memory data, and deletes the TLB entry (step S1704). The application performs a unique recovery process (step S1705) and ends the process.

(2) Building Block The information processing system 1 described above has building blocks 10 to 10e having four CPUs. However, the embodiment is not limited to this, and the building blocks 10 to 10e may include an arbitrary number of CPUs and a memory accessed by each CPU. The CPU and the memory do not have to correspond one-to-one, and the CPU that directly accesses the memory may be a part of the whole.

(3) Packets transmitted by the CPU The CPU 21 described above transmits a packet having a CPU ID and PA as a memory access request. However, the embodiment is not limited to this. That is, the CPU 21 may output a packet storing arbitrary information as long as it can uniquely identify the CPU that accesses the memory to be accessed.

  Further, for example, the CPU 21 may convert the CPU ID into a VC (Virtual Connection) ID and store the VCID. The CPU 21 may store information such as a length indicating the data length in the packet.

(4) Regarding Orders (Instructions) Issued by the CPU As described above, each of the CPUs 21 to 21c issues a request or an order to maintain cache coherence. However, the requests and orders described above are merely examples, and for example, the CPUs 21 to 21c may issue a CAS (Compare And Swap) instruction.

  As described above, when the CPUs 21 to 21c issue a CAS command, even if exclusive control contention frequently occurs among a plurality of CPUs, processing is performed on the cache of each CPU. As a result, the CPUs 21 to 21c can prevent a delay due to the occurrence of memory access and also prevent the transactions between the CPUs from becoming congested.

(5) Control via Hypervisor In the information processing system 1 described above, the example in which the OS accesses the address conversion unit 35 that is hardware has been described. However, the embodiment is not limited to this. For example, a hypervisor (HPV) that operates a virtual machine may access the address conversion unit 35.

  That is, in the node on which the hypervisor operates, the OS does not directly operate the hardware resources of the CPUs 21 to 21c such as the cache and the MMU, but requests the hypervisor to perform the operation. As described above, when receiving control via the hypervisor, each of the CPUs 21 to 21c converts a virtual address into a real address (RA) and then converts the real address into a physical address. .

  In the node where the hypervisor operates, the interrupt processing does not directly interrupt the OS, but interrupts the HPV. In such a case, the hypervisor performs an interrupt by reading the OS interrupt handler. Note that the processing executed by the hypervisor described above is a known processing executed to operate a virtual machine.

(6) Processing Using Partition In the information processing system 1 described above, each of the CPUs 21 to 21c transmits a memory access using one node map. However, the embodiment is not limited to this. For example, each of the building blocks 10 to 10e may operate as a plurality of node groups, and each node group may constitute one logical partition that operates the same firmware (hypervisor).

  In such a case, each of the CPUs 21 to 21c has a node map indicating the access destination CPU and a node map indicating the CPUs in the same logical partition. As described above, each of the CPUs 21 to 21c has the node map indicating the CPUs included in the same logical partition, and therefore should not transfer the error occurrence notification, the down request, the reset request packet, etc. beyond the logical partition. The transfer range of the special packet can be identified.

  Hereinafter, a CPU having a node map showing CPUs included in the same logical partition will be described. FIG. 37 is a diagram for explaining an example of the information processing system. As shown in FIG. 37, the building blocks 10 and 10a operate the logical partition #A, and the building blocks 10b to 10d operate the logical partition #B.

  Here, in the logical partition #A, a plurality of domains #A to #C and firmware #A operate. In the logical partition #B, a plurality of domains #D to #G and firmware #B operate. Firmware #A and firmware #B are, for example, hypervisors. In the domain #A, the application and the OS operate, and in the other domains #B to #G, the application and the OS operate as in the domain #A.

  That is, each of the domains #A to #G is a virtual machine on which an application and an OS operate independently. Here, each of the CPUs 21 to 21c included in the building block 10 may transmit the above-described special packet to each CPU included in the partition #A. However, a special packet may be transmitted to each CPU included in the partition #B. Should not send packets.

  For this reason, the CPU of each building block 10 to 10d has a node map indicating the CPU ID of the CPU included in the same logical partition. For example, the CPU 21 includes a node map 34 that stores a physical address and a CPU ID of a CPU connected to a memory having a storage area indicated by the physical address in association with each other. Further, the CPU 21 has a node map 34a that stores the CPU ID of the CPU included in the same partition as the CPU 21, that is, the partition #A. Note that the node map 34 a is set by the service processor 24 in the same manner as the node map 34.

  Hereinafter, an example of a node map indicating CPU IDs of CPUs included in the same logical partition will be described with reference to the drawings. FIG. 38 is a diagram for explaining an example of a partition. For example, in the example shown in FIG. 38, the partition #A has a building block # 0. The building block # 0 includes a CPU # 0 and a memory to which an address area “# 0” is assigned.

  Further, the partition #B has a building block # 1 and a building block # 2. The building block # 1 includes a CPU # 4, a CPU # 5, a memory to which the address area “# 1” is assigned, and a memory to which the address area “# 2” is assigned. The CPU # 4 accesses the memory to which the address area “# 1” is assigned, and the CPU # 5 accesses the memory to which the address area “# 2” is assigned. The building block # 2 includes a memory to which the CPU # 8 and the address area “# 3” are assigned.

  Next, a node map of CPU # 0 and a node map of CPU # 4 shown in FIG. 38 will be described with reference to FIGS. 39A to 39C. First, a node map stored in the CPU of partition #A will be described with reference to FIGS. 39A and 39B. FIG. 39A is a diagram for describing an example of a node map stored by the CPU of partition #A. FIG. 39B is a diagram for describing an example of a node map indicating the partition #A.

  In the following description, the node ID “0” indicates the building block # 0, the node ID “1” indicates the building block # 1, and the node ID “2” indicates the building block # 2. The CPU ID “0” is the CPU ID of the CPU # 0, the CPU ID “4” is the CPU ID of the CPU # 4, the CPU ID “5” is the CPU ID of the CPU # 5, and the CPU ID “8” is It is assumed that the CPU ID is CPU # 8.

  For example, in the example shown in FIG. 39A, the node map 34 indicates that the address area “# 0” exists in the building block # 0 and the CPU # 0 performs access. Further, the node map 34 indicates that the address area “# 1” exists in the building block # 1 and the CPU # 4 performs access. Further, the node map 34 indicates that the address area “# 2” exists in the building block # 1, and the CPU # 5 performs access. Further, the node map 34 indicates that the address area “# 3” exists in the building block # 2, and the CPU # 8 performs access.

  FIG. 39B shows a node map indicating the partition #A. As illustrated in FIG. 39B, the node map indicating the partition #A includes a valid, a node ID, and a CPU ID in each entry. For example, in the example shown in FIG. 39B, the node map indicates that partition #A includes CPU # 0 of building block # 0.

  For example, in the example shown in FIG. 38, the CPU # 0 has the node maps shown in FIGS. 39A and 39B. Then, when performing memory access, the CPU # 0 identifies the access destination CPU using the node map shown in FIG. 39A. On the other hand, when transmitting a special packet only to CPUs in the same partition, CPU # 0 identifies the destination CPU using the node map shown in FIG. 39B. That is, the CPU # 0 transmits the special packet to the CPU in the partition #A indicated by the node map illustrated in FIG. 39B.

  On the other hand, CPU # 4 has a node map shown in FIG. 39A and a node map shown in FIG. 39C in order to perform memory access. Here, FIG. 39C is a diagram for describing an example of a node map indicating the partition #B. In the example shown in FIG. 39C, the node map indicating the partition #B indicates that the CPU # 4 and CPU # 5 of the building block # 1 and the CPU 38 of the building block # 2 exist in the partition #B. CPU # 4 transmits a special packet to the CPU in partition #B indicated by the node map illustrated in FIG. 39C.

  As described above, the CPU # 1 and the CPU # 4 store the node map in which the address area and the CPU ID are associated with each other and the node map indicating the partition. Then, CPU # 1 and CPU # 4 perform direct memory access to the memory of other nodes using a node map in which address areas and CPUIDs are associated with each other. Further, the CPU # 1 transmits a special packet by using the node map indicating the partition #A. CPU # 4 transmits a special packet using a node map indicating partition #B.

  Thus, each CPU may have a node map having a different value for each partition including itself. Further, when each CPU has a node map having a different value for each partition including itself, it is possible to prevent the special packet from being transmitted beyond the partition.

  Each CPU may indicate an address area to be accessed by a start address and an address mask, or a start address and a length, as in the above embodiment. That is, the CPU # 1 and the CPU # 4 identify the node to be accessed using the start address and the address mask, or the node map indicating the address area to be accessed by using the start address and the length. To do. In addition, CPU # 1 and CPU # 4 transmit special packets using node maps indicating different partitions.

(7) Control via service processor In the information processing system 1 described above, the example in which the service processor 24 accesses the node map 34 as hardware has been described. However, the embodiment is not limited to this, and a configuration other than the service processor 24 may access the node map 34. For example, a basic firmware BIOS (Basic Input / Output System) or HPV operating on one or all of the CPUs 21 to 21c may be configured to access the node map 34.

1 Information processing system 2 XB
10-10e building block 20 nodes 21-21c CPU
22 to 22c Memory 23, 26 Communication unit 24 Service processor 25 Control unit 27, 27a XB connection unit 28 PCIe connection unit 30 Calculation processing unit 31 Calculation unit 32 L1 cache 33 L2 cache 34 Node map 35 Address conversion unit 36 Cache directory management unit 37 packet control unit 37a packet generation unit 37b packet reception unit 37c abnormality detection unit 40 router 41 memory access unit 42 PCIe control unit 42a request generation unit 42b PCIe bus control unit 42c abnormality detection unit

Claims (10)

An information processing apparatus having a plurality of nodes each provided with a storage device and an interconnect connecting the plurality of nodes,
At least one of the plurality of nodes is
An abnormality in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in a storage device included in the one node or another node, or the one node Or detecting an abnormality in any node that can be included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in a storage device included in another node An anomaly detector;
Based on the anomaly detected by the anomaly detection unit, it generates error information about the area to be accessed in the data transfer in which the anomaly is detected among the areas included in the shared memory area, and for the processor in its own node An error information generator that generates an interrupt;
A processor that executes a recovery process for a shared memory including an area to be accessed in the data transfer in which the abnormality is detected based on the error information in response to the interrupt;
Have
The error information generation unit writes back the cache data of the remote node to the shared memory area of the home node due to an abnormality of the remote node when the abnormality is detected by the abnormality detection unit. If not, write data indicating abnormality to the directory,
The processor deletes the data indicating the abnormality from the directory when executing the recovery process. An information processing apparatus having a plurality of nodes each provided with a storage device and an interconnect connecting the plurality of nodes,
At least one of the plurality of nodes is
An abnormality in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in a storage device included in the one node or another node, or the one node Or detecting an abnormality in any node that can be included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in a storage device included in another node An anomaly detector;
Based on the anomaly detected by the anomaly detection unit, it generates error information about the area to be accessed in the data transfer in which the anomaly is detected among the areas included in the shared memory area, and for the processor in its own node An error information generator that generates an interrupt;
A processor that executes a recovery process for a shared memory including an area to be accessed in the data transfer in which the abnormality is detected based on the error information in response to the interrupt;
Have
The error information generation unit writes back the cache data of the remote node to the shared memory area of the home node due to an abnormality of the remote node when the abnormality is detected by the abnormality detection unit. If not, write data indicating an error state in the shared memory area,
The processor clears data indicating the error state from the shared memory area when executing the recovery process. The processor notifies the signal handler corresponding to the shared memory of the node where the abnormality is detected by the abnormality detection unit,
When the signal is notified, the signal handler deletes the address of the shared memory of the node where the abnormality is detected by the abnormality detection unit from the cache memory, and the logical address and the physical address are registered in association with each other. In the table, the correspondence between the physical address of the storage device included in the node corresponding to the abnormality detected by the abnormality detection unit and the logical address corresponding to the physical address is canceled, and the physical address is transferred to the table. The information processing apparatus according to claim 1, wherein the information processing apparatus is treated as a physical address that prohibits registration. The signal handler further executes a recovery process on the shared memory area when the shared memory area of the one node is used by a node in which an abnormality is detected by the abnormality detection unit. The information processing apparatus according to claim 3. Each of the plurality of nodes has a service processor;
The information processing apparatus according to claim 1, wherein the abnormality detection unit detects an abnormality by checking a state at a predetermined time interval between service processors of each node. Each of the plurality of nodes has a node down notification unit and a node down reception unit,
When the node down notification unit of the node in which the abnormality is detected detects the abnormality of the own node, the node down notification unit transmits a packet including information on the detected abnormality through the interconnect.
The node down reception unit of another node different from the node in which the abnormality is detected generates an interrupt based on information included in the received packet. Information processing device. At least one node among the plurality of nodes in the information processing apparatus having a plurality of nodes each having a storage device and an interconnect connecting the plurality of nodes,
An abnormality in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in a storage device included in the one node or another node, or the one node Or detecting an abnormality in any node that may be included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in a storage device included in another node ,
Based on the detected abnormality, among the areas included in the shared memory area, generates error information about the area to be accessed in the data transfer in which the abnormality is detected, and generates an interrupt to the processor in the own node,
Based on the error information in response to the interrupt, a process for executing a recovery process for a shared memory including an area to be accessed in the data transfer in which the abnormality is detected is executed.
In the processing that generates the interrupt, the node in which the abnormality is detected by the processing for detecting the abnormality is a remote node, and the cache data of the remote node is stored in the shared memory area of the home node due to the abnormality of the remote node. If is not written back, write data indicating abnormality to the directory,
The process for executing the recovery process deletes the data indicating the abnormality from the directory when the recovery process is executed. At least one node among the plurality of nodes in the information processing apparatus having a plurality of nodes each having a storage device and an interconnect connecting the plurality of nodes,
An abnormality in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in a storage device included in the one node or another node, or the one node Or detecting an abnormality in any node that may be included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in a storage device included in another node ,
Based on the detected abnormality, among the areas included in the shared memory area, generates error information about the area to be accessed in the data transfer in which the abnormality is detected, and generates an interrupt to the processor in the own node,
Based on the error information in response to the interrupt, a process for executing a recovery process for a shared memory including an area to be accessed in the data transfer in which the abnormality is detected is executed.
In the processing that generates the interrupt, the node in which the abnormality is detected by the processing for detecting the abnormality is a remote node, and the cache data of the remote node is stored in the shared memory area of the home node due to the abnormality of the remote node. Is written back, the data indicating the error state is written to the shared memory area,
The control program characterized in that the process of executing the recovery process clears data indicating the error state from the shared memory area when executing the recovery process. At least one node among the plurality of nodes in the information processing apparatus having a plurality of nodes each provided with a storage device and an interconnect connecting the plurality of nodes,
An abnormality in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in a storage device included in the one node or another node, or the one node Or detecting an abnormality in any node that may be included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in a storage device included in another node ,
Based on the detected abnormality, among the areas included in the shared memory area, generates error information about the area to be accessed in the data transfer in which the abnormality is detected, and generates an interrupt to the processor in the own node,
In response to the interrupt, based on the error information, a process for executing a recovery process for a shared memory including an area to be accessed in the data transfer in which the abnormality is detected is executed.
In the processing that generates the interrupt, the node in which the abnormality is detected by the processing for detecting the abnormality is a remote node, and the cache data of the remote node is stored in the shared memory area of the home node due to the abnormality of the remote node. If is not written back, write data indicating abnormality to the directory,
The control method, wherein the process of executing the recovery process deletes the data indicating the abnormality from the directory when the recovery process is executed. At least one node among the plurality of nodes in the information processing apparatus having a plurality of nodes each provided with a storage device and an interconnect connecting the plurality of nodes,
An abnormality in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node included in a storage device included in the one node or another node, or the one node Or detecting an abnormality in any node that may be included in a data transfer path of data transfer using a shared memory area that can be shared by the one node and the other node, included in a storage device included in another node ,
Based on the detected abnormality, among the areas included in the shared memory area, generates error information about the area to be accessed in the data transfer in which the abnormality is detected, and generates an interrupt to the processor in the own node,
In response to the interrupt, based on the error information, a process for executing a recovery process for a shared memory including an area to be accessed in the data transfer in which the abnormality is detected is executed.
In the processing that generates the interrupt, the node in which the abnormality is detected by the processing for detecting the abnormality is a remote node, and the cache data of the remote node is stored in the shared memory area of the home node due to the abnormality of the remote node. Is written back, the data indicating the error state is written to the shared memory area,
The control method characterized in that the process of executing the recovery process clears data indicating the error state from the shared memory area when the recovery process is executed.

Images