JP4984051B2 - Dynamic degeneration apparatus and method - Google Patents

Description

  The present invention relates to a dynamic degeneration apparatus and method for an I / O device that is duplicated into a master device and a slave device.

  Conventionally, a fault-tolerant computer or the like is known as a device that doubles an I / O controller device typified by a PCI device and improves fault tolerance of the system. For fault-tolerant computers, etc., a method is adopted in which access to the duplicated I / O device is absorbed by the device driver layer of the software so that it can be seen as a single I / O device from the upper layer operating system and applications. Has been. The problem with this method is that a dedicated device driver is required to realize duplexing.

  In order to solve this problem, there is also known a system in which duplication of I / O devices is realized by a hardware layer. However, when the I / O device duplexed by the hardware layer function is degraded due to a failure or the like, a method is taken in which the system is once shut down and then restarted after one of them is disconnected. . This degeneration method is problematic from the viewpoint of fault tolerance of the system.

  Japanese Patent Laying-Open No. 2000-148523 (see Patent Document 1) describes the invention of a “double memory device”. In this dual memory device, the shared memory unit stores data handled by a plurality of processors connected by a bus in duplicate in the active memory and the standby memory. The failure detection unit detects occurrence of a failure in the operational memory. The memory switching unit voluntarily switches the data storage destination from the active memory to the standby memory without arbitrating the right to use the bus when the failure detection unit detects the occurrence of a failure.

  Japanese Patent Laid-Open No. 2001-331467 (see Patent Document 2) describes the invention of a “computer system”. This computer system includes a plurality of computers that can perform arithmetic processing independently, a storage device, and a network. The storage device provides a shared storage area for storing data commonly used by the plurality of computers. The network connects these multiple computers and storage devices. Each computer secures an area corresponding to the storage device on the system memory, and accesses the shared storage area in response to the access to the area. A server device is provided between the storage device and the network. The server device manages the shared storage area and performs processing according to the access from each computer, monitors the operating state of each computer, and according to the monitoring result, each computer accesses the shared storage area. Control permission / prohibition of writing.

  Japanese Patent Laying-Open No. 5-313831 (see Patent Document 3) describes an invention of a “duplex volume control system”. In this duplex volume control method, the duplex volume is accessed to control the volume. A redundant logical volume address storage means; and an operational status storage means indicating the status of the redundant volume. Based on the information in these means, control for the duplex volume is executed. In other words, when a failure occurs in one system of an I / O device that is duplexed and always operates in synchronization with the status of both systems, the overhead when separating the faulty system and continuing operation is reduced. .

  Japanese Patent Laid-Open No. 11-242608 (see Patent Document 4) describes the invention of a “duplex control system”. This redundant control system is a redundant control system that employs data from a master device among the devices when duplicate data is periodically transmitted from two or more devices. Two or more memory means are provided corresponding to each of the two or more devices, each of which stores data transmitted from the corresponding device. The master designating means designates one of the two or more memory means as storing data from the master, and when data arrival from a device corresponding to the memory means is delayed, Redesignate the memory means as storing data from the master.

JP 2000-148523 A JP 2001-331467 A JP-A-5-313831 Japanese Patent Laid-Open No. 11-242608

  An object of the present invention is to provide valid information indicating that duplication of an I / O device is effective in a routing table used when routing a transaction, and a slave indicating a route to a slave device associated with the effective information. By adding the device identification information, the I / O device can be duplicated at the hardware level and the master device can be dynamically degenerated.

  [Means for Solving the Problems] will be described below using the numbers and symbols used in [Best Mode for Carrying Out the Invention]. These numbers and symbols are added in parentheses in order to clarify the correspondence between the description of [Claims] and [Best Mode for Carrying Out the Invention]. However, these numbers and symbols should not be used for the interpretation of the technical scope of the invention described in [Claims].

  The present invention is a dynamic degeneration apparatus for an I / O device that is duplicated by a master device (30) and a slave device (40). The transaction routing controller (143) receives, routes, and sends a transaction. The routing table (142) includes valid information (54) indicating that duplication of the I / O device is valid, and two pieces of identification information associated with the valid information (54), and the master device (30). Master device identification information (53) indicating a route to the slave device, and slave device identification information (55) indicating a route to the slave device (40). The service processor (21) controls the transaction routing controller (143) and the routing table (142).

  In the dynamic degeneration apparatus according to the present invention, the routing table (142) includes a routing record. The routing record stores a first field for storing the valid bit information (54), a second field for storing the master device identification information (53), and a second field for storing the slave device identification information (55). And three fields. In the dynamic degeneration apparatus according to the present invention, when the I / O device is a multifunction I / O device, the routing table (142) includes the valid information (54) and the valid information (54). The master device identification information (53) associated with the slave device identification information (55) and the slave device identification information (55) are provided in function units. In the dynamic degeneration apparatus according to the present invention, the service processor (21), when duplication of I / O devices is effective, in the memory mapped I / O space, the master device (30) and the slave device. (40) is allocated only one address area.

  In the dynamic degeneration apparatus according to the present invention, when the transaction routing controller (143) receives a transaction, the transaction routing controller (143) refers to the memory-mapped I / O space to thereby make an I / O device as a destination of the transaction. Is determined. When the valid information (54) related to the I / O device indicates valid by referring to the routing table (142), the master device identification information (53) and the slave device Based on the identification information (55), the transaction is sent to both the master device (30) and the slave device (40). In the dynamic degeneration apparatus according to the present invention, when the transaction routing controller (143) receives a reply transaction from the slave device (40), the transaction routing controller (143) discards the reply transaction without performing routing.

  The dynamic degeneration method according to the present invention is a method for dynamically degenerating a master device (30) from a duplexed I / O device using the above-described dynamic degeneration apparatus. In the receiving step, the transaction routing controller (143) receives a transaction. In the routing step, the transaction routing controller (143) routes the transaction. The step of referring refers to the routing table (142). In the sending step, when the routing destination is an I / O device and the validity information (54) related to the I / O device indicates validity, the transaction routing controller (143) Based on the master device identification information (53) and the slave device identification information (55), the transaction is sent to both the master device (30) and the slave device (40). In the discarding step, when the transaction routing controller (143) receives a reply transaction from the slave device (40), it is discarded without routing the reply transaction. When degenerating the master device (30) from the duplexed I / O device, the following steps are executed. In the step of inhibiting, the service processor (21) inhibits transmission by the transmitting step. The step of waiting is to wait for a certain time. In the step of degeneration, the service processor (21) degenerates the master device (30). In the updating step, the service processor (21) updates the routing table (142). In the releasing step, the service processor (21) releases the inhibition by the inhibiting step.

  The dynamic degeneration method according to the present invention is a method for dynamically degenerating a master device (30) from a duplexed I / O device using the above-described dynamic degeneration apparatus. In the step of inhibiting, the service processor (21) inhibits the transaction routing controller (143) from sending out a transaction. The step of waiting is to wait for a certain time. In the step of degeneration, the service processor (21) degenerates the master device (30). In the updating step, the service processor (21) updates the routing table (142). In the releasing step, the service processor (21) releases the inhibition by the inhibiting step.

  According to the present invention, in the routing table used when routing a transaction, valid information indicating that duplication of an I / O device is valid, and a slave indicating a route to a slave device associated with the valid information By adding the device identification information, the I / O device can be duplicated at the hardware level, and the master device can be dynamically degenerated.

  When implementing duplexing of I / O devices in the hardware layer, for example, considering the duplexing of I / O devices equipped with memory, the write from the processor to the memory on the I / O device is replicated, For writing to both I / O devices and reading from the I / O devices to the processor, a method of mounting a mechanism for accessing from only one of the I / O devices in a chip set is conceivable. This method works well even when I / O devices are replaced with interfaces such as Fiber Channel (FibreChannel) or SCSI (Small Computer System Interface), and each is replaced with a storage device of the same specifications. Let's go.

  However, the problem with this method is that if a failure occurs in one of the paths from the two chipset systems to the memory or storage device, the path can be dynamically degenerated while continuing to operate the system. It is in a difficult point. The reason is that the access latency to both memory and storage devices generally does not match, so if one path is degenerated at any timing, the in-process transaction is lost and the system continues to operate. This is because it may not be possible.

  The problem associated with such dynamic degradation of I / O devices is that the I / O device has a system core I / O, that is, a real-time clock (RTC), an interrupt controller, and a non-volatile memory (NvRAM) that are essential for the system configuration It becomes even more serious with integrated multi-function I / O cards. For example, considering that only the non-volatile memory in the multi-function I / O card is duplicated and other functions are not duplicated, the above method cannot be used at all.

  FIG. 1 shows a block configuration of a computer system in this embodiment. FIG. 1 shows a configuration in which a duplexed I / O device in a computer system, particularly a multifunction PCI device having a plurality of functions in one I / O device, is duplexed in a master / slave system. In this embodiment, it is possible to duplicate the multifunction I / O device in function units and to dynamically switch from the master device to the slave device during system operation.

  In FIG. 1, processors (Central Processing Units) 11 and 12 are connected to the same front side bus 13 to form a symmetric multiprocessing configuration. The memory controller 14 connected to the FSB 13 manages transaction routing among the CPUs 11 and 12, the memory 15, and the I / O devices 30 and 40. Further, a memory 15 and a plurality of I / O controllers 16 and 17 are connected to the memory controller 14. The I / O controllers 16 and 17 are bus bridges that connect the memory controller 14 and a plurality of PCI components (Peripheral Components Interconnect Bus) 18 and 19. An I / O device 30 is connected to the PCI bus 18, and an I / O device 40 is connected to the PCI bus 19. The I / O device 30 is a multifunction I / O device in which a non-volatile memory (Nonvolatile RAM) 31, a real time clock (Real Time Clock) 32, and an external I / O interface 33 are mounted. The I / O device 40 also has the same constituent elements 41 to 43 as the I / O device 30. The external I / O interfaces 33 and 43 are each connected to an external input / output device.

  In addition, the computer system shown in FIG. 1 is equipped with a service processor 21 that controls the operation of the system. The service processor 21 can control initialization, mode setting, reset, and the like for each element in the system including the memory controller 14. In the figure, communication paths that are not directly related to the description of the present embodiment are not shown.

  In the present embodiment, the number of processors and memories connected to the computer system does not directly relate to the requirements of the present invention, and it is apparent that the present invention can be implemented in any configuration. Similarly, the number of I / O controllers and the number of PCI buses connected to the I / O controller are not involved in the scope of rights of the present invention. In addition to the PCI bus, any I / O bus can be used in addition to the PCI-X bus and the PCI express bus. Furthermore, the functions and configurations of the I / O devices are not limited to the functions and configurations described in this embodiment.

  FIG. 2 shows a functional block diagram of the memory controller 14. Referring to FIG. 2, interfaces connecting the memory controller 14 and the elements 13 and 15 to 17 in the computer system are illustrated as an FSB interface 141, a memory interface 144, and I / O interfaces 145 and 146. A transaction routing controller 143 exists at a position connecting the interfaces 141 and 144 to 146. The transaction routing controller 143 routes transactions such as memory read, memory write, reply, and interrupt input from the interfaces 141 and 144 to 146, and outputs them to appropriate interfaces 141 and 144 to 146. The information included in these various transactions is beyond the scope of the present invention and will not be described in detail, but is a memory access target address, reply data, reply and interrupt source device ID, target device ID, and the like. Further, the transaction routing controller 143 refers to the routing table 142 in order to determine a routing route.

  FIG. 3 shows the format of the routing record stored in the routing table 142. The routing table 142 is a table composed of routing records including a series of information as shown in the figure. That is, the table consists of records storing transaction routing information for each address range in the physical main storage space. In the computer system of FIG. 1, in addition to the memory 15, registers and memory devices on the I / O devices 30 and 40 are placed on a physical storage space in a memory-mapped I / O (Memory-mapped I / O: It is also referred to as “MMIO”.). Information stored in one record of the routing table 142 is routing path information to any device mapped in the physical storage space.

  Looking at the individual fields of the routing record in FIG. 3, an address offset 51, an address range 52, and a master device ID 53 are included, and a valid bit (Vaid) 54 and a slave device ID 55 are added. The address offset 51 and the address range 52 define an address range to which the transaction is routed for memory access from the processor and the I / O device. The address offset 51 indicates the head address in each continuous memory area. The address range 52 indicates the size of the memory area. The master device ID 53 is routing path information to the memory 15 and the I / O devices 30 and 40 mapped to the address in the physical storage space. The valid bit 54 indicates that the mapped device is duplicated and routing to the duplicated slave device is valid (Vaid). The slave device ID 55 becomes routing path information to the duplicated slave I / O device.

  In the present embodiment, when the I / O device is duplexed by the master / slave method, the transaction to the master device 30 is duplicated by the transaction routing controller 143. Then, the transaction is sent to both the master device 30 and the slave device 40. Routing information for duplication is provided in the routing table 142 mounted on the memory controller 14.

  2, the transaction routing controller 143 receives transactions from the FSB interface 141 on the processors 11 and 12 side and the I / O interfaces 145 and 146 on the I / O devices 30 and 40 side. If the transaction is a memory access transaction, the target device of the transaction is determined with reference to the routing table 142. Then, the transaction is sent to the interface directed to the target device. Now, when the routing table 142 is referred to, whether or not the transaction destination device is a duplexed I / O device and there is a valid slave device, that is, whether or not routing to the slave device is valid. It turns out. As a result, when the routing to the slave device is valid, the transaction is sent to the I / O interfaces 145 and 146 toward the master device and the slave device.

  Further, when the transaction routing controller 143 receives a reply transaction for memory access from the FSB interface 141, the I / O interfaces 145 and 146, or the memory interface 144, the routing table 142 is referred to. When it is determined that the device that sent the reply transaction is a valid slave device, the reply transaction is discarded. Otherwise, route to the target device indicated by the reply transaction.

  As described above, in this embodiment, the memory access to the duplexed I / O device is duplicated by the transaction routing controller 143 and transmitted to each device. Therefore, it is possible to realize access to both the master and slave devices with a single memory access to the master device.

  The outline of the operation of the memory controller 14 has been described above. The operation of the memory controller 14 will be further described in detail. First, an initial setting method at system startup will be described. As shown in FIG. 1, the I / O devices 30 and 40 are duplexed. At this time, it is necessary to set the I / O device 30 as a master device and the I / O device 40 as a slave device. This setting is realized by writing information as shown in Table 1 from the service processor 21 to the routing table 142 when the system is started up.

  The setting contents of Table 1 will be described in detail. In the upper part of Table 1, the setting items for the I / O device 30 are defined for three records. In FIG. 1, the I / O device 30 is a multi-function I / O card equipped with three types of functions, and therefore defines an address of the memory mapped I / O space corresponding to each function one record at a time. Here, individual function names are described as A, B, and C, respectively. Further, of the three types of functions, only the function B is set to be duplicated. Therefore, the duplication effective bit corresponding to the function B is defined as true (T), and the other functions A and C are defined as false (F). Also, routing path information to the slave device corresponding to the function B is set in the field of the slave device ID. The slave device corresponding to the function B is the function B ′ of the I / O device 40 (the I / O device 40 side adds “′” and describes it as “B ′”).

  Below the table 1, setting items for the I / O device 40 on the slave side are defined for two records. In FIG. 1, the I / O device 40 is defined as a slave device, but no duplex definition is set for the functions A ′ and C ′ of the I / O device 40. Therefore, the functions A ′ and C ′ of the I / O device 40 are operated as separate devices independent of the functions A and C of the I / O device 30. Below the table 1, routing path information corresponding to these functions A 'and C' is set.

  As a result of such setting, the memory mapped I / O space of each device and function mapped on the physical storage space has a layout as shown in FIG. The functions A, B and C of the I / O device 30 and the functions A ′ and C ′ of the I / O device 40 are individually mapped on the physical storage space. In addition, it can be seen that the function B 'of the I / O device 40 is mapped to the same address as the function B of the I / O device 30, and appears to the software as a single device.

  Next, operations during system operation will be described. It is assumed that the transaction routing controller 143 receives a memory access transaction such as memory read or memory write from the processors 11 and 12 via the FSB interface 141. In this case, the transaction routing controller 143 refers to the routing table 142 and searches for a routing record corresponding to the address range specified by the transaction. When the corresponding routing record is found, the master device ID is taken out therefrom. Then, the transaction is sent to either the memory interface 144 or the I / O interfaces 145 and 146 according to the routing path information to the device that is the target of the transaction. Further, the valid bit 54 of the routing table 142 is referred to and it is confirmed whether or not the target device is a duplexed I / O device. If it is determined that the I / O device is duplicated, the transaction is transmitted to one of the I / O interfaces 145 and 146 according to the routing path information to the device specified by the slave device ID. That is, when the valid bit 54 is valid, the transaction is sent to both the master device and the slave device.

  On the other hand, when the transaction routing controller 143 receives a reply or interrupt for a transaction such as a memory read from any of the interfaces 141 and 144 to 146, the transaction routing controller 143 refers to the routing table 142. Then, a record in which a slave device ID that matches the device ID of the transmission source of the transaction is registered is searched. If there is a matching record and the valid bit 54 is true (T), the transaction is a transaction from a duplicated slave device and is discarded without being routed. In other cases, the transaction routing controller 143 determines a routing path according to the target device ID of the transaction, and sends the transaction to the appropriate interfaces 141 and 144 to 146.

  Next, a procedure for dynamically degenerating a master device of a duplexed I / O device during system operation will be described. When the master device 30 in the duplexed I / O device is degenerated, a transaction is temporarily transmitted from the I / O interfaces 145 and 146 in the memory controller 14 to the I / O devices 30 and 40 under the control of the service processor 21. Deter. The routing table 142 is updated after waiting for the preceding in-process transaction to end. By redefining the slave device 40 of the duplexed I / O devices 30 and 40 as a master device and restarting the operation of the I / O interfaces 145 and 146, master / slave switching of the I / O devices is realized. The procedure described below is realized by the service processor 21 controlling the operation of each element in the system including the transaction routing controller 143. The control procedure is executed by a program installed in the service processor 21.

  First, the service processor 21 inhibits transaction transmission from the I / O interfaces 145 and 146 of the memory controller 14 to the I / O controllers 16 and 17. As a result, the suppressed transaction temporarily stays in a buffer or the like existing in the transaction routing controller 143. Depending on the implementation, transactions that inhibit transmission from the I / O interfaces 145, 146 may be limited to only those directed to the duplexed I / O device. In this case, referring to the routing table 142, it is determined whether or not the target of the transaction is a duplexed I / O device, and transaction transmission / suppression is determined. When the service processor 21 suppresses the transmission of transactions to the I / O controllers 16 and 17, the service processor 21 does not suppress transactions from the I / O controllers 16 and 17 to the I / O interfaces 145 and 146. The memory controller 14 continues to process transactions coming from the I / O device.

  The service processor 21 waits for a certain time in this state. As a result, the reply processing and the like for the in-process transaction received from the processors 11 and 12 and the memory 15 and transmitted via the I / O interfaces 145 and 146 is completed. The waiting time is set to an appropriate value depending on the implementation of the computer system.

  After the waiting time has elapsed and processing of the preceding transaction for the I / O device is completed, the service processor 21 degenerates the master device. The I / O device degeneration method is beyond the scope of the present invention and will not be described in detail here. However, in the case of a PCI device, [a] the configuration (Configuration) of the device depends on the function of the device. ) A method of stopping the operation of the device by operating a register, [b] A method of resetting the PCI bus by operating the bus bridge if there is a bus bridge to which the device is connected, [c] A method of masking so as not to receive a transaction from the device can be considered. If the device is a passive device such as a memory device, it may be left as it is without doing anything.

  When the dynamic degeneration of the master device is completed, the service processor 21 updates the routing table 142. Specifically, of the routing route information described in the record in which the duplicated device is defined, the slave device ID information is overwritten on the master device ID, and the valid bit 54 is rewritten to false (F).

  Finally, the service processor 21 resumes the transaction transmission from the I / O interfaces 145 and 146 to the I / O controllers 16 and 17. As a result, the transaction staying in the transaction routing controller 143 is routed to the new master device in accordance with the updated routing table 142 and sent out from the I / O interfaces 145 and 146. In this way, the master device and slave device of the I / O device duplexed by the master / slave method can be exchanged during system operation.

  As described above, the present embodiment has the following effects. The first effect is that since the valid bit 54 and the slave device ID 55 are added to the routing record stored in the routing table 142, the transaction routing controller 143 searches and refers to the routing record, so that the transaction When it can be determined that the destination I / O device is duplicated, a transaction can be sent to both the duplicated master device and slave device, and the master / slave method of the I / O device is used. Duplication can be realized.

  The second effect is that the mapping of the memory mapped I / O space defined by the address information stored in the routing table 142 can be defined in units of functions of the I / O device. Is a multi-function I / O device having a plurality of functions, it is possible to selectively set whether each function is duplicated in units of functions or whether each function is operated as an individual device.

  The third effect is that the service processor 21 controls the functional blocks 141 to 146 of the memory controller 14 to stop the transaction to the I / O device for a certain period of time, degenerate the master device, and update the routing table 142. Then, by restarting transaction processing, the master device of the duplexed I / O device can be dynamically degenerated and the operation can be taken over to the slave device.

FIG. 1 is a diagram showing a block configuration of a computer system according to the present embodiment. FIG. 2 is a functional block diagram of the memory controller 14. FIG. 3 is a diagram showing a format of a routing record stored in the routing table 142. FIG. 4 is a diagram showing a layout of the memory mapped I / O space.

Explanation of symbols

11, 12 Processor 13 Front side bus 14 Memory controller 15 Memory 16, 17 I / O controller 18, 19 PCI bus 21 Service processor 30, 40 I / O device 31, 41 Non-volatile memory 32, 42 Real-time clock 33, 43 External I / O interface 51 Address offset 52 Address range 53,55 Device ID
54 Valid bits 141 FSB interface 142 Routing table 143 Transaction routing controller 144 Memory interface 145, 146 I / O interface

Claims (6)

A dynamic degeneration device for an I / O device that is duplicated into a master device and a slave device,
A transaction routing controller that receives, routes, and sends transactions;
Valid information indicating that duplication of an I / O device is effective, two pieces of identification information associated with the valid information, master device identification information indicating a route to the master device, and a route to the slave device A routing table having slave device identification information indicating
And said transaction routing controller, ingredients Bei a service processor for controlling said routing table,
The I / O device is assigned an address area in the memory mapped I / O address space,
The service processor is
When duplication of the I / O device is effective, one common address area is allocated to the master device and the slave device in the address space of the memory mapped I / O,
The transaction routing controller
When a transaction is received, by referring to the address space of the memory mapped I / O, an I / O device that is the destination of the transaction is determined,
When the valid information related to the I / O device indicates validity by referring to the routing table, the master device and the slave device identification information are based on the master device identification information and the slave device identification information. A dynamic degeneration apparatus that sends the transaction to both the slave device . The routing table consists of routing records,
The routing record is
A first field for storing the chromatic Kojo report,
A second field for storing the master device identification information;
The dynamic degeneration apparatus according to claim 1, further comprising: a third field for storing the slave device identification information. When the I / O device is a multifunction I / O device,
The routine Gute Buru is,
The dynamic degeneration apparatus according to claim 1, wherein the valid information, the master device identification information associated with the valid information, and the slave device identification information are provided in function units. The transaction routing controller
The dynamic degeneration apparatus according to any one of claims 1 to 3 , wherein when a reply transaction is received from the slave device, the reply transaction is discarded without being routed. A method for dynamically degenerating a master device from a duplexed I / O device using the dynamic degeneration apparatus according to any one of claims 1 to 3 ,
The transaction routing controller receiving a transaction;
The transaction routing controller routing a transaction;
Referring to the routing table;
When the routing destination is an I / O device and the validity information related to the I / O device indicates validity, the transaction routing controller sends the master device identification information, the slave device identification information, Sending the transaction to both the master device and the slave device based on:
When the transaction routing controller receives a reply transaction from the slave device, discarding the reply transaction without routing the reply transaction;
When degenerating a master device from a duplexed I / O device,
The service processor suppressing transmission by the transmitting step;
A step of waiting for a certain period of time;
The service processor degenerates the master device;
The service processor updating the routing table;
A dynamic degeneration method comprising: a step of releasing the inhibition by the service processor. A method for dynamically degenerating a master device from a duplexed I / O device using the dynamic degeneration apparatus according to claim 4 , comprising:
The service processor suppressing transmission of a transaction by the transaction routing controller;
A step of waiting for a certain period of time;
The service processor degenerates the master device;
The service processor updating the routing table;
The service processor has a step of releasing the inhibition by the inhibiting step.

Images