JP2010039685A - Multifunction type computer and control method for multifunction type computer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly achieve guest OS migration in the case of directly assigning an I/O device to the guest OS. <P>SOLUTION: When the virtual machine monitor (VMM) of a first physical computer moves a guest OS to a second physical computer, the VMM copies a first memory assigned to the guest OS to the second memory of a second physical computer during the operation of the guest OS, and classifies the states of the first memory into one of a copied region, a copying region, and a uncopied region, and notifies an I/O switch of the classified result, and when receiving DMA write-in T<SB>x</SB>from the I/O device, the I/O switch compares the destination address of the DMA write-in T<SB>x</SB>with the classification information of the first memory, and writes the DMA write-in T<SB>x</SB>in the first memory and the second memory when the address of the DMA write-in T<SB>x</SB>is the copied region, and writes the address of the DMA write-in T<SB>x</SB>only in the first memory when the address of the DMA write-in T<SB>x</SB>is the copying region and the uncopied region. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a virtual server technology that configures a plurality of virtual computers in one computer and operates OS on each virtual computer, and in particular, a technology for moving an OS on a virtual computer between different computers, that is, a guest OS. It relates to migration technology.

  An IT system in a company includes various servers such as an AP (Application) server that processes information for each business and a DB (Data Base) server that stores and stores business information. A computer comprising a CPU, a memory, an I / O device, etc. is used for these servers. As described above, since the IT system is composed of many servers, a blade server capable of consolidating a plurality of computers in one apparatus is used. For example, Patent Document 1 is known as this type of blade server.

  2. Description of the Related Art In recent years, improvement in semiconductor technology has improved the CPU processing performance by using a multi-core system in which a plurality of processor cores are mounted on one CPU. Along with the improvement in CPU processing performance, there is a case where the processing capability of the CPU is surplus if only one AP server or DB server is operated on one computer. For this reason, in order to efficiently use the processing capacity of the CPU, virtual computer technology for operating a plurality of virtual computers in one computer has been used. On each virtual computer, the OS and the software of the AP server and DB server operate, and it is possible to consolidate a plurality of AP servers and DB servers on one computer.

  In the virtualization technology, virtualization software, so-called virtual machine monitor (VMM) is operated on a computer, and a virtual computer is generated on the virtualization software. An independent OS can be operated on each virtual machine. Here, an OS that runs on a virtual machine is called a guest OS to distinguish it from an OS that runs on a computer in a non-virtualized environment. In the virtualization technology, a technology is known that provides a function called guest OS migration that moves and operates an OS image of a guest OS operating on a virtual computer on a certain computer on a virtual computer on another computer ( For example, Non-Patent Document 1). When this guest OS migration is used, for example, when it is desired to stop the computer on which the guest OS operates, it is possible to move the guest OS to another computer without stopping the user's business server. The original computer can be stopped and maintained.

  In the conventional system, which is the active system used for business and the standby system used as backup, OS and software licenses are required for both the active system and the standby system, whereas in the guest OS migration, one OS and software license is required. This saves license fees. Therefore, guest OS migration is gradually being used in actual operation of IT systems.

On the other hand, in computers, I / O devices such as NIC (Network Interface Card) and FC-HBA (Fiber Channel Host Bus Adapter) are used for network communication with other computers and connection of storage devices. Since the number of I / Os required for each computer (one blade) is different, it is necessary to make the number of I / O devices allocated to the computer flexible. As a technique for compensating for such a problem, a multi-root PCI switch technique is known that enables a plurality of computers and a plurality of I / O devices, which are PCI (Peripheral Component Interconnect) devices, to be connected (for example, non-patent literature). 2). With this technology, the number of PCI devices that can be connected to one computer can be changed in a scalable manner.
JP 2002-32153 A Published “Live Migration of Virtual Machines”, Christopher Clark et al., NSDI'05: 2nd Symposium on Networked Systems Design & Implementation. "Advanced Switching Technology Tech Brief" published by ASI-SIG

  In the virtualization technology as described above, generally, a method for allocating an I / O device to a guest OS includes (1) virtualization software controlling a physical I / O device, and the virtualization software is a virtual I / O device. There are two methods: a method of providing the / O device to the guest OS, and a method of (2) directly allocating the physical I / O device to the guest OS without the control of the virtualization software.

  In the former method (1), the virtual software can control the virtual I / O device, so that the guest OS migration can be easily performed. On the other hand, there is a problem that the performance is lowered because the processing of the virtual software is necessary. is there. On the other hand, the latter method (2) allocates an I / O device directly to a guest OS, so there is no performance degradation. However, in guest OS migration, an I / O device is transferred from a migration source physical computer to a migration destination physical computer. Since the connection needs to be changed, the I / O device needs to be physically connected from both computers. In order to connect this I / O device to both computers, it is possible to use the above-mentioned multi-root I / O switch technology. That is, the setting of the multi-root IO switch may be changed before and after the guest OS migration, and the connection destination of the I / O device may be changed from the migration source computer to the migration destination computer. The non-patent document 2 describes the guest OS migration of the method (1).

  However, the present inventors have found that the guest OS migration using the multi-root I / O switch technology as described above has the following problems.

  In a server used for business, it is necessary to shorten the stop time of the guest OS in order to prevent a timeout in communication between applications. On the other hand, in guest OS migration, it is necessary to copy the memory image of the guest OS to the memory area of the virtual machine on the migration destination computer. For example, if the memory size of the migration target guest OS is 5 GB and the memory image is copied with the standard interface Gigabit Ethernet (registered trademark, the same applies hereinafter), it takes at least 40 seconds to copy the memory. become. When memory copy is performed when the guest OS is stopped, the guest OS stop time takes 40 seconds or more in the above case, which is an unacceptable stop time for a business server. Therefore, instead of copying the memory while the guest OS is stopped, the memory copy is performed while the guest OS is running. It is necessary to shorten the time. Such a memory copy during operation of the guest OS is called a background copy. In the background copy, since the guest OS is operating, there is a memory update (Write) from the CPU or I / O in the copied memory area, and this updated part needs to be copied again. For this reason, if the memory update speed of the I / O device allocated to the guest OS is large compared to the memory copy speed of the interface between the migration source computer and the migration destination computer for memory copy, the memory size that needs to be copied again becomes small. There is a problem of not becoming.

  An interface having a high throughput is generally expensive, so that it is expensive to install in each computer. As an interface having a high throughput, there are, for example, 10 Gigabit Ethernet and InfiniBand. As described above, the larger the allocated memory size of the guest OS, the longer it takes to copy. Therefore, the larger the memory size, the larger the amount of memory that is updated before the copy is completed. There is a problem that the memory size is not reduced.

  Therefore, in the present invention, in the background copy of the guest OS migration when the I / O device is directly assigned to the guest OS in (2) above, in order to solve the above-mentioned problem, the copy has already been made to the multi-root I / O switch. For updates from the I / O device to the area, so-called DMA_Write_Tx (transaction), Tx (transaction, the same applies hereinafter) is duplicated, and DMA_Write is also executed in the memory area of the destination virtual machine on the destination physical computer The purpose is to realize a quick guest OS migration.

  The objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

  The present invention provides a first physical computer having a first processor and a first memory, and a first virtualization unit that virtualizes computer resources of the first physical computer and executes the first virtual computer A second physical computer including a second processor and a second memory, a second virtualization unit that virtualizes computer resources of the second physical computer and executes the second virtual computer, The first virtualization unit is executed by the first virtual computer in a composite computer comprising an I / O switch for connecting an I / O device to the first physical computer and the second physical computer. A method of controlling a composite computer that moves a first OS to the second virtual computer, wherein the first virtualization unit moves the first OS to the second virtual computer The first virtualization unit during the operation of the first OS Copying the memory area in which the memory is allocated to the first OS to the second memory of the second physical computer; and the first virtualization unit is configured to change the state of the first memory to Any one of a copied area that has been copied to the second memory, a copying area that is being copied to the second memory, and an uncopied area that has not been copied to the second memory And classifying the classified memory state and the address of the first memory to the I / O switch as classification information, and the I / O switch assigns the I to the first OS. If a DMA write transaction is received from the / O device, the destination address of the DMA write transaction is compared with the address of the classification information in the first memory; The I / O switch performs the DMA write to both the first memory and the second memory when the address of the DMA write transaction is the copied area as a result of the comparison; The I / O switch includes a step of performing a DMA write only to the first memory when the address of the DMA write transaction is the copy-in-progress region or the non-copy region as a result of the comparison.

  The I / O switch may perform the DMA write only to the first memory when the address of the DMA write transaction is the area being copied or an uncopied area as a result of the comparison. When the address of the write transaction is the area being copied, performing a DMA write to the first memory, and storing the address information of the DMA write transaction in an address storage area set in advance in the first memory; The first virtualization unit obtains the address information stored in the address storage area after the copying of the area being copied is completed, and the second memory from the first memory indicated by the address information And performing a copy.

  According to the present invention, in a guest OS (first OS) migration using background copy, even if the throughput of the memory copy interface of the migration target guest OS is low, the memory that needs to be copied again when the guest OS is stopped Can be reduced, and the stop time of the guest OS can be shortened.

  Further, according to the present invention, in the guest OS migration using the background copy, the size of the memory area that needs to be copied again when the guest OS is stopped is constant regardless of the memory size allocated to the guest OS. Even if the allocated memory size is large, the size of the memory area that needs to be re-copied can be reduced, and the stop time of the guest OS can be shortened.

  Furthermore, it is possible to reliably reduce the stop time of the guest OS during the migration of the guest OS without depending on the computer environment such as the type of interface used for the memory copy or the allocated memory size of the guest OS. Guest OS migration can be implemented with minimal impact on servers and the like. In addition, since the restrictions on the computer environment for guest OS migration are relaxed, the use of guest OS migration can be expanded.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that, in all the drawings for explaining the embodiment, the same members are denoted by the same reference numerals in principle, and the repeated description thereof is omitted in principle.

  First, the configuration of the composite computer according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram of a composite computer according to the first embodiment of the present invention.

  As shown in FIG. 1, in the present embodiment, the composite computer includes one or more physical computers 1011 and 1012 and one or more multi-root I / Os that can connect a plurality of physical computers 1011 and 1012 to an I / O device. O switch 102, one or more I / O devices 1031 and 1032 connected to multi-root I / O switch 102, and one or more device controls connected to physical computers 1011 and 1012 and multi-root I / O switch 102 The unit 104 is configured.

  The physical computers 1011 and 1012 and the I / O devices 1031 and 1032 are connected via the multi-root I / O switch 102 by the I / O interface 150. The device control unit 104, the physical computers 1011 and 1012, and the multi-root I / O switch 102 are connected by a control interface 153. An administrator who manages the complex computer manages the complex computer using the management terminal 105 of the apparatus control unit 104. The management terminal 105 is connected to the apparatus control unit 104 via the management interface 152.

  The device control unit 104 includes device control software 140, migration management software 141, and migration configuration information 142. The device control software 140 sets the power supply control of the physical computers 1011 and 1012 and the multi-root I / O switch 102 and sets the connection between the physical computers 1011 and 1012 and the I / O devices 1031 and 1032 in the multi-root I / O switch 102. It has a function to do.

  Upon receiving a guest OS migration execution request from the management terminal 105, the migration management software 141 receives information on the virtual machines 1231 and 1232 and guest OS 1241 and 1242 from the virtual machine monitors (VMM) 1201 and 1202 of the physical computers 1011 and 1012. Are acquired and stored in the migration configuration information 142, and the progress of guest OS migration is managed.

  The physical computers 1011 and 1012 include a hardware component 106, virtual machine monitors 1201 and 1202 that operate on the hardware component 106, and virtual computers 1231 and 1232 that operate on the virtual machine monitors 1201 and 1202. .

  The hardware component 106 includes at least one CPU (processor) 108, at least one chip set (control hub) 107, at least one memory 1091 and 1092, and at least one guest OS memory used for guest OS migration. A copy I / O 110 is provided. The chip set 107 has a function of a memory control hub that connects the CPU 108 and the memory 1091 and a function of an I / O control hub that connects the CPU 108 and the I / O device 1301 of the multi-root I / O switch 102. Note that the chip set 107 can be mounted separately from the memory control hub and the I / O control hub.

  Guest OSs 1241 and 1242 operate on the virtual machines 1231 and 1232, respectively. In such a compound computer, guest OS migration is performed by moving the migration source guest OS 1241 from the migration source physical computer 1011 to the migration destination physical computer 1012 and the migration destination guest OS 1242. The I / O device 1031 is directly assigned to the migration target guest OS 1242, and is allocated to the migration source guest OS 1241 before the migration, and the allocation is changed to the migration destination guest OS 1242 after the migration.

  The migration source memory 1091 includes at least a memory area (guest memory) 1111 of the migration source guest OS 1241 and a memory area (VMM area) 112 used by the virtual machine monitor 1201. The migration destination memory 1092 for copying the contents of the memory area 1111 of the migration source guest OS 1241 includes at least a memory area (guest memory) 1112 of the migration destination guest OS 1242. During guest OS migration, the memory area 1111 of the migration source guest OS 1241 is transferred to the memory area 1112 of the migration destination guest OS 1241 under the control of the background copy control unit 121 of the migration source virtual machine monitor 1201 while the migration source guest OS 1241 is operating. A background copy 160 is performed. The background copy 160 is performed via the interface wiring 151 by the memory copy interface 110.

  The migration source virtual machine monitor 1201 includes at least a migration control unit 1221 and a background copy control unit 121. The migration destination virtual machine monitor 1202 includes at least a migration control unit 1222. The migration control units 1221 and 1222 perform power control of the virtual machines 1231 and 1232, scheduling control of the guest OSs 1241 and 1242, and control of the multi-root I / O switch 102 according to a request from the migration management software 141. The background copy control unit 121 performs copy progress management of the memory area 1111 of the guest OS 1241 and controls memory copy via the interface 110.

  The multi-root I / O switch 102 includes one or more upstream ports 132 that connect the physical computers 1011 and 1012, one or more downstream ports 133 that connect the I / O devices 1031 and 1032, a switch circuit unit 131, and hardware of the present invention. A downstream port I / O control unit 130 having a hardware function. The upstream port 132 and the switch circuit unit 131, the switch circuit unit 131 and the downstream port I / O control unit 130, and the downstream port I / O control unit 130 and the downstream port 133 are connected by an interface 154 in the I / O switch.

  The downstream port I / O control unit 130 includes an Inbound_Tx (inbound transaction) control unit 134, a migration Tx control unit 1_135, and a migration Tx control unit register 1_136. The function of each component of the downstream port I / O control unit 130 will be described later. In the following description, Tx means a transaction. Inbound_Tx indicates a transaction from the I / O device to the physical computer (or virtual computer).

  Next, information held by the composite computer according to the first embodiment will be described in detail with reference to FIGS. FIG. 2 is an explanatory diagram showing an example of the migration configuration information of the composite computer according to the first embodiment, and FIG. 3 is an example of host physical address information of the migration target guest OS of the composite computer according to the first embodiment. FIG. 4 is an explanatory diagram showing an example of I / O device information to be migrated by the composite computer according to the first embodiment.

  The migration configuration information 142 held by the device control unit 104 includes migration basic information, host physical address information of a migration target guest OS, and migration target I / O device information. When the basic migration information is represented in a table format, as indicated by FT2 in FIG. 2, at least the migration source information K201, the identifier G201 of the physical computer 1011 and the identifier G202 of the virtual computer 1231 operating on the physical computer 1011 are displayed. A virtual I / O switch identifier G203 assigned to the physical computer 1011; information G204 indicating the presence / absence of a device directly assigned to the migration target guest OS 1241; and migration destination physical computer 1012 as migration destination information K202. Identifier G201, an identifier G202 of the migration destination virtual machine 1232, and a virtual I / O switch identifier G203 assigned to the migration destination physical computer 1012. The multi-root I / O switch 102 configures a virtual I / O switch for each physical computer in the I / O switch in order to share the I / O switch among a plurality of physical computers. The / O switch identifier G203 is an identifier for this virtual I / O switch. In the basic migration information FT2 in FIG. 2, the identifier G201 of the physical computer 1011 is “1”, the identifier G201 of the physical computer 1012 is “2”, the identifier G202 of the virtual computer 1231 is “1”, and the identifier G202 of the virtual computer 1232 is “ 4 ”, the migration source virtual I / O switch identifier G203 is“ 3 ”, and the migration destination virtual I / O switch identifier G203 is“ 5 ”.

  The migration management software 141 acquires the information of the physical computer identifier G201 and the virtual computer identifier G202 by, for example, a request via the management terminal 105 by the administrator of the composite computer, and the information of the virtual I / O switch identifier G203. Is acquired from, for example, the apparatus control software 140, and information on the presence / absence of the takeover device G204 is acquired from, for example, the virtual machine monitor 1201 of the migration source.

  The host physical address information of the migration target guest OS is represented in table format as host physical address information FT31 of the migration source guest OS 1241 and host physical address information FT32 of the migration destination guest OS 1242, as shown in FIGS. 3A and 3B. 3A and 3B show host physical memory allocation information for the guest OSs 1241 and 1242, and a guest physical address K301 indicating the start address of the physical address recognized by the guest OS in the memory area allocated to the guest OSs 1241 and 1242; The host physical address K302 indicating the physical address of the physical computer corresponding to the guest physical address K301, and the allocated memory area size K303 are configured.

  In general, since the host physical memory allocated to the guest OS can allocate a plurality of discontinuous memory areas, the host physical address information includes guest physical addresses K301, host physical addresses K302, A set of memory area size K303 is provided according to the number of memory areas. 3A and 3B show information when n memory areas are allocated for the migration source guest OS 1241 and m memory areas are allocated for the migration destination guest OS 1242. Since the virtual machine monitors 1201 and 1202 assign the host physical memory to the guest OSs 1241 and 1242, the migration management software 141 sends the host physical address information from the virtual machine monitors 1201 and 1202 to the guest physical address K301, the host physical address K302, and The size K303 is acquired.

  When the I / O device information to be migrated is represented in a table format, as indicated by FT4 in FIG. 4, at least the identifier K401 of the I / O device and the number of the I / O switch to which the I / O device is connected. K402 and a port number K403 of the I / O switch to which the I / O device is connected.

  Here, for example, when the I / O device is a PCI (Peripheral Component Interconnect) device, the identifier of the I / O device includes a bus number, a device number, and a function I / On number. In the example of the I / O device indicated by G401 in FIG. 4, the I / O device identifier K401 is 15: 0: 0, the I / O switch number K402 is 1, and the port number K403 is 3. .

  Since the migration target virtual machine monitor 1201 holds the migration target I / O device identifier K401, the migration management software 141 acquires the I / O device identifier K401 from the virtual machine monitor 1201. Further, since the device control software 140 holds the switch number K402 and the port number K403 to which the I / O device is connected, the migration management software 141 acquires the switch number K402 and the port number K403 from the device control software 140. . This completes the description of the information held by the composite computer.

  Next, a background copy of the composite computer according to the first embodiment will be described in detail with reference to FIGS. FIG. 5 is an explanatory diagram showing an example of the memory space of the migration source guest OS at the time of background copying of the composite computer according to the first embodiment.

  In the virtual machine monitor 1201 running on the migration source physical computer 1011, during background copy, the background copy control unit 121 divides the guest memory 1111 of the migration source guest OS 1241 into a plurality of memory areas, and the background shown in FIG. In one phase of ground copy, one of the divided memory areas is copied, and the above phase is repeated until all the divided memory areas are copied to the destination guest memory 1112.

  The example of FIG. 5 shows the state of the guest memory 1111 of the migration source guest OS 1241 in the Nth phase 11111 and the N + 1th phase 11112 of the background copy. As illustrated in FIG. 5, the background copy control unit 121 includes, for each phase, (1) copied areas 5011 and 5012, (2) copied areas 5021 and 5022, and (3) uncopied areas 5031 and 5032. Management is divided into three areas.

  In FIG. 5, GPA_MAX is the maximum address of the guest memory 1111. GPA1_N and GPA2_N are a minimum address and a maximum address indicating the area of the copy-in-progress area 5021 of the Nth phase, respectively. GPA1_N + 1 and GPA2_N + 1 are a minimum address and a maximum address indicating the area of the copy-in-progress area 5022 in the (N + 1) th phase, respectively. In the example of FIG. 5, for simplicity, each of the areas (1) to (3) is a continuous memory area, but may be a discontinuous memory area in general.

  The background copy control unit 121 transfers the areas 5011, 5021, and 5031 of (1) to (3) to the migration target I / O of the I / O switch 102 via the configuration interface 161 shown in FIG. The value is set in the migration Tx control unit register 1_136 of the downstream port I / O control unit 130 to which the device 1031 is connected. Note that since the information on the location of the migration target I / O device 1031 is held in the migration configuration information, the background copy control unit 121 acquires the location information of the I / O device 1031 from the migration management software 141. The background copy control unit 121 can use a known method such as a predetermined size unit or an area unit in which addresses are continuous in dividing the memory area of the guest memory 1111.

  Next, control of the downstream port I / O control unit 130 in the Nth phase of background copy will be described.

  When the downstream port I / O control unit 130 receives the Inbound_Tx 162 from the I / O device 1031, the Tx transfer 163 is performed via the Inbound_Tx control unit 134 in FIG. 1. When Inbound_Tx is DMA_Tx for the guest OS 1241 to be migrated, the Tx address is converted from the guest physical address to the host physical address, and the DMA access 163 is performed on the guest memory 1111 of the migration source OS.

  In addition to the normal Inbound_Tx transfer processing (163) of the Inbound_Tx control unit 134, the migration Tx control unit 1_135 performs the following processing in the case of DMA_Tx for the guest OS 1241 to be migrated.

  When Inbound_Tx 162 is DMA_Write_Tx and the destination address is (1) the copied area 5011 shown in FIG. 5, the migration Tx control unit 1_135 duplicates DMA_Write_Tx and duplicates the DMA_Write_Tx copied to the guest memory 1112 of the migration destination OS. DMA_Write 165 is performed.

  In the conventional example, (1) if there is a DMA_Write to the copied area 5011, the migration source memory is updated and the migration destination guest memory 1112 needs to be copied again. Even if DMA_Write to the original occurs, the re-copying can be made unnecessary because the destination guest memory 1112 is updated simultaneously with the source guest memory 1111.

  On the other hand, when Inbound_Tx 162 is DMA_Write_Tx and the destination address is (2) the copy-in-progress area 5012, the background copy and the memory update by DMA_Write_Tx are performed at the same time. When DMA_Write_Tx is copied and DMA_Write 165 is performed on the guest memory 1112 of the migration destination OS, the memory update order of the DMA_Write 165 and the background copy 160 cannot be guaranteed.

  Therefore, when the destination address of DMA_Write_Tx is (2) the copying area 5012, the destination address of DMA_Write_Tx is recorded as a list 164 in the address list buffer 113 in the memory area (VMM area) 112 of the virtual machine monitor 1201. deep. After replication of the copying area 5012 is completed, the migration control unit 1221 of the virtual machine monitor 1201 recopyes the memory area stored in the address list buffer 113 to the destination guest memory 1112. As a result, the same content as the update of the migration source guest memory 1111 by the DMA_Write_Tx can be applied to the migration destination guest memory 1112 as the content of the copying area 5021.

  If Inbound_Tx 162 is DMA_Write_Tx and the destination address is (3) the uncopied area 5013, copying is executed in the subsequent background copy phase, so the migration Tx control unit 1_135 performs no processing, and Inbound_Tx Only the normal Inbound_Tx transfer process of the control unit 134 is performed.

The size of the memory area that needs to be re-copied by DMA transfer in the destination guest memory 1112 is
MS1 / MSt (Formula 1)
It is possible to reduce to the size of Here, MS1 is the memory size (Gbyte) to be copied in one phase, and MSt is the total memory size of the migration source guest OS. For example, when the total memory size of the migration source guest OS is 5 GB and the memory size to be copied in one phase is 100 MB, the memory size that needs to be re-copied can be significantly reduced to 1/50. The memory size that needs to be re-copied in the present invention is expressed by the following equation.
URi × MSt / Vc × MS1 / MSt = URi × MS1 / Vc (Formula 2)
It becomes. Here, URi is the memory update speed (for example, MB / sec) at which the I / O device transfers the memory area 1111 of the guest OS by DMA_Write, and Vc is the copy speed of the background copy (for example, MB / sec). is there. From the above formula 2, it can be seen that the memory size that needs to be copied again does not depend on the total memory size MSt of the guest OS. Therefore, when performing memory copy that requires re-copying when the guest OS 1241 is stopped, the stop time of the guest OS 1241 can be made constant without depending on the total memory size of the guest OS according to the present invention.

  Next, the address list buffer 113 of the composite computer according to the first embodiment will be described in detail with reference to FIG. FIG. 6 is an explanatory diagram showing an example of an address list buffer generated at the time of background copy of the composite computer according to the first embodiment. In the example of FIG. 6, the address list buffer 113 of FIG. 1 is composed of two address list buffers 1131 and 1132 of A and B. As described above, the two address list buffers 1131 and 1132 have a write 602 from the migration Tx control unit 1_135 and a read 603 from the virtual machine monitor 1201 in the background copy of the guest OS migration. This is to prevent writing leakage from the migration Tx control unit 1_135 at the same time when the buffer is cleared. That is, in the example of FIG. 6, the address list buffer A 1131 is used for writing 602 from the migration Tx control unit 1_135, the address list buffer A 1132 is used for reading 603 from the virtual machine monitor 1201, and the reading 603 from the virtual machine monitor 1201 is completed. The address list buffer A / B is exchanged, and the address list buffer A 1131 is used for reading 603 and the address list buffer B 1132 is used for writing 602. In the example of FIG. 6, the address list buffers 1131 and 1132 show a list structure in which one entry 601 is 8B (Byte). Here, each entry is set to 8B because it is assumed that the destination address of DMA_Write_Tx is 64 bits, and the size of one entry can be changed as appropriate if the number of bits necessary to indicate the destination address changes. . Switching of the address list buffer A / B is instructed by the virtual machine monitor 1201 to the migration Tx control unit 1_135 via the configuration interface 604.

  In the example of FIG. 6, two A / B surfaces are used for the address list buffer 113, but a structure similar to the ring buffer 1 surface may be used as another embodiment.

  Next, the downstream port I / O control unit 130 of the composite computer according to the first embodiment will be described in detail with reference to FIGS. FIG. 7 is a block diagram showing an example of the circuit of the downstream port I / O control unit 130 of the composite computer according to the first embodiment, and FIG. 8 is the downstream port I / O of the composite computer according to the first embodiment. FIG. 9 is an explanatory diagram showing an example of the address list recording control register 711 of the O control unit 130. FIG. 9 shows an example of the source address information register 712 of the downstream port I / O control unit 130 of the composite computer according to the first embodiment. FIG. 10 is an explanatory diagram showing an example of the destination address information register 713 of the downstream port I / O control unit 130 of the composite type computer according to the first embodiment, and FIG. 11 is a diagram showing the first embodiment. FIG. 12 is an explanatory diagram showing an example of the address area setting register 715 of the downstream port I / O control unit 130 of the composite computer according to the embodiment. FIG. 12 shows the downstream port of the composite computer according to the first embodiment. FIG. 13 illustrates an example of the migration target device setting register 717 of the downstream port I / O control unit of the composite computer according to the first embodiment. It is explanatory drawing which showed.

  As shown in FIG. 7, the downstream port I / O control unit 130 includes an Inbound_Tx control unit 134, a migration Tx control unit 1_135, a migration Tx control unit register 1_136, and an Inbound_Tx control unit 134 and a Tx from the migration Tx control unit 1_135. And an arbitration circuit 701 that performs the arbitration and transfers the data to the upstream (physical computer side). The migration Tx control unit register 1_136 includes register groups 711 to 717.

  As shown in FIG. 8, the address list recording control register 711 includes registers 801 and 802 indicating the area of the address list storage buffer A / B for recording the destination address of Tx of DMA_Write_Tx, and the downstream port I / O control unit 130. A register 803 for designating a buffer A / B for writing an address list, and a continuous address list recording inhibition for designating that address list recording should be inhibited when the destination address of Tx is close to the destination address previously recorded for the address list A designation register 804 and a designation register 805 having a continuous address size indicating an address range of a destination address for suppressing recording of the address list are configured. In general, since DMA access involves many accesses to a continuous area of a memory, it is possible to reduce the recording amount in the address list storage buffer by suppressing the continuous address list recording. That is, it is possible to reduce the amount of memory reserved for the address list buffer 113.

  As illustrated in FIG. 9, the migration source address information register 712 includes registers 9011 to 901 n that specify the start guest physical address of the memory area n of the migration source guest OS 1241 for the n memory areas, and the memory area of the migration source guest OS 1241. The registers 9021 to 902n for specifying the size of n and the registers 9031 to 903n for specifying the start host physical address (offset) corresponding to the memory area n of the migration source guest OS 1241.

  The information set in the migration source address information register 712 is the host physical address information of the migration target guest OS in the migration configuration information 142, and is the information shown in FT31 in FIG. 3A. The migration control unit 1221 of the migration source physical computer 1011 acquires the host physical address information of the migration target guest OS from the migration management software 141 and sets the address information in the migration source address information register 712.

  The migration destination address information register 713 is a register similar to the migration source address list recording control register 711. However, as shown in FIG. 10, the migration destination guest OS 1242 has a memory area m corresponding to the m memory areas. Registers 10011 to 1001m for specifying the start guest physical address, registers 10021 to 1002m for specifying the size of the memory area m of the migration destination guest OS 1242, and a start host physical address (offset) corresponding to the memory area m of the migration destination guest OS 1242 Registers 10031 to 1003m.

  The information set in the migration destination address information register 713 is the host physical address information of the migration target guest OS in the migration configuration information 142, and is the information shown in FT32 of FIG. 3B. The migration control unit 1221 acquires the host physical address information of the migration target guest OS from the migration management software 141 and sets the address information in the destination address information register 713.

  The migration destination physical computer information register 714 designates the migration destination physical computer 1012 and is designated by, for example, the identifier of the virtual I / O switch assigned to the physical computer 1012. This is the information shown in the migration basic information of the migration configuration information 142, that is, the identifier G203 of the virtual I / O switch of the migration destination K202 in FIG.

  The migration control unit 1221 acquires the basic migration information from the migration management software 141 and sets the migration destination physical computer in the migration destination physical computer information register 714.

  As shown in FIG. 11, the address area setting register 715 is a register for designating (1) the copied area and (2) the copying area at the time of the background copy described in FIG. In the example of FIG. 11, (2) registers 11011 to 1101P for designating P memory areas as copying areas, and (1) registers 11021 to 1102Q for designating Q memory areas as copied areas. As described in FIG. 5, the amount (or number) of (1) the copied area and (2) the area being copied changes depending on the background copy phase, so the background copy control unit 121 changes the address area for each phase. The setting register 715 is reset.

  As shown in FIG. 12, the migration Tx control valid register 716 includes a register 12001 that designates enabling address list recording, which is a function of the migration Tx control unit 1_135, and a register 12002 that designates enabling replication of DMA_Write_Tx. Consists of The migration Tx control valid register 716 is set by the migration control unit 1221 at the start of guest OS migration.

  As illustrated in FIG. 13, the target device setting register 717 includes K I / O devices that the I / O device 1031 connected to the downstream side (I / O device side) of the downstream port I / O control unit 130 has. For the function, it is composed of registers 13011 to 1301K for specifying a device ID and registers 13021 to 1302K for specifying whether the corresponding device ID is directly assigned to the migration target guest OS 1241.

  As an example of an I / O device having a plurality of I / O device functions, a PCI device, a multi-function device, a multi-port device, a PCI-SIG SR-I / OV (Single Root-I / O Virtualization), or There are devices compatible with MR-I / OV (Multi Root-I / O Virtualization).

  The information set in the target device setting register 717 is the migration target I / O device information in the migration configuration information 142, and is the information shown in FT4 in FIG. The migration control unit 1221 acquires migration target I / O device information from the migration management software 141 and sets the acquired device information in the target device setting register 717. The registers of the migration Tx control register unit 1_136 are controlled by software via the control I / F 736 shown in FIG.

  Next, the configuration of the Inbound_Tx control unit 134 in FIG. 7 will be described. As an I / O device transaction received from the downstream side, an Inbound_Tx 720 from the I / O device 1031 toward the CPU 108 is analyzed by a TLP decoder 703 in a TLP (Transaction Layer Packet) header 721 to determine a Tx type 722. Examples of the Tx type 722 include Memory Read / Write Request, Message Request, Completion, and the like. DMA_Tx 723 is generated by the migration source guest address conversion unit 704 by the migration source address information register 712 by converting the destination address of Tx from the guest physical address of the migration source guest OS 1241 to the host physical address of the physical computer 1011 by the information 742 in the migration source address information register 712. Then, the output is output as Inbound_Tx 725 of the Inbound_Tx control unit 134 by the selection circuit 702, and further transferred to the upstream 726 via the arbitration circuit 701.

  Tx other than DMA_Tx passes through the Inbound_Tx control unit 134 and is transferred to the upstream 726 via the arbitration circuit 701 as the output Inbound_Tx 725 of the Inbound_Tx control unit by the selection circuit 702.

  Next, the configuration of the migration Tx control unit 1_135 in FIG. 7 will be described. The Inbound_Tx 720 received from the downstream side of the multi-root I / O switch 102 is obtained by analyzing the TLP header 727 of the Tx by the movement target device determination unit 705 and determining whether the TLP type is DMA_Write_Tx and information 747 in the movement target device setting register 717. Based on this, it is determined 729 whether the migration Tx control is valid Tx. In the migration validity determination 729, the function selection circuit 706 determines whether the address list recording and the DMA_Write_Tx replication are valid based on the information 746 in the migration Tx control validity register 716. When one or more of address list recording and DMA_Write_Tx replication is valid, the address range determination unit 707 determines whether the destination address 728 is (1) the copied area 731 from the destination address 728 of the DMA_Write_Tx and the information 745 of the address area setting register 715. (2) It is determined whether the area 730 is being copied.

  (2) In the case of the area 730 being copied, the address list recording control unit 710 is designated by the register 8031 that designates the address list storage buffer A / B from the destination address 728 and the information 741 of the address list recording control register 711. The address list recording DMA_Tx 734 for recording the destination address 728 in the address list buffer 113 is generated. The address list recording Tx 734 is transferred to the upstream 726 via the arbitration circuit 701.

  (1) In the case of the copied area 731, the routing change unit 708 outputs the routing information 733 to the destination of the DMA_Write_Tx copied from the information 744 in the migration destination physical computer information register 714. Next, the migration source guest address conversion unit 709 duplicates the DMA_Write_Tx 732 based on the DMA_Write_Tx 732, the routing information 733, and the information 743 of the migration destination address information register 713, and transmits the routing information so as to be transferred to the migration destination physical computer 1012. 733 is changed to generate a duplicate DMA_Write_Tx 735 in which the destination address of Tx is converted from the guest physical address of the migration destination guest OS 1242 to the host physical address of the migration destination physical computer 1012. The duplicate DMA_Write_Tx 735 is transferred to the upstream 726 via the arbitration circuit 701 and reaches the migration destination guest memory 1112 of the migration destination physical computer 1012.

  Next, a control method of the composite computer according to the first embodiment will be described. First, a guest OS migration control method will be described with reference to FIG. FIG. 14 is a flowchart showing an example of a guest OS migration control method of the composite computer according to the first embodiment of the present invention. The guest OS migration is started by the migration management software 141 of the device control unit 104 when the administrator of the compound computer is instructed to migrate the guest OS via the management terminal 105 (step S1401). ). Here, the command from the administrator of the composite computer is the trigger for starting the guest OS migration. However, the operation by software, for example, the operation management software (not shown) of the composite computer is abnormal due to the heartbeat of the physical computer 1011 or the like. The occurrence of an error may be monitored and an abnormality detected may be used as a trigger for starting the guest OS migration.

  The migration management software 141 performs configuration management for guest OS migration, including the migration source and migration destination physical computers 1011 and 1012, virtual computers 1231 and 1232, and guest OSs 1241 and 1242 instructed from the management terminal 105 in step S1401. Information is acquired from the virtual machine monitors 1201 and 1202 and the device control software 140 and stored in the migration configuration information 142 (step S1402). In step S1402, the migration destination physical computer 1012 has sufficient computer resources to check whether the guest OS migration is possible, the creation of the migration destination virtual computer 1232, and the memory area of the migration destination guest OS 1242. 1112 is assigned. These processes may be performed by using known or well-known methods. The apparatus control software 140 of the apparatus control unit 104 checks the computer resources and then instructs the physical computer 1012 of the movement destination to generate the virtual machine 1232 and move it. The allocation of the memory area 1112 of the destination guest OS 1242 can be commanded.

  Next, the migration source migration control unit 1221 initializes the migration in the migration Tx control unit register 1_136 of the multi-root I / O switch 102 (step S1403). In this initial setting, the following settings are made.

  The source migration control unit 1221 sets the address list recording buffer related registers 801 to 803 and the continuous address list recording suppression designation register 804 for the address list recording control register 711 of the migration Tx control unit register 1_136. In addition, when the continuous address list recording suppression is executed, the continuous address size designation register 805 is set.

  The migration source migration control unit 1221 sets the host physical address information of the migration source guest OS 1241 in the migration source address information register 712. The migration source migration control unit 1221 sets the host physical address information of the migration destination guest OS 1242 in the migration destination address information register 713. Further, the migration source migration control unit 1221 sets the virtual I / O switch identifier G203 assigned to the migration destination physical computer 1012 in the migration destination physical computer information register 714. The migration target designation is set for the device ID of the I / O device 1031 directly assigned to the migration source guest OS 1241 in the migration target device setting register 717.

  Next, the background copy control unit 121 of the migration source physical computer 1011 divides the memory area 1111 of the migration source guest OS into memory areas to be transferred in each phase of background copy as an initial setting of the background copy (Step S1). S1404). In step S1404, the background copy interface 110 is also initialized.

  Next, the background copy control unit 121 performs the background copy phase 1 process as described above. First, the background copy control unit 121 sets the memory areas of (1) copied area 5011, (2) copying area 5021, and (3) uncopied area 5031 shown in FIG. It is set to 715 (step S1405). In the first phase 1, the address list recording valid setting 12001 and DMA_Write_Tx duplication valid setting 12002 are set in the migration Tx control valid register 716, and the migration Tx control unit 1_135 is set to be valid.

  Next, the background copy control unit 121 performs memory copy of the phase 1 (2) in-copying area 5021 (step S1406). Next, it is determined whether the memory copy of all phases of the memory area 1111 of the migration source guest OS 1241 divided as described above has been completed (step S1407). If all phases have not been completed, the processing from step S1405 to step S1407 is performed as the memory copy processing of the next phase. On the other hand, when all phases are completed, the migration source migration control unit 1221 stops scheduling of the migration source virtual machine 1231 and stops the migration source guest OS 1241 (step S1408).

  Next, the migration control unit 1221 acquires the address list in which the memory has been updated from the address list buffer 113 using DMA_Write, and re-copy the memory area at the address to the memory area 1112 of the migration destination guest OS (step S1409). . Here, the address list is acquired in step S1409. However, the migration control unit 1221 switches the address list buffers A / B 1131 and 1132 during the background copy in steps S1405 to S1407, so that the address list can be changed as needed. And an address that needs to be copied again may be managed separately.

  Since the preparation for moving the migration source guest OS 1241 to the migration destination guest OS 1242 is completed through the above steps, the migration management software 141 passes the device I / O device 1031 to be migrated to the migration destination physical computer 1012 via the device control software 140. So that the setting of the multi-root I / O switch 102 is changed.

  Also, the migration control unit 1221 of the migration source physical computer 1011 sets the address list record valid designation 12001 of the migration Tx control unit 1_135 to invalid, and sets the DMA_Write_Tx duplication valid designation 12002 to invalid (step S1410). .

  Next, the migration control unit 1222 of the migration destination virtual machine monitor 1202 directly assigns the migration target I / O device 1031 to the migration destination guest OS 1242, and starts scheduling of the migration destination virtual machine 1232. As a result, the migration destination guest OS 1242 starts operating, and the guest OS migration is completed (step S1411). The migration management software 141 notifies the administrator of the compound computer via the management terminal 105 that the guest OS migration has been completed (step S1412).

  The above is the guest OS migration control method. The time during which the guest OS is stopped is the period indicated by Ts from S1408 to S1411 in FIG. 14, and the processing time of the re-memory copy (step S1409), that is, the size of the memory area that needs to be re-copied OS stop time is determined.

  When the migration control is instructed from the apparatus control unit 104 to the migration source physical computer 1011 by the above processing, the background copy unit 121 transfers the guest memory 1111 assigned to the migration source guest OS 1241 to the migration destination guest memory 1112. A background copy is performed. Since background copy is performed while the migration source guest OS 1241 is in operation, when the guest OS 1241 requests the I / O device to perform DMA reading, the I / O device returns DMA_Write_Tx. At this time, if the destination address of DMA_Write_Tx is the copying area 5031, the migration Tx control unit 135 of the multi-root I / O switch 102 sets the destination address of the DMA_Write_Tx for the copying area 5031 to the address list of the physical computer 1011 that is the migration source. Store in the buffer 113. Then, the migration control unit 1221 of the migration source transfers the migration source virtual memory 1231 to the migration destination guest memory from the migration source guest memory 1111 for the addresses stored in the address list buffer 113 after the migration source virtual machine 1231 is executed after the background copy is completed. By copying to 1112 again, it is possible to ensure that the contents of the migration source guest memory 1111 and the migration destination guest memory 1112 are the same. Thereafter, the I / O device assigned to the migration source guest OS 1241 is switched to the migration destination physical computer 1012, and then the migration destination virtual machine 1232 and the guest OS 1242 are activated to complete the migration.

  Next, a control method of the downstream port I / O control unit 130 will be described with reference to FIG. FIG. 15 is a flowchart showing an example of processing performed by the downstream port I / O control unit 130 of the composite computer according to the first embodiment of this invention. When receiving the Inbound_Tx 720 from the downstream side (step S1501), the downstream port I / O control unit 130 determines whether the Tx type is DMA from the TLP header portion of the Tx (step S1502). When Tx is not DMA, the reception Tx 720 is transferred to the upstream side as it is (step S1503).

  On the other hand, when Tx is DMA, it is further determined whether Tx is DMA_Write (step S1504). When Tx is DMA_Write_Tx, the process of the migration control unit 1_135 is executed (step S1505), and when Tx is not DMA_Write_Tx, the process of the migration control unit 1_135 is not executed. A control method of the migration control unit 1_135 will be described later with reference to FIG.

  Next, the downstream port I / O control unit 130 converts the destination address of DMA_Tx from the guest physical address of the migration source guest OS 1241 to the host physical address of the migration source physical computer 1011, and transfers DMA_Tx to the upstream side. (Step S1506).

  Next, a control method of the migration Tx control unit 1_135 will be described with reference to FIG. FIG. 16 is a flowchart illustrating an example of a control method of the migration Tx control unit 1_135 of the composite computer according to the first embodiment of this invention.

  When the Inbound DMA_Write_Tx is received in step S1504 of FIG. 15, the migration Tx control unit 1_135 performs processing (step S1601). The migration Tx control unit 1_135 analyzes the received TLP header 727 of Tx 720 and determines whether or not the Tx is from the migration target I / O device 1031 (step S1602). If the received Tx 720 is not a migration target, the process of the migration Tx control unit 1_135 is terminated. On the other hand, if the received Tx 720 is a migration target, the destination address of the received Tx 720 is any one of the (1) copied area 5011, (2) copying area 5021, and (3) uncopied area 5031 shown in FIG. A determination is made (step S1603).

  When the destination address of the received Tx 720 is (3) the uncopied area 5031, the process of the migration Tx control unit 1_135 is terminated. If the destination address is (1) the copied area 5011, it is determined whether or not the DMA_Write_Tx replication is valid (step S1604). If the replication is valid, the routing to the destination physical computer 1012 that transfers the duplicated DMA_Write_Tx. Setting is performed (step S1605).

  Next, the migration Tx control unit 1_135 converts the destination address of the duplicate DMA_Write_Tx from the guest physical address of the migration destination guest OS 1242 to the host physical address of the migration source physical computer 1012 (step S1606), and transfers the duplicate DMA_Write_Tx735 to the upstream side. (Step S1607), and the process of the migration Tx control unit 1_135 is terminated.

  If it is determined in step S1604 that the DMA_Write_Tx copy is invalid, the process of the migration Tx control unit 1_135 is terminated. On the other hand, if it is determined in step S1603 that the destination address is (2) the copying area 5012, the migration Tx control valid register 716 is referenced to determine whether the address list recording is valid (step S1608). As a result of this determination, if the address list recording is valid, the migration Tx control unit 1_135 refers to the address list recording control register 711 to determine whether the address list recording suppression 804 for continuous addresses is valid (step S1609). ).

  When the continuous address address list recording suppression 804 is valid, the destination address value of the reception Tx_720 and the previously recorded address value match within the size of the continuous address size designation register 805 in FIG. It is determined whether or not (step S1610). If it is determined in step S1609 that the address list recording suppression 804 for continuous addresses is invalid, the process proceeds to step S1611 without performing the process in step S1610. If it is determined in step S1610 that the offsets do not match, an address list recording Tx 734 for recording the destination address of the reception Tx 720 in the address list buffer 113 is generated and transferred to the upstream side (step S1611).

  Next, the address value recorded this time is held as a recorded address value inside the circuit of the migration Tx control unit 1_135 (step S1612), and the process of the migration Tx control unit 1_135 is ended. If the address list recording is valid in the determination in step S1608 and if the offsets match in the determination in step S1610, the process of the migration Tx control unit 1_135 is terminated.

  As described above, when performing the guest OS migration, if DMA_Write_Tx addressed to the migration source guest OS 1241 is generated in the copying area 5021 by background copy, the migration Tx control unit 1_135 stores the memory area of the migration source virtual machine monitor 1201. The destination address of DMA_Write_Tx is recorded in the address list buffer 113 in 112, and after the copy of the copying area 5021 is completed, the migration source migration control unit 1221 moves the migration source guest memory 1111 of the destination address of the address list buffer 113. Is copied to the destination guest memory 1112. Accordingly, the same contents as the update of the migration source guest memory 1111 by the DMA_Write_Tx can be applied to the migration destination guest memory 1112 as the content of the copying area 5021.

  In the guest OS migration, the size of the memory copied from the migration source guest memory 1111 to the migration destination guest memory 1112 can be reduced to the size of one phase of the background copy. It is possible to greatly reduce the time required for copying the memory from the destination to the destination, and to significantly reduce the stop time of the guest OS during migration.

  As described above, according to the present invention, in the guest OS migration using the background copy, even if the throughput of the memory copy interface 160 of the migration target guest OS is small, recopying is required when the guest OS is stopped. The size of the memory area can be reduced, and the stop time of the guest OS can be shortened.

  Regardless of the memory size allocated to the guest OS, the size of the memory area that needs to be re-copied when the guest OS is stopped is one phase of background copy, and even if the memory size allocated to the guest OS is large, re-copying is performed. However, it is possible to reduce the size of the required memory area and to shorten the stop time of the guest OS.

  As described above, it is possible to reliably reduce the stop time of the guest OS during the migration of the guest OS without depending on the computer environment such as the interface type for memory copy and the allocated memory size of the guest OS. Guest OS migration can be implemented with minimal impact on servers and the like. In addition, since the restrictions on the computer environment for guest OS migration are relaxed, the use of guest OS migration can be expanded.

  The above is the control method of the composite computer according to the first embodiment.

<Second Embodiment>
Next, the configuration of the composite computer according to the second embodiment of the present invention will be described. The configuration of the composite computer in the second embodiment is the same as that of the first embodiment of the present invention shown in FIG. 1 except that the internal configuration of the multi-root I / O switch is different. is there. Therefore, only the part different from the first embodiment will be described below, and the description of the same part will be omitted.

  FIG. 17 is a block diagram of a composite computer according to the second embodiment of the present invention. As shown in FIG. 17, the multi-root I / O switch 102 includes one or more upstream ports 132 that connect the physical computers shown in FIG. 1, one or more downstream ports 133 that connect I / O devices, and a switch circuit unit 131. And a downstream port I / O control unit 130, and a transfer control unit 1703.

  The transfer control unit 1703 is connected to the switch circuit unit 131 by the I / O switch internal interface 154. The downstream port I / O control unit 130 includes the same Inbound_Tx control unit 134 as that of the first embodiment, a migration Tx control unit 2_1701 and a migration Tx control unit register 2_1702 which are different from those of the first embodiment. The transfer control unit 1703 includes a migration Tx control unit 3_1704 and a migration Tx control unit register 3_1705.

  In the second embodiment, the migration Tx control unit 2_1701 determines Tx to be a migration Tx control target, duplicates Inbound_Tx_ as it is, generates a duplicate DMA_Write_Tx 1710, and once transfers it to the migration Tx control unit 3_1704. The migration Tx control unit 3_1704 receives the duplicate DMA_Write_Tx 1710 that is a duplicate of the transferred Inbound_Tx, and the destination address of the Tx_1710 is the (1) copied area 5011 of the migration source guest OS memory area 1111 shown in FIG. In the migration Tx control unit 3_1704, DMA_Write_Tx is duplicated and DMA_Write 1712 is performed on the guest memory 1112 of the migration destination OS.

  On the other hand, in the case of (1) Copying area 5012 in the memory area 1111 of the migration source guest OS shown in FIG. 5, the migration Tx control unit 3_1704 generates an address list recording Tx 1711 and the memory area 112 of the virtual machine monitor 1201. The destination address of the reception Tx 1710 is recorded as a list in the internal address list buffer 113.

  In the first embodiment shown in FIG. 1, the number of circuits of the migration Tx control unit 1_135 is required for the number of downstream ports 133. On the other hand, in the second embodiment shown in FIG. 17, among the circuits of the migration Tx control unit 1_135, a circuit that duplicates DMA_Write_Tx corresponding to the migration Tx control unit 3_1704, a circuit that generates the address list record Tx, and those By using the migration Tx control unit register 3_1705, which is a register related to the circuit, shared by a plurality of downstream ports 133, it is sufficient to provide one circuit.

  Therefore, the second embodiment shown in FIG. 17 has the advantage that the circuit scale of the LSI mounted on the multi-root I / O switch 102 can be smaller than that of the first embodiment shown in FIG. is there.

  Next, the downstream port I / O control unit 130 and the transfer control unit 1703 of the composite computer according to the second embodiment of the present invention will be described in detail with reference to FIGS.

  18 is a block diagram showing an example of a circuit of the downstream port I / O control unit 130 of the composite computer according to the second embodiment, and FIG. 19 is a transfer control unit 3_1704 of the composite computer according to the second embodiment. It is a block diagram showing an example of the circuit.

  In the second embodiment, as shown in FIG. 18, the downstream port I / O control unit 130 includes an Inbound_Tx control unit 134, a migration Tx control unit 2_1701, a migration Tx control unit register 2_1702, an Inbound_Tx control unit 134, It comprises an arbitration circuit 701 that arbitrates between Tx from the migration Tx control unit 2_1701 and transfers it upstream.

  The Inbound_Tx control unit 134 is the same as that of the first embodiment of the present invention. The migration Tx control unit register 2_1702 includes a migration source address information register 712, a migration Tx control enable register 716, and a migration target device setting register 717. The configuration of these registers is the same as that of the first embodiment of the present invention.

  Next, a circuit of the migration Tx control unit 2_1701 will be described. Inbound_Tx 720 received from the downstream side is subjected to migration Tx control based on whether or not TLP header 727 of Tx is analyzed by movement target device determination unit 705 and the type of TLP is DMA_Write_Tx, and information 747 of movement target device register 717. A determination 729 is made whether the Tx is valid. In the migration validity determination 729, the function selection circuit 706 determines whether the address list recording and the DMA_Write_Tx replication are valid based on the information 746 in the migration Tx control validity register 716. When one or more of address list recording and DMA_Write_Tx duplication are valid, the duplicate DMA_Write_Tx transfer unit 1801 duplicates DMA_Write_Tx 732 and generates a duplicate DMA_Write_Tx 1802 with the destination being the transfer control unit 1703. The duplicate DMA_Write_Tx 1802 is transferred to the upstream 726 via the arbitration circuit 701.

  In the second embodiment, the transfer control unit 1703 arbitrates between the output Tx from the migration Tx control unit register 3_1705, the migration Tx control unit 3_1704, and the migration Tx control unit 3_1704 as shown in FIG. It consists of an arbitration circuit 701 for transferring.

  The migration Tx control unit register 3_1705 includes an address list recording control register 711, a migration destination address information register 713, a migration destination physical computer information register 714, and an address area setting register 715. The configuration of these registers is the same as that of the first embodiment of the present invention.

  Next, a circuit of the migration Tx control unit 3_1704 will be described. The migration Tx control unit 3_1704 receives the duplicate DMA_Write_Tx 1802 transferred from the downstream port I / O control unit 130 of each downstream port 133 of the multi-root I / O switch 102 from the input side.

  The address range determination unit 707 determines whether the destination address 1911 is (1) a copied area 1913 or (2) an area 1912 being copied from the destination address 1911 of the duplicate DMA_Write_Tx 1802 and the information 745 of the address area setting register 715. (2) In the case of the area 1912 being copied, the address list recording control unit 710 is designated by the register 8031 that designates the address list storage buffer A / B from the destination address 1911 and the information 741 of the address list recording control register 711. The address list recording DMA_Tx 1915 for recording the destination address 1911 in the address list buffer 113 of the source physical computer 1011 is generated. The address list recording Tx — 1915 is transferred as an output 1917 via the arbitration circuit 1901.

  (1) In the case of the copied area 1913, the routing change unit 708 outputs the routing information 1914 to the destination of the DMA_Write_Tx to be copied from the information 744 in the migration destination physical computer information register 714. Next, the migration source guest address conversion unit 709 duplicates the DMA_Write_Tx 1802 based on the DMA_Write_Tx 1802, the routing information 1914, and the information 743 of the migration destination address information register 713, and transmits the routing information so as to be transferred to the migration destination physical computer 1012. In 1914, a duplicate DMA_Write_Tx 1916 in which the destination address of Tx is converted from the guest physical address of the migration destination guest OS 1242 to the host physical address of the migration destination physical computer 1012 is generated. The duplicate DMA_Write_Tx 1916 is transferred to the destination guest memory 1112 as the output 1917 via the arbitration circuit 1901.

  Also in the second embodiment as described above, the same operations and effects as those in the first embodiment can be obtained.

<Third Embodiment>
Next, the configuration of the composite computer according to the third embodiment of the present invention will be described. The configuration of the composite computer in the third embodiment is the same as the configuration of the composite computer of the first embodiment of the present invention shown in FIG. 1 and the downstream port I / O control unit 130 of the multi-root I / O switch 102. The internal configuration is the same except that the internal configuration of the chip set 107 on the physical computers 1011 and 1012 is different. Therefore, only the part different from the first embodiment will be described below, and the description of the same part will be omitted.

  FIG. 20 is a block diagram of a composite computer according to the third embodiment of the present invention. As shown in FIG. 20, the downstream port I / O control unit 130 is composed of a migration Tx control unit 4_2002 and a migration Tx control unit register 4_2003 different from those of the first embodiment, and the downstream port I / O control unit 130 of the first embodiment shown in FIG. The Inbound_Tx control unit 134 does not have. In the second embodiment, Tx from the I / O device 1031 passes through the downstream port I / O control unit 130, and Inbound_Tx 2010 is transferred to the guest address conversion unit 2001 of the chipset 107 of the migration source physical computer 1011. Is done.

  When the received Tx 2010 is DMA_Tx, the guest address conversion unit 2001 converts the destination address from the guest physical address of the migration source guest OS 1241 to the host physical address of the migration source physical computer 1011 in 2011, and stores it in the memory area 1111 of the migration source guest OS. Transferred. The migration Tx control unit 4_2002 generates the address list record Tx164 and the duplicate DMA_Write_Tx2012 as in the first embodiment shown in FIG. 1. In the third embodiment, the migration Tx control unit 4_2002 uses the duplicate DMA_Write_Tx2012 destination. The address is not transferred and transferred to the guest address conversion unit 20012 of the chip set 107 of the destination physical computer 1012. The guest address conversion unit 20012 of the migration destination physical computer 1012 converts the destination address of the received duplicate DMA_Write_Tx2012 from the guest physical address of the migration destination guest OS 1242 to the host physical address of the migration destination physical computer 1012 2013, and the memory of the migration destination guest OS Transfer to area 1112.

  Note that the configuration disclosed in Japanese Patent Application Laid-Open No. 2004-220218 can be used for the guest address conversion units 20010 and 20012 on this type of chipset.

  Next, the downstream port I / O control unit 130 of the composite computer according to the third embodiment of the present invention will be described in detail with reference to FIG. FIG. 21 is a block diagram showing an example of a circuit of the downstream port I / O control unit 130 of the composite computer according to the third embodiment. In the third embodiment, as shown in FIG. 21, the downstream port I / O control unit 130 arbitrates between Tx from the migration Tx control unit 4_2002, the migration Tx control unit register 4_2003, and the migration Tx control unit 4_2002, The arbitration circuit 701 is configured to transfer to the upstream, and the Inbound_Tx control unit 134 of the first embodiment illustrated in FIG. 1 is not provided.

  The migration Tx control unit register 4_2003 includes an address list recording control register 711, a migration destination physical computer information register 714, an address area setting register 715, a migration Tx control valid register 716, and a migration target device setting register 717. . The configuration of these registers is the same as that of the first embodiment of the present invention.

  In the downstream port I / O control unit 130 of the third embodiment, the Inbound_Tx 720 received from the downstream is transferred to the upstream 726 via the arbitration circuit 701 without processing. The migration Tx control unit 4_2002 has a configuration in which the migration destination guest address conversion unit 709 of the migration Tx control unit 1_135 of the first embodiment shown in FIG. 7 is replaced with a duplicate DMA_Write_Tx transfer unit 2101. Are the same. That is, in the third embodiment shown in FIG. 21, when duplicating DMA_Write_Tx, DMA_Write_Tx 2110 in which the destination address of Tx is not changed is generated and transferred to upstream 726 via arbitration circuit 701. On the other hand, the generation of the address list record Tx is the same circuit as that of the first embodiment shown in FIG.

  Also in the third embodiment as described above, by performing conversion of the destination address of DMA_Write_Tx in the chip set 107 of the migration source and migration destination physical computers 1011 and 1021, respectively, the same operation as in the first embodiment, An effect can be obtained.

  In the above embodiments, the CPU 108 and the chip set 107 are separated from each other. However, the CPU 108 and the chip set 107 may be mounted on the same LSI.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

  As described above, the present invention can be applied to a computer virtualization technology, particularly to a computer system that performs guest OS migration for moving a guest OS on a virtual computer between physical computers.

1 is a block diagram of a composite computer according to a first embodiment of this invention. FIG. FIG. 8 is an explanatory diagram illustrating an example of migration configuration information of a composite computer according to the first embodiment. FIG. 9 is an explanatory diagram illustrating an example of host physical address information of a migration source of a migration target guest OS of the composite computer according to the first embodiment. FIG. 3 is an explanatory diagram illustrating an example of host physical address information of a migration destination of a migration target guest OS of the composite computer according to the first embodiment. FIG. 3 is an explanatory diagram illustrating an example of I / O device information to be migrated by the composite computer according to the first embodiment. FIG. 8 is an explanatory diagram illustrating an example of a memory space of a migration source guest OS during background copy of the composite computer according to the first embodiment. FIG. 10 is an explanatory diagram illustrating an example of an address list buffer generated in the background copy of the composite computer according to the first embodiment. 1 is a block diagram illustrating an example of a circuit of a downstream port I / O control unit of a composite computer according to a first embodiment. FIG. 3 is an explanatory diagram illustrating an example of an address list recording control register of a downstream port I / O control unit of the composite computer according to the first embodiment. FIG. 10 is an explanatory diagram illustrating an example of a source address information register of a downstream port I / O control unit of the composite computer according to the first embodiment. FIG. 9 is an explanatory diagram illustrating an example of a migration destination address information register of the downstream port I / O control unit of the composite computer according to the first embodiment. FIG. 10 is an explanatory diagram illustrating an example of an address area setting register of a downstream port I / O control unit of the composite computer according to the first embodiment. FIG. 9 is an explanatory diagram illustrating an example of a migration Tx control valid register of a downstream port I / O control unit of the composite computer according to the first embodiment. FIG. 3 is an explanatory diagram illustrating an example of a migration target device setting register of a downstream port I / O control unit of the composite computer according to the first embodiment. 5 is a flowchart illustrating an example of control of guest OS migration of the composite computer according to the first embodiment of this invention. 5 is a flowchart illustrating an example of processing performed by the downstream port I / O control unit of the composite computer according to the first embodiment of this invention. 5 is a flowchart illustrating an example of control performed by the migration Tx control unit 1 of the composite computer according to the first embodiment of this invention. FIG. 10 is a block diagram of a composite computer according to a second embodiment of this invention. FIG. 9 is a block diagram illustrating an example of a circuit of a downstream port I / O control unit of the composite computer according to the second embodiment. FIG. 9 is a block diagram illustrating an example of a circuit of a transfer control unit of a composite computer according to a second embodiment. FIG. 10 is a block diagram of a complex computer according to a third embodiment of this invention. FIG. 10 is a block diagram illustrating an example of a circuit of a downstream port I / O control unit of a composite computer according to a third embodiment.

Explanation of symbols

102 Multi-Route I / O Switch 1031, 1032 I / O Device 104 Device Control Unit 105 Management Terminal 107 Chip Set 108 CPU
113 Address List Buffer 121 Background Copy Control Unit 130 Downstream Port I / O Control Unit 131 Switch Circuit Unit 134 Inbound_Tx Control Unit 135 Migration Tx Control Unit 1
136 Migration Tx control unit register 1
140 Device control software 141 Migration management software 142 Migration configuration information 1011 and 1012 Migration source physical computers 1091 and 1092 Memory 1111 Migration source guest OS memory 1112 Migration destination guest OS memory 1201 and 1202 Virtual machine monitor 1221 and 1222 Migration control unit 1231 Migration source Virtual machine 1232 Destination virtual machine 1241 Migration source guest OS
1242 Destination Guest OS

Claims (12)

A first physical computer comprising a first processor and a first memory;
A first virtualization unit that virtualizes computer resources of the first physical computer and executes the first virtual computer;
A second physical computer comprising a second processor and a second memory;
A second virtualization unit for virtualizing computer resources of the second physical computer and executing the second virtual computer;
An I / O switch for connecting the first physical computer and the second physical computer to an I / O device;
The first virtualization unit is a complex computer including a migration control unit that moves a first OS executed on the first virtual computer to the second virtual computer,
The first virtualization unit includes:
When the migration control unit moves the first OS to the second virtual machine, the first virtualization unit assigned to the first OS during the operation of the first OS. A background copy unit that copies an area of one memory to a second memory of the second physical computer;
The background copy section
The state of the first memory includes a copied area where copying to the second memory has been completed, a copying area where copying to the second memory is being performed, and copying to the second memory Is classified into one of the uncopied unfinished areas, and the classified memory state and the address of the first memory are notified to the I / O switch as classification information,
The I / O device performs a DMA access with a first memory allocated to a first OS of the first physical computer allocated in advance,
The I / O switch is
When a DMA write transaction is received from the I / O device assigned to the first OS, the destination address of the DMA write transaction is compared with the address of the classification information in the first memory, and When the address of the DMA write transaction is the copied area, the DMA write transaction is DMA-written to both the first memory and the second memory, and the address of the DMA write transaction is set to the area being copied or not. In the case of a copy area, the composite computer has an I / O control unit for performing DMA writing only in the first memory. In the compound computer according to claim 1,
The I / O control unit
When a DMA write transaction is received from the I / O device assigned to the first OS, the destination address of the DMA write transaction is compared with the address of the classification information in the first memory, and the DMA When the address of the write transaction is the copy-in-progress area, perform DMA write to the first memory, store the address information of the DMA write transaction in the address storage area set in advance in the first memory,
The first virtualization unit includes:
After completion of copying of the copy-in-progress area, the address information stored in the address storage area is acquired, and copying is performed from the first memory indicated by the address information to the second memory. Type calculator. In the compound computer according to claim 2,
The first virtualization unit includes:
As the progress status of the background copy unit, the first memory is classified into one of the copied area, the copying area, and the uncopied area, and the copying of the copying area is completed In this case, the copy area is reclassified as a copied area, a part or all of the uncopied area is newly reclassified as a copy area, the I / O switch is notified, and the copy A composite computer that performs copying from a first memory reclassified from a middle area to a copied area to a second memory. In the compound computer according to claim 1,
The I / O control unit
When a DMA write transaction is received from an I / O device assigned to the first OS, the destination address of the DMA write transaction is changed from the guest physical address of the first OS to the host physical address of the first physical computer. When writing to the first memory and DMA writing to the second memory, the destination address of the DMA write transaction is changed from the guest physical address of the first OS to the second physical computer. A compound computer that converts to a host physical address and writes to the second memory. In the compound computer according to claim 1,
The first physical computer has a first control hub that connects the I / O switch to a first processor and a first memory;
The first control hub includes a first address conversion unit that converts a guest physical address of a first OS of the first virtual machine into a corresponding host physical address of the physical machine;
The second physical computer has a second control hub that connects the I / O switch, a second processor, and a second memory;
The second control hub has a second address conversion unit that converts a guest physical address of the first OS to be moved to the second virtual machine to a host physical address of the second physical machine,
When the first control hub receives a DMA write from the I / O switch, the first address conversion unit of the first physical computer sets the DMA write address to the guest physical of the first OS. Convert the address to the host physical address of the first physical computer and write to the first memory,
When the second control hub accepts the DMA write from the I / O switch, the second address conversion unit of the second physical computer sets the DMA write address to the guest of the first OS. A composite computer, wherein a physical address is converted into a host physical address of a second physical computer and written to a second memory. In the compound computer according to claim 1,
A management computer that manages the movement of the first OS operating on the first physical computer of the composite computer to the second physical computer;
The management computer, as the migration configuration information to the first virtualization unit at the start of migration of the first OS, identifiers of the migration source and migration destination physical computers, identifiers of the migration source and migration destination virtual computers, A composite computer characterized by notifying transfer physical host address information from a transfer source and transfer destination guest physical address and an identifier of an I / O device assigned to the first OS. A first physical computer having a first processor and a first memory; a first virtualization unit for virtualizing computer resources of the first physical computer and executing the first virtual computer; A second physical computer having a second processor and a second memory, a second virtualization unit for virtualizing computer resources of the second physical computer and executing the second virtual computer, and the first A first computer in which the first virtualization unit is executed in the first virtual computer in a composite computer comprising a physical computer and an I / O switch for connecting a second physical computer and an I / O device. A method of controlling a composite computer that moves the OS of the computer to the second virtual computer,
When the first virtualization unit moves the first OS to the second virtual machine, the first virtualization unit moves the first OS to the first OS during the operation of the first OS. Copying the allocated area of the first memory to the second memory of the second physical computer;
The first virtualization unit indicates a state of the first memory, a copied area where copying to the second memory is completed, and a copying area where copying to the second memory is being executed. Categorizing into any uncopied area that has not been copied to the second memory and notifying the I / O switch of the classified memory state and the address of the first memory as classification information When,
When the I / O switch receives a DMA write transaction from the I / O device assigned to the first OS, the destination address of the DMA write transaction and the address of the classification information in the first memory And a step of comparing
The I / O switch performs a DMA write of the DMA write transaction to both the first memory and the second memory when the address of the DMA write transaction is the copied area as a result of the comparison. When,
The I / O switch, as a result of the comparison, performing a DMA write only to the first memory if the address of the DMA write transaction is the area being copied or an uncopied area;
A control method for a composite computer, comprising: In the control method of the compound computer according to claim 7,
The I / O switch performs a DMA write only to the first memory when the address of the DMA write transaction is the area being copied or an uncopied area as a result of the comparison,
When the address of the DMA write transaction is in the copy-in-progress area, the DMA write is performed to the first memory, and the address information of the DMA write transaction is stored in an address storage area set in advance in the first memory. Steps,
The first virtualization unit obtains the address information stored in the address storage area after the copying of the area being copied is completed, and transfers the address information from the first memory to the second memory. A step of copying,
A control method for a composite computer, comprising: In the control method of the compound computer according to claim 8,
The first virtualization unit obtains the address information stored in the address storage area after the copying of the area being copied is completed, and transfers the address information from the first memory to the second memory. The steps to copy are
When copying of the copying area is completed, the copying area is reclassified as a copied area, and a part or all of the uncopied area is newly reclassified as a copying area, A control method for a complex computer, which notifies an I / O switch and performs copying from the first memory reclassified to the copied area from the copying area to the second memory. In the control method of the compound computer according to claim 7,
When the I / O switch receives a DMA write transaction from an I / O device assigned to the first OS, the destination address of the DMA write transaction is first determined from the guest physical address of the first OS. When the data is converted to the host physical address of the physical computer and then written to the first memory, and the DMA write is performed to the second memory, the destination address of the DMA write transaction is the guest physical address of the first OS A control method for a composite computer, wherein the second physical computer is converted to a host physical address and written to the second memory. In the control method of the compound computer according to claim 7,
The first physical computer has a first control hub that connects the I / O switch to a first processor and a first memory;
The first control hub includes a first address conversion unit that converts a guest physical address of a first OS of the first virtual machine into a corresponding host physical address of the physical machine;
The second physical computer has a second control hub that connects the I / O switch, a second processor, and a second memory;
The second control hub has a second address conversion unit that converts a guest physical address of the first OS to be moved to the second virtual machine to a host physical address of the second physical machine,
When the first control hub receives a DMA write from the I / O switch, the first address conversion unit of the first physical computer sets the DMA write address to the guest physical of the first OS. Converting the address to the host physical address of the first physical computer and writing to the first memory;
When the second control hub receives a DMA write from the I / O switch, the second address conversion unit of the second physical computer sets the DMA write address to the guest of the first OS. Converting the physical address to the host physical address of the second physical computer and writing to the second memory;
A control method for a composite computer, further comprising: In the control method of the compound computer according to claim 7,
A management computer for managing the movement of the first OS operating on the first physical computer of the composite computer to the second physical computer;
When the management computer starts migration of the first OS to the first virtualization unit as migration configuration information, the identifiers of the migration source and migration destination physical computers, the migration source and migration destination virtual computers, Control of the compound computer further comprising a step of notifying the translation information of the migration source and the migration destination guest physical address to the host physical address and the identifier of the I / O device allocated to the first OS Method.

Images