JP2017215872A - Backup control server, and application data backup method for service control server - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a backup control server configured to guarantee RPO of backup data of a service control server in a large-scaled network, and an application data backup method for the service control server.SOLUTION: A backup control server 100 comprises: a backup target server management part 110 for managing a service control server of a backup target; a backup data management part 120 for storing or deleting back-up collected application data; a backup control server execution management part 130 which uses a backup execution management table 160 to execute the backup collection in accordance with a predetermined schedule and monitor an execution status; and a backup congestion control part 140 which searches for and switches to an alternative virtualized host server or transfer route from a load situation within a network, and manages an average switching time for each transfer route switching pattern and a free band for each transfer route.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a backup control server and an application data backup method for a service control server.

In recent years, in order to dynamically use various NW services in a wide area network (hereinafter referred to as NW as appropriate), virtualization of NW services by NFV (Network Function Virtualization) has been studied. In NFV, it is known that there is also a service that needs to connect user services with an L2NW (layer 2 network) for only Ethernet (registered trademark) transfer.
On the other hand, a large-scale wide-area NW is often constructed with an L3NW (layer 3 network) using IP routing from the viewpoint of scaleability and fault tolerance. Therefore, in NFV, NVO3 (Network Virtualization Over L3) that realizes L2 transfer by constructing a virtual L2 tunnel on the L3NW has been studied.

  For example, VXLAN extends L2 communication (broadcast domain) via L3 by tunneling L2 communication with L3 by utilizing a 24-bit VXLAN ID called “VXLAN Network Identifier (VNI)”. In VXLAN, a VTEP (Virtual Tunnel End Point) is installed for each virtual switch or physical server of the hypervisor. VTEP is implemented at the connection point of physical L3NW and L2NW. VTEP manages the correspondence between the MAC address of each virtual machine and the VNI of that virtual machine in a table.

  It is important for network operators to effectively use the resources of the entire network to maintain transfer quality and reduce network costs. Network virtualization technology has been proposed as a technology for flexibly responding to sudden and irregular changes in the network due to future service diversification and realizing effective use of network resources.

  In addition, for example, a technique for realizing a virtual machine VM (Virtual Machine) serving as a plurality of virtual computers on one physical server with software called a hypervisor (for example, a VM management control program) is known. . Each virtual machine can operate an operating system independently. A virtual machine server, which is a physical server on which a plurality of virtual machines operate, is a computer such as a workstation or a personal computer. In an environment in which a plurality of virtual machine servers are connected via a network, virtual machines realized on the virtual machine server can also communicate via the network.

The physical server (physical machine) includes, for example, at least one virtual machine, a virtual switch, and a hypervisor. In addition, the failure detection monitoring function transmits a control message for confirming continuity between the physical machine and a network device such as a switch or a router connected to the physical machine. The control message is transmitted via a virtual port of a connected virtual switch, a virtual switch, a NIC (Network Interface Card) of a physical machine, an L2 (Layer 2) switch, an L3 switch, a physical router, or the like. Moreover, in the transmission of the control message, ping (Packet Internet Groper) or the like is executed as one aspect.
As an advantage of virtual machine creation, portability can be secured and flexible operation can be performed by using a migration technology that moves virtual machines between different physical servers.

In order to realize the migration, it is necessary to switch the network information (VLAN (Virtual Local Area Network) information, routing information) for accessing the virtual machine following the switching of the virtual machine.
IEEE 802.1Qbg exists as a prior art for automating network reconfiguration (see Non-Patent Document 1). IEEE 802.1Qbg is intended to realize external offload of virtual switches and to automatically set port protocols in conjunction with live migration. IEEE802.1Qbg can transmit the information of the virtual switch to the physical switch, and can move the setting to another physical switch when the virtual machine is switched.

  Non-Patent Documents 1 to 3 describe a technique for guaranteeing RPO (Recovery Point Objective) of backup data within the same data center (DC). RPO is the target of the age of backup data to be restored when data is damaged due to a system failure or the like. The RPO is an index that defines how new the backup data must be at least, and the backup method and frequency are determined based on the index.

FIG. 15 is a diagram showing a data backup configuration in the data center (DC).
As shown in FIG. 15, a data center (hereinafter also referred to as DC) 1 includes a virtualization host server 10 (virtualization host server # 1) composed of physical servers and a virtualization host server 20 (virtualization host). Server # 2) and storage 30.
The virtualization host server 10 (virtualization host server # 1) has service control servers 11 to 13 (# 1 to #M) as virtualization servers. The virtualization host server 20 (virtualization host server # 2) has backup control servers 21 to 23 (# 1 to #N) as virtualization servers.
The DC1 includes service control servers 11 to 13 (# 1 to #M), backup control servers 21 to 23 (# 1 to #N), and a storage 30, which are connected via a LAN (not shown). By connecting to each other and accommodating a plurality of them, one communication base is formed in the communication network.
The service control servers 11 to 13 (# 1 to #M) are virtualized servers that are connected to NW devices on the user side via a LAN and are responsible for controlling them in order to provide various NW services. .

Conventionally, migration is mainly performed within the same DC, and the data transfer time does not vary greatly. In recent years, since DC has been widely distributed, data transfer is performed via a network, and therefore, there is a possibility that the transfer time varies greatly depending on the load condition of the network.
The backup control servers 21 to 23 (# 1 to #N) periodically collect backup data of application data for the service control servers 11 to 13 (# 1 to #M) so as to guarantee RPO. It is a virtualization server that performs adaptive control. Specifically, the backup control servers 21 to 23 (# 1 to #N) dynamically allocate as many virtual servers (backup control servers) as necessary according to the load when the number of backup target servers is concentrated at a certain point in time. Newly generate and respond. Further, the backup control servers 21 to 23 (# 1 to #N) perform leveling by rearranging the schedules so that the numbers are as uniform as possible when there are mountains and valleys when the number of servers scheduled to be collected in the backup schedule is present. By using this backup load leveling, it is possible to reduce the impact on services that are required to operate in real time.

The virtualization host servers 10 and 20 (# 1, # 2) are physical servers that emulate virtual hardware for executing different OSs of a plurality of virtualization servers.
The storage 30 stores backup data of the service control server.

So far, the guarantee of RPO of backup data in the same DC has been described.
Even in a wide area where there are a plurality of DC bases or where physical servers are distributed, the virtualization server (virtualized service control server) does not interrupt the service by live migration to other DC bases. Can move freely.

FIG. 16 is a diagram for explaining a backup service execution method of a virtualization server that freely moves between data centers.
As shown in FIG. 16, the virtualized service control server 11 can freely move between DC1 to DC4 (# 1 to # 4) distributed in a large-scale network. As indicated by reference symbols a and b in FIG. 16, the backup control server 21 periodically collects backup data of application data from the service control server 11.

MAGONIA API Specification (Distributed Processing Platform) Overview, [online], [Search May 25, 2016], Internet <URL: http: // www. Ntt.co.jp/news2015/1502/pdf/150219a_1 .pdf> MAGONIA API Specification (Distributed Processing Platform) Specification, [online], [Search May 25, 2016], Internet <URL: http: // www. Ntt.co.jp/news2015/1502/pdf/ 150219a_2.pdf> Detailed explanation VMware vSphere 4.1 ITpro Easy scale-out and scale-up, [online], [Search May 25, 2016], Internet <URL: http://itpro.nikkeibp.co.jp/article/COLUMN/20110726/ 362843 /〉

  However, there is the following problem when backing up a virtual server that freely moves between data centers due to the advancement of DC wide area distribution. In other words, if a communication failure or congestion on the route for transferring backup data occurs and the backup time becomes remarkably long, it becomes impossible to guarantee the RPO to be recovered when a failure occurs in the target server. For example, as indicated by the symbol b in FIG. 16, the backup data transfer time may vary greatly depending on the network load situation. In this case, the backup control server 21 may not be able to guarantee RPO. If the RPO cannot be guaranteed, the reliability of the NW is greatly reduced.

  The present invention has been made in view of such a background, and the present invention provides a backup control server that guarantees RPO of backup data of a service control server in a large-scale network, and an application data backup method of the service control server. This is the issue.

  In order to solve the above-mentioned problem, the invention according to claim 1 is a backup control server that backs up application data of a virtualized service control server that moves between data centers distributed in a network. A backup target server management unit that manages the management list of the service control server, a backup data management unit that stores or deletes application data collected for backup, and backup control that executes backup collection according to a predetermined schedule and monitors the execution status A server execution management unit, and a backup congestion control unit that searches for an alternative physical server and transfer route based on a load situation of resources in the network and controls switching of the alternative physical server and transfer route. And a backup control server, characterized in that.

  According to a fourth aspect of the present invention, there is provided an application data backup method for a backup control server that backs up application data of a virtualized service control server that moves between data centers distributed in a network. A backup target server management process for managing the management list of the service control server, a backup data management process for storing or deleting application data collected for backup, and a backup control server for executing the backup collection according to a predetermined schedule and monitoring the execution status Based on the execution management process and the load status of the resources in the network, search for an alternative physical server and transfer route, and control the switching of the alternative physical server and transfer route. And up congestion control process, and the application data backup method for service control server characterized in that it comprises a.

  In this way, when a virtualized service control server moves freely to any data center without interrupting the service due to live migration in a large-scale network composed of a plurality of data centers distributed over a wide area. The RPO can be guaranteed even if the time for backing up the application data varies greatly depending on the load status of the physical server (virtualization host server) that executes the transfer network and each virtualization service control server. By guaranteeing the RPO, it is possible to guarantee the communication quality at the time of trouble to the user.

  In the invention according to claim 2, the backup control server execution management unit sets the latest backup completion time and backup start time, backup execution time, backup data collection status, RPO time, untransferred data for each service control server. The backup control server is characterized by managing the size or the backup execution schedule.

  By doing so, the backup control server execution management unit refers to the backup execution management table storing these management information, and performs any of the following: live migration execution, transfer route switching, or emergency backup to virtual storage. Can be determined.

  According to a third aspect of the present invention, the backup congestion control unit searches for and switches an alternative virtualization host server and transfer route according to a load situation in the network, and performs average switching for each transfer route switching pattern. The backup control server is characterized by managing free bandwidth for each time or transfer route.

  By doing so, the backup congestion control unit can execute the optimal transfer route switching or the emergency backup to the virtual storage based on the average switching time for each transfer route switching pattern and the free bandwidth for each transfer route. Can do.

  The invention according to claim 5 is a service in which a backup control server that backs up application data of a virtualized service control server that moves between data centers distributed in a network is provided in each data center. An application data backup method for a control server, comprising: a backup target server management process for managing a management list of the service control server to be backed up; a backup data management process for storing or deleting backup collected application data; and backup collection. Based on the backup control server execution management step that executes according to a predetermined schedule and monitors the execution status, and searches for an alternative physical server and transfer route based on the load status of the resources in the network, Backup congestion control step for controlling switching of the physical server and the transfer route, and when the service control server is moved to the physical server of another data center via the network by live migration, Deleting the service control server moved from the management list of the backup control server of the data center of the migration source, and the service control server moved from the management list of the backup control server of the data center of the migration destination And the backup data held by the backup control server at the migration source data center when the first backup collection is completed for the service control server moved at the destination data center. And erasing And application data backup method of the service control server, wherein the door.

  In this way, when a virtualized service control server moves freely to any data center without interrupting the service due to live migration in a large-scale network composed of a plurality of data centers distributed over a wide area. The RPO can be guaranteed even if the time for backing up the application data varies greatly depending on the load status of the physical server (virtualization host server) that executes the transfer network and each virtualization service control server. In addition, since the physical server is automatically replaced and the transfer route is switched on the network side without intervention of the maintenance person, the burden on the maintenance person is reduced. In addition, by smoothing the local traffic load in the network, stable operation of the network can be promoted.

  The invention according to claim 6 is the backup execution when the service control server is moved to the physical server of another data center via the network by live migration, or when the backup collection state monitoring timer expires. A step of determining a transfer delay factor from the situation, and another physical server in a low load state in the same data center when the transfer delay factor is in a high load state of the physical server in the data center of the movement destination And a step of performing live migration again on the service control server to another physical server.

  In this way, the local traffic load in the DC can be smoothed by performing live migration again to another physical server in a low load state in the same DC, and stable operation of the network. Can be promoted.

  According to the seventh aspect of the present invention, when the service control server is moved to a physical server in another data center via the network by live migration, backup execution is performed when the backup collection state monitoring timer expires. A step of determining a transfer delay factor from the situation, and another transfer route in which the transfer delay factor is in a low load state when the transfer route connecting the data center of the movement source and the data center of the movement destination is in a high load state And a step of switching to another transfer route. The application data backup method of the service control server is characterized by comprising:

  By doing this, when the transfer route connecting the destination DCs is in a high load state, the local traffic load in the network is smoothed by switching to another transfer route in a low load state. It is possible to promote stable operation of the network.

  In the invention according to claim 8, when the service control server is moved to a physical server in another data center via the network by live migration, backup execution is performed when a backup collection state monitoring timer expires. A step of determining a transfer delay factor from the situation, and the transfer route connecting the physical server of the migration destination data center or the migration source data center and the migration destination data center as a transfer delay factor is in a high load state Temporarily backing application data to storage belonging to the physical server of the destination data server or the physical server of the destination data center if the transfer route is not found, and temporarily stored The backup control server backs up the backup data. And a step of recovering when the opportunity arises concurrent collection number of servers is small up execution schedule, and application data backup method of the service control server, which comprises a.

  In this way, even if a problem occurs in the virtualized service control server when the backup control server is outside the DC, the state can be returned to the RPO, so that multiple data centers are distributed over a wide area. The reliability of a large-scale network to be arranged can be improved. Further, the backup control server can collect the backup data of the virtual storage in the DC and save it in its own storage, and can improve the reliability at the time of emergency backup.

  ADVANTAGE OF THE INVENTION According to this invention, the backup control server which guarantees RPO of the backup data of the service control server in a large-scale network, and the application data backup method of a service control server can be provided.

It is a figure which shows the structure of wide area large-scale network and DC provided with the backup control server which concerns on embodiment of this invention. It is a functional block diagram of a backup control server according to an embodiment of the present invention. It is a figure which shows an example of the backup execution management table with which the backup control server execution management part of the backup control server which concerns on embodiment of this invention is provided. It is a figure which shows an example of the backup object server management list with which the backup object server management part of the backup control server which concerns on embodiment of this invention is provided. It is explanatory drawing when backup data collection can be performed without going through the transfer network of the backup control server which concerns on embodiment of this invention (<operation pattern 1>). It is a figure which shows the control sequence of FIG. It is explanatory drawing when it is necessary to perform backup data collection via the transfer network of the backup control server which concerns on embodiment of this invention (<operation pattern 2>). It is a figure which shows the control sequence of FIG. It is explanatory drawing when it is necessary to perform backup data collection via the transfer network of the backup control server which concerns on embodiment of this invention (<operation pattern 3>). It is a figure which shows the control sequence of FIG. It is a figure which shows the control sequence in case backup data collection needs to be performed via the transfer network of the backup control server which concerns on embodiment of this invention (<operation pattern 4>). It is explanatory drawing in case backup data collection needs to be performed via the transfer network of the backup control server which concerns on embodiment of this invention. It is explanatory drawing of operation | movement when the backup control server which concerns on embodiment of this invention cannot find the virtualization host server of a switching destination, and a transfer route. It is a figure which shows the control sequence of emergency backup collection when the virtualization host server of the switching destination of the backup control server which concerns on embodiment of this invention, or a transfer route does not exist. It is a figure which shows the backup structure of the data in DC of the network system which concerns on a prior art. It is a figure explaining the backup service execution method of the virtualization server which moves freely between the data centers of the network system which concerns on a prior art.

A network system and the like in a mode for carrying out the present invention (hereinafter referred to as “the present embodiment”) will be described below with reference to the drawings.
This is an example applied to the application data backup method of the service control server that enables the return to the previous state within the RPO time when a problem occurs in the service control server.
FIG. 1 is a diagram showing a configuration of a wide area large-scale network and a DC including a backup control server according to an embodiment of the present invention.
As shown in FIG. 1, in the network system, a backup control server 100 is connected to DCs 200 and 300 (DC # 1 and DC # 2) via a transfer network 5.
The network system constructs an L2 tunnel (for example, VXLAN) for wide area distributed DC on the transfer network 5, and a backup control server 100 connected to the transfer network 5 and a DC 300 ( Packets are transmitted to and received from the virtualization host server 341 (virtualization host server # 1) in DC # 2). In the example of FIG. 1, the backup control server 100 is connected to the virtualization host server 341 (virtualization host server # 1) in the DC300 (DC # 2).

  DC200, 300 (DC # 1, DC # 2) are generic names of facilities / buildings where IT (Information Technology) devices such as servers and network devices are installed and operated, and any names may be used. DC200 (DC # 1) and DC300 (DC # 2) can exist across regions and countries. For example, DC200 (DC # 1) is based in Tokyo and DC300 (DC # 2) is based in Osaka. In the DCs 200 and 300 (DC # 1, DC # 2), a plurality (a plurality of) processing servers are accommodated, and each processing server is connected by a DC network (not shown).

  The DC 200 (DC # 1) includes a virtualized service control server 211 (service control server # 1) and a service control server 212 (service control server # 2). The service control server 211 (service control server # 1) has a transfer function for service A, for example, and the service control server 212 (service control server # 2) has a transfer function for service B, for example. Note that the service control servers # 1 and # 2 (virtualization) both have a transfer function for the service A, and the service control server # 1 is an active system and the service control server # 2 is a standby system. Good.

The DC 300 (DC # 2) includes a virtualization host server 341 (virtualization host server # 1) that is a physical server, a virtualization host server 342 (virtualization host server # 2), and a virtual storage 323.
The virtualization host server 341 (virtualization host server # 1) (physical) has a virtualized service control server 212 (service control server # 2).
The virtualization host server 342 (virtualization host server # 2) (physical) has a virtualized service control server 212 (service control server # 2).

As will be described later, the service control server 212 (service control server # 2) (virtualization) of the virtualization host server 341 (virtualization host server # 1) of the DC300 (DC # 2) is the DC200 (DC # 1). The service control server 212 (service control server # 2) (virtualization) is moved to DC300 (DC # 2) by live migration (see symbol o in FIG. 1). Furthermore, the service control server 212 (service control server # 2) (virtualization) of the virtualization host server 342 (virtualization host server # 2) of the DC300 (DC # 2) is the virtualization host of the DC300 (DC # 2). The service control server 212 (service control server # 2) (virtualization) virtualized by the server 341 (virtualization host server # 1) (physical) is transferred to the DC300 (DC #) by live migration (see symbol r in FIG. 1). 2) moved to the virtualization host server 341 (virtualization host server # 1).
In this embodiment, VXLAN is taken as an example of the L2 over L3 connection technology, but NVGRE (Network Virtualization using Generic Routing Encapsulation) or the like may be used.

FIG. 2 is a functional block diagram of the backup control server 100.
As shown in FIG. 2, the backup control server 100 includes a backup target server management unit 110, a backup data management unit 120, a backup control server execution management unit 130, a backup congestion control unit 140, and a network topology DB 150.
The backup target server management unit 110 manages the backup target server management list 231 (see FIG. 4) of the backup target service control server. Specifically, the backup target server management unit 110 adds / deletes the target server by live migration from the backup target server management list 231 (see FIG. 4) from which the backup control server 100 collects backup data.

  The backup control server data management unit 120 stores or deletes application data collected for backup. Specifically, the backup data management unit 120 performs generation management of the backup data of each service control server, and sequentially deletes data of servers that are outdated or not applicable.

  The backup execution management unit 130 executes backup collection according to a predetermined schedule and monitors the execution status. Specifically, the backup control server execution management unit 130 uses the backup execution management table 160 (see FIG. 3) to collect the latest backup completion time, backup start time, backup execution time, backup data for each target service control server. Manages status, RPO time, untransferred data size, and backup execution schedule.

  The backup congestion control unit 140 searches for an alternative virtual host server and a transfer route according to the load status in the network and switches them. Further, the backup congestion control unit 140 manages the average switching time for each transfer route switching pattern and the free bandwidth for each transfer route.

  The network topology DB 150 is a data database that manages transfer network information connecting each data center, and configuration information of a virtualization host server, a backup control server, and a virtual storage for each data center.

FIG. 3 is a diagram illustrating an example of the backup execution management table 160 provided in the backup control server execution management unit 130.
As shown in FIG. 3, the backup execution management table 160 includes, for each target service control server ID 161, an RPO time 162, a final backup completion time 163, an average backup execution time 164 for the latest N (N is an arbitrary natural number), a backup The scheduled start time 165, the actual backup start time 166, the elapsed time 167 since the backup started, the backup data collection status 168, and the untransferred data size 169 are stored.

FIG. 4 is a diagram illustrating an example of the backup target server management lists 231 and 331 (see FIG. 5) included in the backup target server management unit 110.
As shown in FIG. 4, the backup target server management lists 231 and 331 store service control servers managed by the backup control server. For example, the backup target server management unit 110 refers to the backup target server management lists 231 and 331, so that the backup control server # 1 must take a backup of the service control servers 1 to 5 among the service control servers 1 to 10. I understand that I have to do it.

Hereinafter, an application data backup method of the backup control server 100 configured as described above will be described.
The backup control server 100 may be in the same DC as the service control server to be backed up or outside the DC. Hereinafter, a case where the backup control server 100 is outside the DC will be described.
[When backup control server 100 is outside DC]
First, an operation example when the backup control server 100 is outside DC will be described.
5 to 10 are explanatory diagrams when the backup control server 100 is outside the DC.
When the backup control server 100 is outside the DC, further, when backup data can be collected without going through the transfer network (<operation pattern 1>: FIGS. 5 and 6), and via the transfer network. The backup data can be collected (operation pattern 2: FIGS. 8 to 10). Hereinafter, it demonstrates in order.

<Operation pattern 1> (See FIGS. 5 and 6)
<Operation pattern 1> is an operation example when backup data collection can be executed without going through the transfer network 5 (that is, without operating the backup control server 100 of FIG. 1).
FIG. 5 is an explanatory diagram of a case where backup data collection can be executed without going through the transfer network 5 (<operation pattern 1>). FIG. 6 is a diagram showing the control sequence of FIG.
As shown in FIG. 5, the DC 200 (DC # 1) includes a virtualized backup control server 221 (backup control server # 1), a migration target service control server 211 (service control server # 1), and backup target server management. A virtual storage 230 for storing the list 231 (FIG. 4) is provided. The DC 300 (DC # 2) includes a virtualized backup control server 222 (backup control server # 2), a migration target service control server 211 (service control server # 2), and a backup target server management list 331 (FIG. 4). ) Is stored.
The backup control server 221 (backup control server # 1) and the backup control server 222 (backup control server # 2) are both separate from the backup control server 100 shown in FIG. 1, and are provided for each DC site. Is. When the backup control server 221 (backup control server # 1) and the backup control server 222 (backup control server # 2) exist for each DC site, the backup control server 100 shown in FIG. 1 is not used. Note that the example of FIG. 5 corresponds formally to the prior art shown in FIG.

As shown in FIG. 5, when backup control servers 221 and 322 exist for each DC200, 300 (DC # 1, DC # 2) site, as shown in FIG. The migration target service control server 211 (service control server # 1) is deleted from the backup target server management list 231 of the backup control server 221 (backup control server # 1) on the DC 200 (DC # 1) side. Then, after performing live migration (see symbol d in FIG. 5), the migration target service control server 211 (service control server # 1) is added to the backup target server management list 331 of the backup control server 222 (backup control server # 2). To do.
5, the backup control server 222 (backup control server # 2) of DC300 (DC # 2) is the service control server to be moved to which the backup control server 221 (backup control server # 1) has moved. Backup data is collected for 311 (service control server # 1).
In this <operation pattern 1>, the backup data transfer time does not depend on the load status of the transfer network 5. The backup control servers 221 and 222 can guarantee RPO.

<Operation pattern 1> in FIG. 5 will be described with reference to the control sequence in FIG.
The target service control server 211 (service control server # 1) on the DC 200 (DC # 1) side notifies the backup control server 221 (backup control server # 1) of live migration (destination DC # 2) (step). S101).
The backup control server 221 (backup control server # 1) checks whether there is a backup control server that manages the destination DC (step S102).
The backup control server 221 (backup control server # 1) shifts to <Operation Pattern 2> (described later) when there is no backup control server that manages the destination DC, and shifts to the following when there is the backup control server. (Step S103).

The backup control server 221 (backup control server # 1) instructs the backup control server 312 (backup control server # 2) of the destination DC to the service control server 211 (service control server # 1) to be moved (step) S104).
The migration target service control server 211 (service control server # 1) requests the backup control server 221 (backup control server # 1) to delete from the backup target server management list 231 (FIG. 4) (step S105).
The backup control server 221 (backup control server # 1) receives this deletion request and deletes it from the backup target server management list 231 (FIG. 4) (step S106).

The migration target service control server 211 (service control server # 1) on the DC 200 (DC # 1) side performs live migration to the migration target service control server 211 (service control server # 1) on the DC 300 (DC # 2) side. (Step S107).
The migration-target service control server 211 (service control server # 1) on the DC300 (DC # 2) side that has been live migrated is added to the backup target server management list 331 (FIG. 4) in the backup control server 312 (backup control server # 2). An additional request is made (step S108).
The backup control server 312 (backup control server # 2) collects backup data in the next cycle (step S109).
The backup control server 312 (backup control server # 2) requests the backup control server 221 ((service control server # 1) to delete the backup data of the service control server # 1 (step S110).
In response to this deletion request, the backup control server 221 (backup control server # 1) instructs the virtual storage 230 (virtual storage # 1) to delete the backup data of the service control server # 1 (step S111).
The virtual storage 230 (virtual storage # 1) deletes the backup data of the service control server # 1 (step S112).

<Operation pattern 2> (See FIGS. 7 and 8)
<Operation pattern 2> is an operation example when backup data collection needs to be executed via the transfer network 5 (without operating the backup control server 100 of FIG. 1).
FIG. 7 is an explanatory diagram when backup data collection needs to be executed via the transfer network 5 (<operation pattern 2>). FIG. 8 is a diagram showing the control sequence of FIG.

As shown in FIG. 7, there is no backup control server at the DC200, 300 (DC # 1, DC # 2) base, and the virtual network is located in another DC400 other than the DC200, 300 (DC # 1, DC # 2) base. Assume that there is a backup control server 421 (backup control server # 1).
As shown in FIG. 7, when there is no backup control server for each DC200, 300 (DC # 1, DC # 2) site, as shown by the symbol f in FIG. 5, the movement target of DC200 (DC # 1) It is assumed that the service control server 211 (service control server # 1) has been live-migrated and moved to the DC 300 (DC # 2) (see symbol f in FIG. 7).
After live migration and migration to DC300 (DC # 2), backup control server 421 (backup control server # 1) virtualized to DC400 at another site collects backup data. At this time, there is a case of backup data collection that is completed within the RPO as indicated by reference numeral g in FIG. 7 and a case of backup data collection that may not be completed within the RPO as indicated by reference numeral h of FIG. . In the case of backup data collection that may not be completed within the RPO, the backup will not be completed even if a certain time has elapsed.
As described above, when it is necessary to execute backup data collection via the transfer network 5, if the transfer route of the backup data in the transfer network 5 becomes long, the load status of the transfer network 5 may be affected.

<Operation pattern 2> in FIG. 7 will be described with reference to the control sequence in FIG. The control sequence of FIG. 8 includes processing for delay detection of backup data collection and delay factor determination. <Operation pattern 2> shifts to step S103 of the control sequence of FIG. 6 when there is no backup control server at the DC200, 300 (DC # 1, DC # 2) base.
As shown in FIG. 7, it is assumed that a virtual backup control server 421 (backup control server # 1) exists in the DC 400 at another site.
The backup control server 421 (backup control server # 1) of the DC 400 at another base is a service control server 311 (service control) virtualized by the virtualization host server 341 (virtualization host server # 1) of DC 200 (DC # 1). Backup data collection is executed for the server # 1) via the backup data transfer route # 1 in the transfer network 5 (step S201).
The backup control server 421 (backup control server # 1) sets a monitoring timer (step S202).
The backup control server 421 (backup control server # 1) confirms that the backup data collection is not completed even when the time elapsed by the monitoring timer expires (step S203).

The backup control server 421 (backup control server # 1) sees (refers to) the backup execution management table 160 (see FIG. 3), and performs service control in another virtualization host server that uses the transfer route # 1. It is confirmed whether the server backup has been normally completed (step S204).
The backup control server 421 (backup control server # 1) determines whether the backup of the service control server is normally completed or not completed (step S205).
In the case of normal completion, the backup control server 421 (backup control server # 1) determines that the corresponding virtualization host server # 1 is in a high load state and shifts to <operation pattern 3> (step S206).
If not completed, the backup control server 421 (backup control server # 1) determines that the transfer route # 1 is in a high load state and proceeds to <operation pattern 4> (step S207).

  As described above, when backup data collection is not completed, the backup control server 421 (backup control server # 1) successfully completes backup of the service control server in the other virtualization host server using the transfer route # 1. To determine whether the cause of the incomplete backup data collection is the high load state of the virtualization host server # 1 or the high load state of the transfer route # 1. I have to.

<Operation pattern 3> (See FIGS. 9 and 10)
<Operation pattern 3> has a delay factor in the virtualization host server when it is necessary to execute backup data collection via the transfer network 5 (without operating the backup control server 100 in FIG. 1). It is an operation example.
FIG. 9 is an explanatory diagram when backup data collection needs to be executed via the transfer network 5 (<operation pattern 3>). FIG. 10 is a diagram showing the control sequence of FIG.

As shown in FIG. 9, there is a virtualized backup control server 421 (backup control server # 1) in the DC 400 at another site other than the DC 300 (DC # 2) site.
As indicated by reference sign i in FIG. 9, the backup control server 421 (backup control server # 1) does not complete backup data collection.
However, since the backup of the service control server in another host server using the same transfer route has been completed normally (see steps S204 and S205 in FIG. 8), the virtualization host server # 1 may be in a congested state. It can be determined that there is. In this case, the target service control server is live migrated again to another host server # 2. Specifically, as indicated by the symbol i in FIG. 9, the service control server 211 of the virtualization host server # 1 of the DC 300 (DC # 2) is live migrated again to another virtualization host server # 2.

<Operation pattern 3> in FIG. 9 will be described with reference to the control sequence in FIG. The control sequence in FIG. 9 is a countermeasure example when the delay factor is the virtualization host server. <Operation pattern 3> shifts to the case where the corresponding virtualization host server # 1 determines in the high load state in step S206 of the control sequence of FIG.
As shown in FIG. 10, it is assumed that a virtualized backup control server 221 (backup control server # 1) exists in the DC 200 (DC # 1).
The backup control server 221 (backup control server # 1) determines that the virtualization host server # 1 is in a high load state (step S301).
The backup control server 221 (backup control server # 1) looks at the backup execution management table 160 (see FIG. 3), and each other virtualization host server in the DC 300 (DC # 2) that uses the transfer route # 1. Compare the sum of the remaining time until the RPO of the service control server # 1, the average time required for live migration, and the average backup time for each other virtualization host server in DC # 2, and the load is the lowest ( A search is made for another virtualization host server having a large comparison value difference (step S302).
It is determined whether there is another virtual host server with the lowest load (the comparison value difference is large) (step S303).
When there is no other virtualization host server with the lowest load, the backup control server 221 (backup control server # 1) determines that live migration cannot be executed on the other virtualization host server. > (Step S303).

When there is another virtualization host server with the lowest load, the backup control server 221 (backup control server # 1) instructs the service control server # 1 to perform live migration to the other virtualization host server # 2. (Step S304).
The backup control server 221 (backup control server # 1) makes the service control server # 1 virtual server 342 (virtual host server 342 (virtual host server # 1) and virtual host server 341 (virtual host server # 1) of DC200 (DC # 1). The live migration instruction is issued so as to move to the network host server # 2) (step S305).

The virtualization host server 341 (virtualization host server # 1) of DC200 (DC # 1) performs live migration of the service control server 211 (service control server # 1) to the virtualization host server 342 (virtualization host server # 2). (Step S306). In other words, the service control server 211 (service control server # 1) of the virtualization host server # 1 of the DC300 (DC # 2) is live migrated again to another virtualization host server # 2.
The backup control server 221 (backup control server # 1) transfers the backup data transfer route # in the transfer network 5 to the service control server 211 (service control server # 1) of the DC 300 (DC # 2) that has been live migrated again. The backup data collection is executed via 1 (step S307).

<Operation pattern 4> (See FIG. 11)
<Operation pattern 4> is a case where the delay factor is in the transfer route when it is necessary to execute backup data collection via the transfer network 5 (without operating the backup control server 100 in FIG. 1). It is an operation example.
FIG. 11 is a diagram showing a control sequence when backup data collection needs to be executed via the transfer network 5 (<operation pattern 4>). The control sequence in FIG. 11 is a handling example when the delay factor is the transfer route. <Operation pattern 4> shifts when transfer route # 1 is determined to be in a high load state in step S207 of the control sequence of FIG.
The backup control server 221 (backup control server # 1) determines that the virtualization host server # 1 is in a high load state (step S401).
The backup control server 221 (backup control server # 1) looks at the backup execution management table 160 (see FIG. 3), and shows the remaining time until the RPO of the service control server # 1 (average time and transfer route for transfer route switching) The other backup routes with the lowest load (large difference in comparison value) are searched for by comparing the sum with the backup time calculated from each available bandwidth (step S402).
When there is no other transfer route with the lowest load, the backup control server 221 (backup control server # 1) determines that live migration cannot be executed with the other transfer route and shifts to <operation pattern 5>. (Step S403).

If there is another transfer route with the lowest load, the backup control server 221 (backup control server # 1) performs switching to another transfer route (for example, transfer route # 2) (step S404).
The backup control server 221 (backup control server # 1), the virtualization host server 341 (virtualization host server #) of the DC 200 (DC # 1) so as to change the transfer route to another transfer route (for example, transfer route # 2). 1) is instructed (step S405).

The service control server 211 (service control server # 1) of the virtualization host server 341 (virtualization host server # 1) of the DC 200 (DC # 1) designates the transfer route # 2 as the transfer route identifier at the time of encapsulation. The backup data is transmitted (step S406).
The backup control server 221 (backup control server # 1) sends backup data to the service control server 211 (service control server # 1) of the DC300 (DC # 2) via the transfer route # 2 whose transfer route has been changed. Collection is executed (step S407).
The above has described <operation pattern 1> to <operation pattern 4> when the backup control server 100 illustrated in FIG. 1 is outside DC.

Next, the operation of the backup control server 100 will be described.
[Operation of Backup Control Server 100]
The backup control server 100 automatically backs up application data to a service control server (virtualization server) in a wide area large-scale network.
<Operation pattern 5> (See FIG. 12)
<Operation pattern 5> is an operation example when it is necessary to execute backup data collection by operating the backup control server 100 of FIG.
Note that <operation pattern 5> indicates that there is no other virtual host server having the lowest load in step S303 of the control sequence in FIG. 11, or another transfer having the lowest load in step S403 of the control sequence in FIG. Transition from when there is no route.

FIG. 12 is an explanatory diagram when it is necessary to execute backup data collection by operating the backup control server 100 of FIG. 1 via the transfer network 5.
As shown in FIG. 12, the backup control server 100 is connected to the DC 300 (DC # 2) via the transfer network 5. As indicated by a symbol k in FIG. 12, the backup control server 100 executes backup data collection via the backup data transfer route # 1 in the transfer network 5.
Now, it is assumed that the completion of uniform backup data collection is delayed in service control servers in a plurality of virtualization host servers that use the same transfer route. In this case, there is a possibility that the transfer route # 1 is congested. As indicated by reference numeral 1 in FIG. 12, the backup control server 100 switches the transfer route # 1 of the backup data in the transfer network 5 to another transfer route # 2 having a large free bandwidth.
As described above, when the backup collection is not completed even after a predetermined time has elapsed, the backup control server 100 analyzes the backup execution management table 160 (FIG. 2) and determines that the transfer route # 1 is in a congestion state. Then, the backup control server 100 executes switching to the transfer route # 2 having a large free bandwidth and continues the backup collection of the service control server # 1.

Next, the operation when the backup control server 100 cannot find the switching destination virtualization host server or transfer route will be described.
FIG. 13 is an explanatory diagram of the operation when the backup control server 100 cannot find the switching destination virtualization host server or transfer route.
As indicated by the symbol m in FIG. 13, the backup control server 100 is executing backup data collection.
When the remaining time until RPO is short, the virtual host server 341 (virtual host server # 1) of the DC300 (DC # 2) performs an emergency backup (described later) to the belonging virtual storage 323 (virtual storage # 1) To do.
As indicated by the symbol n in FIG. 13, the virtualization host server 341 (virtualization host server # 1) of the DC300 (DC # 2) estimates the timing when the transfer route is free after the emergency backup, and the backup control server 100 Data to other virtual storage.

When the backup control server 100 cannot find the switching destination virtualization host server or the transfer route, the virtualization host server 341 (virtualization host server # 1) of the DC300 (DC # 2) The backup data of the service control server # 1 is collected in the virtual storage 323 (virtual storage # 1) belonging to (attached to) 341 (virtualization host server # 1).
The backup control server 100 refers to the backup execution management table 160 (see FIG. 2) and estimates the emergency backup data from the virtual storage 323 (virtual storage) of the DC300 (DC # 2) at the timing when the number of simultaneous backup collection servers is small. Collect from # 1).

Next, emergency backup will be described.
[Emergency backup]
FIG. 14 is a diagram showing a control sequence for emergency backup collection when there is no switching destination virtualization host server or transfer route. The control sequence in FIG. 14 is an example of a case where backup data collection needs to be executed via the transfer network 5.
As shown in FIG. 14, the backup control server 221 (backup control server # 1) of the DC 200 (DC # 1) has a virtual host of the DC 300 (DC # 2) when there is no switching destination virtualization host server or transfer route. The backup data is urgently transferred to the virtual storage # 1 (see FIG. 13) belonging to the virtualization host server 341 (virtualization host server # 1) (step S501).
The backup control server 221 (backup control server # 1) is a virtual storage for the service control server 311 (service control server # 1) of the virtualization host server 341 (virtualization host server # 1) of DC300 (DC # 2). An emergency backup is instructed to # 1 (step S502).
The service control server 311 (service control server # 1) of the DC 300 (DC # 2) urgently transfers the backup data of the virtual storage 323 (virtual storage # 1) (step S503).
The service control server 311 (service control server # 1) of the DC 300 (DC # 2) sends an emergency backup completion response to the backup control server 221 (backup control server # 1) of the DC 300 (DC # 2) (step S504).

The backup control server 221 (backup control server # 1) refers to the backup execution management table 160 (see FIG. 3) and extracts the timing when the number of simultaneous backup collection servers is small (step S505).
The backup control server 221 (backup control server # 1) of the DC 200 (DC # 1) collects emergency backup data at the extracted timing (step S506).
The backup control server 221 (backup control server # 1) sends backup data to the service control server 211 (service control server # 1) of the DC300 (DC # 2) via the transfer route # 1 whose transfer route has been changed. Recovery is executed (step S507).

Hereinafter, the application data backup method of the backup control server 100 will be described with reference to the overall configuration diagram of FIG. 1 again.
As shown by a symbol o in FIG. 1, the virtualized service control server can freely move to an arbitrary DC without interrupting the service by live migration. In the example of FIG. 1, the service control server 311 (service control server # 2) of the DC 200 (DC # 1) performs the service of the service control server 212 (service control server # 1) of the DC 300 (DC # 2) by live migration. It can move to the control server 212 (service control server # 1). The backup data transfer time is influenced by the load condition in the transfer network 5 and has a feature that the fluctuation is large. If a communication failure or congestion on the route for transferring the backup data occurs and the backup time becomes remarkably long, it becomes impossible to guarantee the RPO to be recovered when a failure occurs in the target server.

As indicated by reference sign p in FIG. 1, the backup control server 100 executes backup data collection via the backup data transfer route # 1 in the transfer network 5.
As shown by the symbol q in Fig. 1, if the backup is not completed even after a certain period of time, analyze the delay factor and take action for each factor (see Countermeasure Patterns (1), (2), (3)) RPO can be guaranteed.

<Action pattern (1)>
As indicated by the symbol r in FIG. 1, when the virtualization host server is congested, live migration is performed again to another virtualization host server with a low negative load (<operation pattern 3> (see FIGS. 9 and 10)).

<Action pattern (2)>
As shown by reference numeral s in FIG. 1, when the backup data transfer route (here, transfer route # 1) is congested, the backup data transfer route (here, transfer route # 2) is switched to another transfer route having a larger free bandwidth (here, transfer route # 2). 5> (see FIG. 12)).

<Action pattern (3)>
As shown by the symbol t in FIG. 1, when the remaining time until the RPO is short, the emergency backup is executed on the belonging virtual storage (<emergency backup> (see FIG. 14)).
As shown by a symbol u in FIG. 1, after emergency backup, data is migrated to the virtual storage of the backup control server 100 in anticipation of the timing when the transfer route is free.

  As described above, the backup control server 100 stores or deletes the backup target server management unit 110 that manages the backup target server management list 231 (see FIG. 4) of the backup target service control server and the application data collected for backup. The backup data management unit 120, the backup execution management table 160 (see FIG. 3) are used to perform backup collection according to a predetermined schedule and monitor the execution status, and the load status in the network. The backup congestion control unit 140 that manages the average switching time for each transfer route switching pattern and the free bandwidth for each transfer route is connected to each data center while searching for an alternative virtual host server and transfer route using Feed comprising network information, virtualization per data center host server and the backup control server, the network topology DB150 for managing configuration information of the virtual storage, a.

As a result, in a large-scale network composed of a plurality of data centers distributed over a wide area, application data can be transferred when a virtualized service control server freely moves to any data center without interrupting the service by live migration. RPO can be guaranteed even if the backup time fluctuates greatly depending on the load status of the physical server (virtualized host server) that executes the transfer network and each virtual service control server. By guaranteeing the RPO, it is possible to guarantee the communication quality at the time of trouble to the user.
In addition, since the physical server is automatically replaced and the transfer route is switched on the network side without intervention of the maintenance person, the burden on the maintenance person is reduced. In addition, by smoothing the local traffic load in the network, stable operation of the network can be promoted.
In this embodiment, the backup control server 100 (see FIG. 1) can collect backup data of the virtual storage in the DC and store it in its own storage, thereby improving the reliability at the time of emergency backup. Can do.

In addition, among the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Further, each of the above-described configurations, functions, and the like may be realized by software for interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function is stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), an IC (Integrated Circuit) card, an SD (Secure Digital) card, an optical disk, etc. It can be held on a recording medium. Further, in this specification, the processing steps describing time-series processing are not limited to processing performed in time series according to the described order, but are not necessarily performed in time series, either in parallel or individually. The processing (for example, parallel processing or object processing) is also included.

DESCRIPTION OF SYMBOLS 5 Transfer network 100,221,322,421 Backup control server 110 Backup object server management part 120 Backup data management part 130 Backup control server execution management part 140 Backup congestion control part 150 Network topology DB
160 Backup execution management table 200, 300, 400 Data center (DC)
211, 212, 312 Virtualized service control server 230, 330 Virtual storage 231, 331 Backup target server management list 313 Virtual switch 341, 342 Virtualization host server consisting of physical servers 323 Virtual storage

Claims (8)

A backup control server for backing up application data of a virtualized service control server that moves between data centers distributed in a network,
A backup target server management unit for managing a management list of the service control server to be backed up;
A backup data management unit for storing or deleting application data collected for backup;
A backup control server execution management unit that executes backup collection according to a predetermined schedule and monitors the execution status;
A backup congestion control unit that searches for an alternative physical server and a transfer route based on a load status of resources in the network and controls switching of the alternative physical server and the transfer route. server. The backup control server execution management unit manages the latest backup completion time and backup start time, backup execution time, backup data collection status, RPO time, untransferred data size, or backup execution schedule for each service control server. The backup control server according to claim 1, wherein: The backup congestion control unit searches and switches an alternative virtual host server and transfer route according to the load status in the network, and manages the average switching time for each transfer route switching pattern or the free bandwidth for each transfer route. The backup control server according to claim 1, wherein: A backup control server application data backup method for backing up virtualized service control server application data moving between data centers distributed in a network,
A backup target server management process for managing a management list of the service control servers to be backed up;
Backup data management process for storing or deleting application data collected for backup,
A backup control server execution management process for executing backup collection according to a predetermined schedule and monitoring the execution status;
A backup congestion control step of searching for an alternative physical server and a transfer route based on a load status of the resource in the network and controlling switching of the alternative physical server and the transfer route;
An application data backup method for a service control server, comprising: A backup control server that backs up application data of a virtualized service control server that moves between data centers distributed in the network is a service control server application data backup method provided in each of the data centers,
A backup target server management process for managing a management list of the service control servers to be backed up;
Backup data management process for storing or deleting application data collected for backup,
A backup control server execution management process for executing backup collection according to a predetermined schedule and monitoring the execution status;
A backup congestion control step of searching for an alternative physical server and a transfer route based on a load state of the resource in the network and controlling switching of the alternative physical server and the transfer route, and
When the service control server moves to the physical server of another data center via the network by live migration,
Deleting the moved service control server from the management list of the backup control server of the source data center;
Adding the service control server moved from the management list of the backup control server of the data center to be moved;
Erasing the backup data held by the backup control server at the migration source data center when the first backup collection is completed for the service control server moved at the migration destination data center; An application data backup method for a service control server, comprising: When the service control server moves to a physical server in another data center via the network by live migration,
Determining the transfer delay factor from the backup execution status when the backup collection status monitoring timer expires;
Selecting another physical server in a low load state in the same data center when a transfer delay factor is in a high load state of the physical server in the data center of the destination;
The service control server application data backup method according to claim 4, further comprising: performing live migration again on the service control server to another physical server. When the service control server moves to a physical server in another data center via the network by live migration,
Determining the transfer delay factor from the backup execution status when the backup collection status monitoring timer expires;
Selecting another transfer route in a low load state when the transfer route connecting the data center of the movement source and the data center of the movement destination is in a high load state when a transfer delay factor is;
6. The application control data backup method for a service control server according to claim 4, further comprising a step of switching to another transfer route. When the service control server moves to a physical server in another data center via the network by live migration,
Determining the transfer delay factor from the backup execution status when the backup collection status monitoring timer expires;
The physical server of the replacement destination when the physical server of the data center of the migration destination as a transfer delay factor, or the transfer route connecting the data center of the migration source and the data center of the migration destination is in a high load state, or A step of temporarily backing up application data to a storage belonging to the physical server of the data center of the migration destination when the transfer route is not found;
5. The service control server application according to claim 4, wherein the backup control server collects the temporarily stored backup data at a timing when the number of simultaneously executing collection servers in the backup execution schedule is small. Data backup method.

Images