JP6554042B2 - Data relocation system, data relocation apparatus, and data relocation method - Google Patents

Description

  The present invention relates to a data relocation system, a data relocation apparatus, and a data relocation method.

  Conventionally, in a distributed system, techniques for constructing a large-scale communication system have been proposed. For example, Non-Patent Document 1 describes an optimum that follows changes in the execution environment by detecting changes related to infrastructure on a distributed service platform that enables a distributed system, and repeatedly calculating and redeploying the optimum deployment destination. Technologies have been proposed to realize various distributed deployments. In the construction of a system as described above, in general, requirements for processing time of data manipulation are very strict, and real time property is strongly required for data update.

  FIG. 4 is a diagram showing an example of the configuration of a network 100 including a distributed system 102 according to the prior art. As shown in FIG. 4, the network 100 includes a maintenance terminal 101, distributed systems 102, service providing devices 103a and 103b, and user terminals 104a, 104b, 104c and 104d. Each of these components is connected so that various signals and data can be transmitted and received in one direction or in both directions. The distributed system 102 includes a plurality of nodes 102a, 102b, 102c, and 102d having a DB (Data Base). With the development of information communication technology, a DB requires a real-time response to mass access from an external device. In order to respond to such demands, it is becoming common for DBs to operate in a distributed and redundant manner.

  Hereinafter, with reference to FIG. 5A and FIG. 5B, assuming a case where data is provided to a service that requires a particularly high real-time response (for example, a service for large-scale users such as communication call processing and games), A data rearrangement method will be described. FIG. 5A is a diagram for explaining the first half of the data rearrangement method in the distributed system 102 according to the related art. As shown in FIG. 5A, for example, the accommodating apparatus 100b which is the relocation destination of the data D102 is determined in consideration of the call volume and the weight of the data. FIG. 5B is a diagram for explaining the latter half of the data rearrangement method in the distributed system 102 according to the prior art. As shown in FIG. 5B, for example, when the data D101 is actually rearranged from the storage device 100a to the storage device 100c, the maintenance manager performs the data transfer work, Implement split construction.

“Development of distributed service platform technology to optimize cloud environment on wide area network”, [online], Fujitsu Laboratories Ltd. System Software Research Laboratories, Distributed Platform Research Department, [Search February 1, 2016], Internet < http://pr.fujitsu.com/jp/news/2014/03/14.html>

  However, the above-described conventional techniques have the following problems. For example, since the call volume and data weight vary greatly depending on time and data attributes, even if the call volume and data weight at a certain point are taken into consideration, it is optimal according to the subsequent operation and traffic changes. The relocation destination changes. For this reason, the effect assumed at the time of examination of a data relocation place may not be acquired. In addition, the data migration work, the site division work, etc. may be very large in maintenance operation for the system operator. These are factors that hinder optimal data rearrangement and reduce the data update rate.

  The disclosed technology has been made in view of the above, and is a data relocation system, a data relocation device, and a data relocation that automatically perform optimal data relocation on data held in a distributed manner Intended to provide a method.

  In order to solve the above-described problems and achieve the object, a data rearrangement system disclosed in this application includes, in one aspect, a plurality of storage devices and a plurality of opposing devices connected to any of the plurality of storage devices. A data relocation system having a device, wherein each of the plurality of storage devices is a storage device having a minimum total value of costs from each of the plurality of opposing devices among the plurality of storage devices, A determination unit that determines a storage device that is a relocation destination of data held in any storage device in the data relocation system, and the rearrangement of the data to the storage device determined by the determination unit And a rearrangement unit.

  Further, in one aspect, the data relocation apparatus disclosed in the present application has a minimum total cost from each of a plurality of opposing devices connected to any of the plurality of accommodating devices. And a relocating unit for relocating the data with respect to the accommodating device determined by the determining unit. And an arrangement portion.

  Furthermore, in one aspect, the data relocation method disclosed in the present application is executed in a data relocation system having a plurality of accommodation devices and a plurality of opposing devices connected to any of the plurality of accommodation devices. A data relocation method, wherein at least one of the plurality of storage devices is the smallest total value of the costs from each of the plurality of opposing devices among the plurality of storage devices. Determining the storage device to be the relocation destination of the data stored in any storage device in the data relocation system, and the storage device determined in the determination step, A rearrangement step of rearranging.

  The data relocation system, the data relocation device, and the data relocation method disclosed in the present application have an effect that the optimal data relocation can be automatically performed on the distributed and held data.

FIG. 1A is a diagram showing an example of a distributed system before data relocation. FIG. 1B is a diagram showing an example of a distributed system after data relocation. FIG. 2 is a diagram showing an example of the configuration of the storage device according to the present embodiment. FIG. 3 is a diagram illustrating that the information processing by the data rearrangement program is specifically realized using a computer. FIG. 4 is a diagram showing an example of the configuration of a network including a distributed system according to the prior art. FIG. 5A is a diagram for explaining the first half of the data rearrangement method in the distributed system according to the related art. FIG. 5B is a diagram for explaining the latter half of the data rearrangement method in the distributed system according to the related art.

  Hereinafter, embodiments of a data relocation system, a data relocation apparatus, and a data relocation method disclosed in the present application will be described in detail with reference to the drawings. Note that the data relocation system, data relocation apparatus, and data relocation method disclosed in the present application are not limited by the following embodiments.

  First, a distributed system 1 as a data relocation system according to an embodiment disclosed in the present application will be described. FIG. 1A is a diagram showing an example of the distributed system 1 before relocation of data D1. As shown in FIG. 1A, the distributed system 1 transfers data D1, D2, and D3 identified by “050-AAAA”, “050-BBBB”, and “050-CCCC” to the storage devices 10a, 10b, and 10c, respectively. Disperse and hold. In FIG. 1A, the numerical values described between the accommodating devices 10a to 10f and the opposing devices 10g to 10i represent communication costs between the respective paths.

  For example, the data D1 identified by the data ID “050-AAAA” is currently held by the accommodating device 10a, the call volume from the opposite device 10g is “1”, and the call volume from the opposite device 10h is “2”. The call volume from the opposing apparatus 10i is "3". The data D2 identified by the data ID “050-BBBB” is held by the accommodating device 10b, and the call volumes from the opposing device 10g, the opposing device 10h, and the opposing device 10i are “2”, “2”, “1”. Similarly, the data D3 identified by the data ID “050-CCCC” is held by the accommodating device 10c, and the call volumes from the opposing device 10g, the opposing device 10h, and the opposing device 10i are “3” and “1”, respectively. , “0”.

  FIG. 1B is a diagram illustrating an example of the distributed system 1 after the data D1 is rearranged. In the present embodiment, of the data D1, D2, and D3 held in a distributed manner, the data D1 is used as the data to be rearranged. As shown in FIG. 1B, regarding the data D1 having the data ID “050-AAAA”, the total costs of the accommodation apparatuses 10a to 10f are “13”, “9”, “10”, “11”, “10”, respectively. , “10”. Therefore, the data D1 disposed in the accommodation device 10a is rearranged in the accommodation device 10b having the minimum value "9" as the total cost as shown by the arrow Y1. Thereby, the total cost at the time of data update using data D1 is reduced. Further, the data D1 having the data ID “050-AAAA” is rearranged in the vicinity of the opposite apparatus 10i having the maximum call volume “3”, and the data arrangement is optimized.

  FIG. 2 is a diagram showing an example of the configuration of the storage device 10a according to the present embodiment. As illustrated in FIG. 2, the storage device 10 a includes a cost calculation unit 11, a relocation destination determination unit 12, a data relocation unit 13, and a data holding unit 14. Each of these components is connected so that various signals and data can be input and output in one or both directions. The cost calculation unit 11 calculates communication costs in order from the data with the largest call volume from the plurality of opposing devices 10g to 10i. The rearrangement destination determination unit 12 performs data rearrangement on a storage device having a minimum total communication cost (total cost) from each of the plurality of opposing devices 10g to 10i among the plurality of storage devices 10a to 10f. Decide on the first storage device. The data relocation unit 13 relocates the data to the determined storage device. The data holding unit 14 holds data to be rearranged.

  The configuration and operation of the storage device 10a have been representatively described above, but the configuration and operation of the storage devices 10b to 10f other than the storage device 10a are the same as the configuration and operation of the storage device 10a. Detailed description will be omitted.

  As described above, the distributed system 1 includes the plurality of storage devices 10a to 10f and the plurality of counter devices 10g to 10i connected to any of the plurality of storage devices 10a to 10f. Each of the plurality of storage devices 10 a to 10 f includes a rearrangement destination determination unit 12 and a data rearrangement unit 13. The relocation destination determination unit 12 assigns the accommodation device 10b having the smallest total communication cost (total cost) from each of the plurality of opposing devices 10g to 10i to the distributed system 1 among the plurality of accommodation devices 10a to 10f. The storage device that is the relocation destination of the data held in the data holding unit 14 of any of the storage devices 10a is determined. The data rearrangement unit 13 rearranges the data with respect to the accommodation apparatus 10b determined by the rearrangement destination determination unit 12.

  According to the distributed system 1, the influence of fluctuations in traffic and data accommodation due to the passage of time is suppressed, and highly real-time data relocation is realized. That is, the distributed system 1 evaluates the communication cost based on the network configuration with little fluctuation due to the passage of time, automatically calculates the optimum data arrangement destination using the result of the evaluation, and executes the data rearrangement. . Thereby, it is possible to improve the data update speed (for example, the upgrade speed of the data held in the opposite apparatus) by the optimal data rearrangement. In addition, the distributed system 1 performs automatic data relocation, so that the maintenance operation does not increase even in the data migration for improving the real-time property, thereby reducing the burden on the maintenance manager accompanying the maintenance operation. Is possible.

  Further, in the distributed system 1, the relocation destination determination unit 12 of the accommodation device 10a, when there are a plurality of accommodation devices having the minimum total communication cost, the data among the plurality of accommodation devices 10a to 10f. The storage device closest to the opposing device 10i having the largest call volume for the device may be determined as the storage device to be the relocation destination of the data. As a result, even if there is a plurality of storage devices with the lowest communication cost with respect to certain data, data can be preferentially relocated in the vicinity of the opposing device where the number of times of reference to the data is high, which is more efficient It is possible to realize data relocation, and thus, an increase in data update speed.

  Here, the closest accommodation device is not necessarily the accommodation device with the smallest physical distance, for example, the minimum number of hops, the shortest data transmission / reception time, the best data communication quality, or the minimum communication cost. It may be a containing device.

  In the distributed system 1, each of the plurality of accommodating devices 10a to 10f searches for a predetermined optimum route in order from the data with the largest call volume from the plurality of opposing devices 10g to 10i (for example, Dijkstra method, Bellman Ford method). May be further included to calculate the communication cost. In this aspect, the relocation destination determination unit 12 determines the accommodation device having the smallest total value of the communication costs calculated by the cost calculation unit 11 as the accommodation device to be the relocation destination of the data. As a result, for example, it is possible to preferentially shift data from a high reference frequency to an optimal storage device, so it is possible to realize more efficient data relocation, and thus, an improvement in data update speed.

  In the above embodiment, each of the accommodation devices 10a to 10f in the distributed system 1 has the above-described data relocation function individually. However, the above function is separate from the accommodation devices 10a to 10f. It may be equipped with a centralized device configured in In such an aspect, the centralized device may be disposed either inside or outside the distributed system 1.

  Also, in the above embodiment, the communication cost is illustrated as an evaluation target when determining an accommodation device to which data is to be relocated, but in comparison with call volume and data weight (accommodation efficiency), time lapse Other indicators (for example, delay time between devices such as the storage device and the facing device) may be used as the evaluation target, as long as the index has a small fluctuation due to. The communication cost is not limited to the physical distance, and the number of hops, the speed and state of the line, the communication cost, etc. may be used. Furthermore, the call volume considered in the case where there are a plurality of accommodating apparatuses in which the total value of the communication costs is minimum is, for example, the number of times of reference of data, but may be the frequency of use of data.

  Further, the trigger for executing the data rearrangement process is, for example, a change in the total value of communication costs related to the data (for example, a change in the accommodation device that minimizes the total cost for certain data), but is not limited thereto. Instead, it may be a change of the opposite device having the maximum call volume for the data, or a change of the accommodation device closest to the opposite device.

(Data relocation program)
FIG. 3 is a diagram illustrating that the information processing by the data rearrangement program is specifically realized by using the computer 200. As shown in FIG. 3, the computer 200 includes, for example, a memory 201, a CPU (Central Processing Unit) 202, a hard disk drive interface 203, a disk drive interface 204, a serial port interface 205, a video adapter 206, a network, and the like. An interface 207 is included, and these units are connected by a bus C.

  As shown in FIG. 3, the memory 201 includes a ROM (Read Only Memory) 201a and a RAM (Random Access Memory) 201b. The ROM 201a stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 203 is connected to the hard disk drive 208 as shown in FIG. The disk drive interface 204 is connected to the disk drive 209 as shown in FIG. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 209. The serial port interface 205 is connected to, for example, a mouse 210 and a keyboard 211 as shown in FIG. The video adapter 206 is connected to, for example, the display 212 as shown in FIG.

  Here, as shown in FIG. 3, the hard disk drive 208 stores, for example, an OS (Operating System) 208a, an application program 208b, a program module 208c, program data 208d, and data D1 to D3. That is, the data relocation program according to the disclosed technology is stored in, for example, the hard disk drive 208 as a program module 208 c in which an instruction to be executed by the computer 200 is described. Specifically, the program module 208c in which various procedures for executing the same information processing as the cost calculation unit 11, the relocation destination determination unit 12, and the data relocation unit 13 described in the above embodiment are written on the hard disk Stored in drive 208. Further, data used for information processing by the data rearrangement program is stored in the hard disk drive 208, for example, as program data 208d. Then, the CPU 202 reads the program module 208 c and the program data 208 d stored in the hard disk drive 208 into the RAM 201 b as necessary, and executes the various procedures.

  Note that the program module 208c and the program data 208d related to the data rearrangement program are not limited to being stored in the hard disk drive 208, but are stored in, for example, a removable storage medium and read by the CPU 202 via the disk drive 209 or the like. May be. Alternatively, the program module 208c and the program data 208d relating to the data rearrangement program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.), and the network interface 207 May be read out by the CPU 202.

  Further, each component of the distributed system 1 described above does not necessarily have to be physically configured as illustrated. That is, the specific aspect of the dispersion and integration of each device is not limited to the one shown in the drawings, but all or a part thereof is functionally or physically dispersed in any unit depending on various loads, usage conditions, etc. -It can also be integrated and configured. For example, regarding the accommodation apparatus 10a, the relocation destination determination unit 12 and the data relocation unit 13 may be integrated as one component. Conversely, the relocation destination determination unit 12 may be distributed to a portion that decides the relocation destination from the total value of the communication costs and a portion that decides the relocation destination from the maximum value of the traffic volume. Further, the hard disk drive 208 may be connected as an external device of the distributed system 1 via a network or a cable.

1 Distributed system 10a, 10b, 10c, 10d, 10e, 10f Accommodating device 10g, 10h, 10i Counter device 11 Cost calculation unit 12 Relocation destination determination unit 13 Data relocation unit 14 Data holding unit 100 Networks 100a, 100b, 100c, 100d, 100e, 100f Accommodating device 100g, 100h, 100i Opposing device 101 Maintenance terminal 102 Distributed systems 102a, 102b, 102c, 102d Nodes 103a, 103b Service providing devices 104a, 104b, 104c, 104d User terminal 200 Computer 201 Memory 201a ROM
201b RAM
202 CPU
203 Hard Disk Drive Interface 204 Disk Drive Interface 205 Serial Port Interface 206 Video Adapter 207 Network Interface 208 Hard Disk Drive 208a OS
208b Application program 208c Program module 208d Program data 209 Disk drive 210 Mouse 211 Keyboard 212 Display C Bus D1, D2, D3, D101, D102, D103 Data

Claims (5)

A data relocation system having a plurality of storage devices and a plurality of opposing devices connected to any of the plurality of storage devices,
Each of the plurality of storage devices is
The total value of the cost from each of the plurality of opposing devices is calculated among the plurality of accommodation devices , and when there is one accommodation device having the smallest total value of the costs, the data of the accommodation device is the data If there are a plurality of storage devices which are to be relocation destinations of data held in any of the storage devices in the relocation system and there are a plurality of storage devices having the smallest total value of the costs, A determining unit that determines the accommodating device closest to the opposite device having the maximum call volume for the data as the accommodating device to be the relocation destination of the data ;
A data holding unit for holding the data in a storage device;
A data rearrangement system comprising: a rearrangement unit that rearranges the data with respect to the accommodation device determined by the determination unit when the data is held in the data holding unit. The determination unit recalculates the total value of the costs from each of the plurality of opposing devices , and when the total value of the costs fluctuates, the storage device having the smallest total value of the costs is re-stored as 2. The data rearrangement system according to claim 1, wherein the data rearrangement system is determined to be a rearrangement destination of data held in any of the accommodation devices in the arrangement system. Each of the plurality of storage devices is
The calculation unit further calculates the cost in order from data with a large call volume from the plurality of opposing devices,
The data re-storing device according to claim 1, wherein the determining unit determines an accommodating device having a minimum total value of costs calculated by the calculating unit as an accommodating device to be a relocation destination of the data. Placement system. The total value of the costs from each of the plurality of opposing devices connected to any of the plurality of accommodation devices among the plurality of accommodation devices is calculated, the total value of the costs is calculated, and the total value of the costs is If there is one smallest accommodating device, the accommodating device is determined as the accommodating device that is the relocation destination of the data held in any of the accommodating devices in the data relocation system, and the total of the costs When there are a plurality of storage devices with the smallest value, among the plurality of storage devices, the storage device closest to the opposing device with the largest call volume for the data is the storage device that is the relocation destination of the data. A decision part to decide;
A data holding unit for holding the data in a storage device;
A data rearrangement device comprising: a rearrangement unit that rearranges the data relative to the accommodation device determined by the determination unit when the data is held in the data holding unit. A data relocation method executed in a data relocation system having a plurality of storage devices and a plurality of opposing devices connected to any of the plurality of storage devices,
Each of the plurality of storage devices is
The total value of the cost from each of the plurality of opposing devices is calculated among the plurality of accommodation devices , and when there is one accommodation device having the smallest total value of the costs, the data of the accommodation device is the data If there are a plurality of storage devices which are to be relocation destinations of data held in any of the storage devices in the relocation system and there are a plurality of storage devices having the smallest total value of the costs, A determination step of determining the accommodation device closest to the opposite device having the largest call volume for the data as the accommodation device to which the data is to be relocated ;
A data holding step of holding the data in a storage device;
When the data is held by the data holding step, the data rearrangement method includes: a rearrangement step of rearranging the data with respect to the storage device determined in the determination step.

Images