JP7385214B2 - Data management system and data management method - Google Patents

Description

The present invention relates to a data management system and a data management method for controlling data arrangement in distributed storage.

BACKGROUND ART Conventionally, cache control technology is known as a technology for distributing data in a data storage location convenient for a terminal or application (hereinafter referred to as a "user terminal") that accesses the data. The characteristics of the location (convenient location) selected as a data storage location include, for example, a location with good QoS (Quality of Service) defined by the presence or absence of delay, and a location with low data transfer energy due to physical proximity. Can be mentioned.

Cache control technology provides a function to temporarily store data used by a user terminal near the user terminal, assuming the locality (often accessed from the same location) characteristic of the user terminal. This temporarily stored data is called cache data and is a temporary copy of the original data. On the other hand, storage capacity is limited, and cache data that is considered unnecessary is deleted. LRU (Least Recent Used) and TLRU (Time aware Least Recent Used), which are called replacement algorithms, are known as algorithms that are determined to be unnecessary (see Non-Patent Documents 1 and 2).

LRU is an algorithm in which the last part of a data string arranged in order from the most recently accessed data is to be deleted. Further, TLRU is LRU that takes into consideration TTL (Time To Live) and TTU (Time To Use). The TTL indicates the limit of the time that data can exist in the cache, and is the expiration date of the data. Since the original data may change over time after the cache data is generated, the validity of the data is guaranteed by setting the TTL. Furthermore, TTU indicates the limit of time that data can survive from the time it is generated, and is the so-called expiration date of data.

Kai Cheng et al., “LRU-SP: A Size-Adjusted and Popularity-Aware LRU Replacement Algorithm for Web Caching,” Proceedings 24th Annual International Computer Software and Applications Conference. COMPSAC2000, IEEE, October 2000. Giovanni Neglia et al., “Access-time aware cache algorithms”, 2016 28th International Teletraffic Congress (ITC 28), IEEE, September 2016.

In the conventional technology described above, for example, data is stored in the order in which it was input in a storage device, and when the storage capacity becomes full, the data at the beginning of the data string (old data) is deleted, and the data that is newly input is stored. Data is stored at the end of the data string. Furthermore, when data in the storage device is accessed, that data is moved to the end. In other words, new data and data that was last accessed recently are treated as important data. However, these indicators are not sufficient to measure the importance of data.
Furthermore, with the above-mentioned conventional technology, if the storage device is full, data can only be deleted from the storage device, which causes inconvenience, for example, when there is data that cannot be deleted even if it is old.

The present invention has been made in view of these circumstances, and an object of the present invention is to store data without deleting it even when the storage capacity of a storage device is insufficient.

A data management system according to the present invention is a data management system that includes a data management device that is a node in a storage network that includes a plurality of nodes that function as storage devices, and the data management system includes a data management device that is a node that functions as a storage device. device, and a forced migration target data selection device and a forced migration destination selection device that are connected to the data management device as its own node, and the data management device has hot storage and cold storage for storing data. and a status monitoring unit that acquires status information including the number of accesses to the data stored in the hot storage and the cold storage, and the forced migration target data selection device is configured such that the data management device , upon receiving data to be stored in the storage unit, the number of future accesses predicted based on the number of accesses up to the present time for the data and the data to be stored in the hot storage is determined such that the larger the number of accesses, the higher the value. A usefulness evaluation value, which is an evaluation value, is calculated, and the data with the smallest usefulness evaluation value is selected as data to be forcibly moved, and the forced movement destination selection device is configured to obtain the information obtained by the state monitoring unit of the data management device. Using the information of the minimum value of the usefulness evaluation value of the data held in the hot storage of each of the other nodes other than the own node, the data management device selects another node holding data with a minimum gender evaluation value as a forced migration destination node of the forced migration target data, and the data management device moves the forced migration target data to the forced migration destination node; If there is no free space in the hot storage, archiving, which is a process of moving the data stored in the hot storage to the cold storage, is performed.

According to the present invention, even if the storage capacity of a storage device is insufficient, data can be stored without being deleted.

FIG. 1 is a diagram showing the configuration (tree structure) of a storage network according to the present embodiment. FIG. 2 is a diagram for explaining an overview of spontaneous movement in a storage network configuration (linear connection) according to the present embodiment. 1 is a diagram showing the overall configuration of a storage network including a data management system according to the present embodiment. FIG. 2 is a diagram for explaining details of the functions of a spontaneous movement processing unit and the functions of each device related to spontaneous movement according to the present embodiment. FIG. 3 is a diagram for explaining access unevenness in this embodiment. FIG. 3 is a diagram for explaining details of the functions of a forced movement processing unit and the functions of each device related to forced movement according to the present embodiment. FIG. 3 is a diagram for explaining forced movement processing in this embodiment. It is a diagram showing the overall configuration of a modified example of the data management system according to the present embodiment. It is a diagram showing the overall configuration of a modified example of the data management system according to the present embodiment. It is a flowchart showing the flow of data arrangement control by spontaneous movement of the data management system according to the present embodiment. It is a flowchart which shows the flow of the optimal node selection process of the data management system based on this embodiment. It is a flowchart showing the flow of data arrangement control by forced migration of the data management system according to the present embodiment. It is a flowchart which shows the flow of data generation interval adjustment of the data management system concerning this embodiment. 1 is a hardware configuration diagram showing an example of a computer that implements the functions of the data management device and each device according to the present embodiment; FIG. It is a figure showing the example of composition of the data management device which is a modification of the data management system concerning this embodiment.

Next, a mode for carrying out the present invention (hereinafter referred to as "this embodiment") will be described. First, an overview of the present invention will be explained.

<Summary>
Today, the Internet of Things (IoT) continues to expand rapidly, and a huge number of diverse devices are being connected to networks. The vast amount of data generated by these devices is expected to be utilized. For example, many surveillance cameras are already installed on streets and stores, recording daily footage. In addition, operational data of factories, etc. is collected using sensor devices and the like. These large amounts of data are used as big data for various services and data analysis.

There are several ways to manage this large amount of data, especially real-time data (live data): (1) management at the data center, (2) management using a storage network, that is, distributed storage within the network, and (3) data generation ( There are three possible methods: management by the device (Publisher) that collects the information. Among these, (1) management at a data center may cause traffic to concentrate near the aggregation point, leading to network congestion. (3) When managing devices themselves, the data storage capacity of IoT devices is usually small, so even if it is possible to manage static data other than live data (legacy data stored in the device in advance), It is difficult to store large amounts of live data. On the other hand, (2) storage networks maintain data in storage (each node as distributed storage) located at an appropriate location near the user terminal (Subscriber) requesting data acquisition, thereby achieving efficient Data management can be achieved.

In addition, in cases where hot storage, which is conventional storage means for storing live data (for example, HDD (Hard Disk Drive), etc.), is difficult to cope with due to capacity and cost, it is possible to use cold storage and hot storage together. Conceivable. Here, "hot storage" refers to a storage medium that is always in operation and can satisfy real-time requirements, such as a conventional HDD or SSD (Solid State Drive). This hot storage has a fast response speed of, for example, 10 ⁻⁸ to 10 ⁰ (s), but has a limited capacity and is more expensive than cold storage. Furthermore, "cold storage" is a storage medium that cannot satisfy real-time requirements, such as a DVD (Digital Versatile Disc: registered trademark), CD, or BD (Blu-ray Disc: registered trademark). This cold storage has a slow response speed of, for example, 10 ⁰ to 10 ⁴ (s), but has an extremely large capacity (assuming it is infinite if the number of storage media is increased) and is lower in cost than hot storage. In addition, a response to a request for data stored in this cold storage can be achieved by, for example, physically moving a robot arm to retrieve a DVD, etc. stored on a shelf, and setting it in a data reading/writing module, etc. This may be done manually, or by setting a DVD or the like in a data reading/writing module or the like.

By equipping each node of the storage network with hot storage and cold storage, data that does not want to be deleted even if the access time is long is moved from hot storage to cold storage and stored. This allows highly useful data that needs to respond to real-time requests to be stored in hot storage, and less useful data to be stored in cold storage, thereby preventing data deletion. The information can be continuously stored in the memory, increasing user convenience.

In this storage network, for example, as shown in FIG. 1, each node of the storage network is connected in a tree structure. Further, each node may be linearly connected by a link, as shown in FIG. 2, which will be described later. A link connecting nodes (storage nodes) may be composed of multiple switches and multiple transmission paths, or a small-scale network (referred to as an underlay network).
Then, in a storage network to which nodes each having hot storage and cold storage are connected, an appropriate node is selected to place data. Here, an appropriate node is, for example, a location with good QoS (Quality of Service) expressed by the degree of delay (a location with little delay), or a location that is physically close to the user terminal making the data acquisition request. Refers to nodes where the data transfer energy is low, such as.

In conventional data placement control using this storage network, it is assumed that the following control will be executed when hot storage capacity overflows.
(Control 1) Data with low usefulness is moved from hot storage within the node to cold storage (hereinafter referred to as "archive").
(Control 2) Move highly useful data to appropriate hot storage on other nodes.

In the data management system 1000 according to this embodiment (see FIG. 3, which will be described later), further improvements are made to the processes assumed in (Control 1) and (Control 2) above. Data placement is controlled by ``movement'' and ``adjustment of data generation interval according to real-time requests.''

[Voluntary movement]
First, spontaneous movement will be explained.
Spontaneous movement in this embodiment refers to when a node exists in a more optimal location for a user terminal (Subscriber) requesting data acquisition than the current data holding node, in order to reduce communication costs, etc. This refers to the process of spontaneously moving the data to the corresponding node (the "optimal node" to be described later).
The above (control 2) migration to another node is a process of moving data whose performance can be improved by changing the holding node when the hot storage capacity is overflowed. This (control 2) is triggered by the timing when capacity overflow occurs, but in the spontaneous movement of the data management system 1000 (FIG. 3) according to this embodiment, in order to improve performance earlier, capacity overflow is triggered. The optimal node is selected and moved when the target data satisfies a predetermined condition ("spontaneous movement condition" to be described later), regardless of the timing of occurrence.

Whether or not certain data is subject to spontaneous movement is determined based on "usefulness evaluation value" and "ubiquity of access."
The usefulness evaluation value is an evaluation value in which the larger the predicted future number of accesses to the data based on the number of accesses up to the present moment, the higher the value. The higher the usefulness evaluation value, the higher the possibility that the data will be accessed from a user terminal in the future, that is, the data is highly useful.
Furthermore, access unevenness indicates the bias in the number of accesses from adjacent nodes that are access sources to data held by a node. In other words, for certain data, the higher the access unevenness of one node among multiple adjacent nodes, the more that data is being accessed from a single direction. This indicates that the probability that the node) is a demand point node is low. Note that the demand point node is a node that accepts data acquisition requests from user terminals (Subscribers) that it accommodates. If the desired data is held in the demand point node, compared to the case where data is requested and acquired from other nodes, less data transfer energy is required, delays, etc. are reduced, and QoS is improved.

Specifically, as shown in FIG. 2, for example, when each node of the storage network is linearly connected, data "A" is stored in the hot storage of node N ₁₀ (current node), and the data "A" is stored in the hot storage of node N 10 (current node). Suppose that "A" satisfies predetermined spontaneous movement conditions. Here, the node _N50 that receives the acquisition request for data "A" from the user terminal U becomes the demand point node. If the data "A" currently stored in the node has many accesses from the node _N50 side, which is the demand point node _, the access unevenness of the node N20 adjacent to the node _N10 (in the direction of the node _N50 ) is It gets expensive. Therefore, the higher the access unevenness, that is, the more data "A" is accessed from a single direction, the lower the probability that the current node is the demand point node, and the more optimal (closer to the demand point node) node (optimal indicates that the value of moving to node N ₃₀ ) is high.
In FIG. 2, by the optimal node selection process described later, node N ₃₀ which is closer to demand point node N ₅₀ is selected as the optimal node, and data "A" stored in node N ₁₀ is spontaneously moved to node N ₃₀ . An example is shown below.
Details of this optimal node selection process will be described later with reference to FIG. 11.

[Forced movement]
Next, forced migration will be explained.
When a data overflow occurs when trying to store new data in a certain node, as in (control 1) above, among the data stored in that node, data with low usefulness (usefulness evaluation value) can be archived and stored in cold storage. However, if another node holds data with an even lower usability evaluation value, capacity overflow will occur in order to hold data with a higher usability evaluation value in the own node's hot storage. Instead of archiving the data with the lowest usefulness evaluation value of a node, the lowest data with even lower usefulness evaluation values in other nodes should be archived.
In other words, in forced migration, when the hot storage capacity of a certain node overflows, the data with the lowest usability evaluation value of that node will be transferred to the data with the usability evaluation value of all the nodes that make up the storage network (movable nodes described later). If it is not the minimum, the node having data with the minimum usefulness evaluation value is selected as the forced migration destination node among all the nodes (movable nodes). Then, the data with the lowest usefulness evaluation value of the node whose hot storage capacity has overflowed is moved to the selected forced migration destination node.
Details of this forced relocation will be described later.

[Data generation interval adjustment according to real-time requests]
Next, data generation interval adjustment in response to real-time requests will be explained.
A device (Publisher) that generates (collects) data transmits a set of data generated in real time within a predetermined time to a node at predetermined time intervals as one data group. Regarding the device that generates this data group (referred to as "data" in a broad sense), if the real-time demand level (for example, a value proportional to the number of accesses within a predetermined time after data generation) is high, the data By shortening the generation interval, the AoI (Age of Information) of the device can be reduced. Note that AoI means the freshness of information (indicates the elapsed time from the generation time regarding the latest provided data).
Furthermore, regarding this data (data group), there are data that is accessed a lot within a certain period of time and data that is accessed a few times. If the AoI is not considered for this stored data, there is a possibility that storage will be wasted more than necessary. For example, if the AoI is increased in a device that generates data (data group) with a small number of accesses, a large amount of data will be generated as one piece of data, transmitted to a node, and stored in hot storage. On the other hand, if, for example, a certain device generates a data group (data) in a 60-minute cycle, it generates it in a 10-minute cycle, and the data capacity of one piece of data can be reduced. Therefore, the transfer energy (network load) for moving the data can be reduced. As a result, it is possible to reduce the number of links that are excluded from the use of data movement, and increase the number of movable nodes (details will be described later) that are candidates for data placement destinations (optimal nodes) in this embodiment.
Therefore, for the latest data of a device with high real-time demand, the AoI is adjusted to be smaller, that is, the data generation interval is adjusted to be shorter. On the other hand, for the latest data of a device with low real-time demand, the AoI is adjusted to be larger, that is, the data generation interval is adjusted to be longer.

The data management system 1000 and the like according to this embodiment will be described below.

<This embodiment>
FIG. 3 is a diagram showing the overall configuration of a storage network including the data management system 1000 according to this embodiment.
As shown in FIG. 3, the data management system 1000 includes a data management device 1, a voluntary movement condition determination device 21, and a voluntary movement destination selection device 22, which are a group of devices used to realize the above-described “voluntary movement”. and an archive instruction device 23, a forced migration target data selection device 31 and a forced migration destination selection device 32, which are a device group used to realize the above-mentioned "forced migration", and the above-mentioned "data generation interval adjustment". It also includes a generation interval adjustment device 40 used for realizing this.
The data management device 1 is each of the above-mentioned nodes that functions as a storage device in a storage network, and includes a finite capacity storage section 100 made up of hot storage and an infinite capacity storage section 105 made up of cold storage. A plurality of the data management systems 1000 are connected via a network to construct a storage network as a whole.

A group of devices related to spontaneous movement, such as a spontaneous movement condition determination device 21, a spontaneous movement destination selection device 22, and an archive instruction device 23, are provided for each data management device 1, and there is no communication between these devices and within the data management device 1. Communication connection is established.
A forced migration target data selection device 31 and a forced migration destination selection device 32, which are a group of devices related to forced migration, are provided for each data management device 1, and are communicably connected between these devices and to the data management device 1. .
Further, a generation interval adjustment device 40, which is a device related to data generation interval adjustment, is provided for each data management device 1, and is communicatively connected to the data management device 1 and a device (publisher) that generates data.

These voluntary movement condition determination device 21, voluntary movement destination selection device 22, archive instruction device 23, forced movement target data selection device 31, forced movement destination selection device 32, and generation interval adjustment device 40 include a control section, an input/output section, It is constituted by a computer equipped with a storage unit (all not shown). When each device requires the status information described below of each data management device 1 in the processing that it performs, it processes by referring to the status information stored in the storage unit 12 of the data management device 1 connected to itself. Execute. A detailed description of the functions provided by each device will also be described in the detailed description of the data management device 1.

≪Description of the data management device and each device≫
Next, details of the data management device 1 and each device related to "voluntary movement,""forcedmovement," and "data generation interval adjustment" will be explained with reference to FIG. 3 and the like.
The data management device 1 stores data from a communication-connected device (Publisher) P in a hot storage (finite capacity storage unit 100), and also stores data from the hot storage (finite capacity storage unit 100) as needed. Execute archiving to move data to storage (infinite capacity storage unit 105). In addition, the data management device 1 performs the above-mentioned “voluntary movement,” “forced movement,” and “data generation interval adjustment according to real-time requests” in cooperation with the above-mentioned devices, thereby achieving more appropriate data arrangement control. conduct.
This data management device 1 includes a control section 10, an input/output section 11, and a storage section 12.

The input/output unit 11 inputs/outputs information with other data management apparatuses 1, each device that generates data (Publisher), a plurality of user terminals (Subscribers) it accommodates, and the like. In addition, the input/output unit 11 is connected to each device (voluntary movement condition determination device 21, voluntary movement destination selection device 22, archive instruction device 23, forced movement target data selection device 31, forced movement destination selection device 32) that is communicatively connected to the input/output unit 11. , and the generation interval adjustment device 40). The input/output unit 11 includes a communication interface for transmitting and receiving information via a communication line, and an input/output interface for inputting and outputting information between an input device such as a keyboard and an output device such as a monitor (not shown). configured.

The storage unit 12 includes a hard disk, a flash memory, a RAM (Random Access Memory), a data reading/writing module, and the like.
This storage unit 12 (storage means) includes a finite capacity storage unit 100 made up of the hot storage described above and an infinite capacity storage unit 105 made up of the cold storage described above.
In addition, the storage unit 12 further includes information on the access sources (other data management devices 1 and user terminals) and the number of accesses to the data it stores, as well as information on the storage capacity of other data management devices 1 (hot storage (free space, etc.), information on the connection status (for example, link usage) with other data management devices 1 (hereinafter, this information will be collectively referred to as "status information"), etc. are stored. Note that in the network configuration of each node of the storage network described above, if links are used, statistical values such as the total amount of usage and the average amount of usage of each link are used as the usage status (status information) of the links.
The storage unit 12 stores programs for executing each functional unit of the control unit 10 and information (predetermined threshold values, etc.) necessary for the processing of the control unit 10.

The control unit 10 is in charge of overall processing executed by the data management device 1, and as shown in FIG. , and a forced movement processing section 150.

The status monitoring unit 110 monitors the access source for each data of each data management device 1 and the number of times it has been processed (the number of accesses “A” to be described later), the usage status of links between each data management device 1 (link usage amount), and the number of times each data is processed. Information (status information) such as the free capacity of the finite capacity storage unit 100 (hot storage) of the data management device 1 and the data with the minimum usefulness evaluation value is acquired and stored in the storage unit 12.
Note that in the status monitoring unit 110, it is set in advance from which data management device 1 in the storage network the status information etc. can be acquired. For example, when the storage network is configured with a linear connection, it is set so that status information etc. can be acquired from up to N hops (N is a positive integer) of adjacent data management devices 1 (details will be described later). Then, the status monitoring unit 110 acquires (the latest) information of the data with the smallest validity evaluation value among the data stored in the finite capacity storage unit 100 (hot storage) of each adjacent data management device 1. Remember it.
Furthermore, the status monitoring unit 110 similarly measures information on the access source and the number of accesses for the data that it stores in the finite capacity storage unit 100 and the infinite capacity storage unit 105, and stores the information in the storage unit 12.

The acquisition request processing unit 120 receives a data acquisition request from a user terminal (not shown) accommodated in its own data management device 1 or another data management device 1, and stores a finite capacity storage in the storage unit 12. The data stored in the unit 100 (hot storage) or the infinite capacity storage unit 105 (cold storage) is returned as response information.
Further, if the acquisition request is not data stored in its own storage unit 12 but is stored in another data management device 1, the acquisition request processing unit 120 Transfer the acquisition request to 1.
When the acquisition request processing unit 120 receives an acquisition request, the acquisition request processing unit 120 outputs the access source of the data and the number of times the acquisition request has been made (the number of accesses “A”) to the status monitoring unit 110, thereby causing the storage unit 12 to store the information.

The spontaneous movement processing unit 130 calculates the above-mentioned usefulness evaluation value and access unevenness for certain data. Note that the data that is the target of calculating the usefulness evaluation value and access ubiquity (hereinafter referred to as "target data") may be obtained by, for example, receiving a request to acquire certain data. The data may be used as the calculation target, or each piece of data stored in the finite capacity storage unit 100 (hot storage) may be extracted at predetermined time intervals and used as the calculation target.
Then, the spontaneous movement processing unit 130 transmits the calculated usefulness evaluation value and access unevenness to the spontaneous movement condition determination device 21. Then, when the spontaneous movement condition determination device 21 determines that the predetermined spontaneous movement condition based on the usefulness evaluation value and the access omnipresence is satisfied, the spontaneous movement processing unit 130 selects the target data as the subject of spontaneous movement. (hereinafter referred to as "voluntary movement target data").
Furthermore, the spontaneous movement processing unit 130 receives from the spontaneous movement destination selection device 22 the result of selecting the optimum node as the movement destination for the data to be moved spontaneously, and moves the data to the data management device 1 (spontaneous movement).
Hereinafter, the functions of the spontaneous movement processing unit 130 and the functions of each device related to spontaneous movement will be described in detail with reference to FIG. 4 and the like.

FIG. 4 is a diagram for explaining the details of the functions of the spontaneous movement processing unit 130 and the functions of each device related to spontaneous movement according to the present embodiment.
As shown in FIG. 4, the spontaneous movement processing section 130 includes a usefulness evaluation section 131, an access unevenness calculation section 132, a spontaneous movement execution section 133, and an archive execution section 134.

The usefulness evaluation unit 131 evaluates the usefulness of each data (target data) as an evaluation value in which the larger the predicted value of the future number of accesses (from the present time to the future) based on the number of accesses up to the present time, the higher the value. Calculate the value.
Specifically, for example, the usefulness evaluation unit 131 approximates the cumulative number of accesses on the time axis with a function that converges to a certain value, based on the premise that there will be no accesses after a sufficient amount of time has passed, and then calculates the convergence value and Let the difference in the cumulative number of accesses at time t be the usefulness evaluation value.
After calculating the usefulness evaluation value for each data (target data), the usefulness evaluation unit 131 transmits the information to the spontaneous movement condition determination device 21 .

The access uneven distribution calculation unit 132 calculates the access uneven distribution as an indicator of the bias in the number of accesses to the data from nodes adjacent to the node holding the data. This will be explained in detail below.
FIG. 5 is a diagram for explaining the uneven distribution of access in this embodiment.
Here, the explanation will be given assuming that the topology of the storage network is a linear connection of nodes N ₁ , N ₂ , N ₃ , . . . , N ₇ . As a premise, each node (data management device 1) collects status information of nearby nodes and links. Further, it is assumed that the data D _x held by the node N ₄ shown in FIG. 5 has been accessed a total of M times by time t. The breakdown is A ⁱ _t times (Σ ⁷ _i=1 A ⁱ _t =M) from the user terminal accommodated by node N _i . Here, “A ⁱ _t ” times is the number of accesses at node i at time t. However _, it is assumed ^that node N ₄ cannot obtain state information _{, etc.} beyond _two hops, and ^for data _D The node N ₄ obtains information indicating that the access has been made (A ⁶ _t +A ⁷ _t ) times.

In the example shown in FIG. 5, for example,
max(Σ ³ _i=1 A ⁱ _t , A ⁴ _t , Σ ⁷ _i=5 A ⁱ _t )= Σ ³ _i=1 A ⁱ _t
When , data D _x is accessed more often from the direction of node N ₃ . For this reason, the access uneven distribution calculation unit 132
Σ ³ _i=1 A ⁱ _t /M
Let be access omnipresence.
When the access uneven distribution calculation unit 132 calculates the access uneven distribution for each data (target data), the access uneven distribution calculation unit 132 transmits the information to the spontaneous movement condition determination device 21.

Returning to FIG. 4, the spontaneous movement condition determination device 21 satisfies a predetermined spontaneous movement condition based on the usefulness evaluation value calculated by the usefulness evaluation unit 131 and the access unevenness calculated by the access unevenness calculation unit 132. In this case, the data is determined to be data subject to spontaneous movement (voluntary movement target data).

This predetermined spontaneous movement condition is, for example, a condition in which both the usefulness evaluation value and the access ubiquity are evaluated to be sufficiently high. Specifically, the conditions are that the usefulness evaluation value is greater than or equal to a predetermined threshold (predetermined first threshold), and the access unevenness is greater than or equal to a predetermined probability (ratio) (predetermined second threshold). Satisfying this is a condition for spontaneous movement. This predetermined threshold value and predetermined probability (ratio) are set in advance.
The spontaneous movement condition determining device 21 transmits information to the effect that the data is determined to be data subject to spontaneous movement to the spontaneous movement destination selection device 22.

The spontaneous movement destination selection device 22 selects an optimal node (indicating a node to be moved from the node holding the spontaneous movement target data) to another node in a position where the communication cost of the user terminal when accessing data is reduced. (voluntary movement destination node).
This optimal node selection process is executed according to the predetermined procedure shown below.
Note that, as shown in FIG. 5, the description will be made assuming that the data D _x held by the node N ₄ is data to be spontaneously moved.

(Procedure 1) Remove nodes located in directions other than those with high access unevenness from candidate nodes (nodes from which node N ₄ can obtain state information). Note that here, as an example, nodes up to two hops ahead will be described as nodes (candidate nodes) from which the node (data management device 1) that holds the data to be spontaneously moved can obtain state information. Here, the candidate nodes from the point of view of node N ₄ are nodes N ₂ , N ₃ , N ₄ , N ₅ , and N ₆ .
Therefore, in (procedure 1), for example, if the direction with high access unevenness is toward node N ₃ as seen from node N 4 , nodes _{N 5 and} N ₆ are removed from the candidate nodes, and nodes N ₂ and N ₃ are removed from the candidate nodes. , N ₄ candidate nodes are narrowed down.
As a result, candidate nodes located at positions where communication costs increase when a user terminal accesses data are excluded. In other words, candidate nodes are narrowed down to candidates located in positions where communication costs can be reduced.

(Step 2) Select a movable node based on the link usage status to the candidate node.
For example, if the sum of the link usage amount and the size of data D _x at time t is smaller than the upper limit usage amount set in advance for each link, it is determined that the link can be used for data movement.
Specifically, among the links shown in FIG. 5 from which node N ₄ can obtain state information, only link L ₂₃ becomes a candidate when the above sum value exceeds the upper limit usage and does not satisfy the condition. Among the nodes (nodes N ₂ , N ₃ , N ₄ ), nodes N ₃ and N ₄ that do not include link L ₂₃ in the communication path during movement are selected as movable nodes.

(Step 3) A holdable node is selected based on the minimum value of the usefulness evaluation value of data held by the moveable node.
For example, the usefulness evaluation value of data D _x is S _x , and the minimum values of the usefulness evaluation values of data held by nodes N ₃ and N ₄ are S ³ _min and S ⁴ _min (S ³ _min , S ⁴ _min < S _x ), nodes N ₃ and N ₄ are selected as holdable nodes among the moveable nodes.
Note that if the validity evaluation value S _x of the data D _x is the minimum value in the node (node N ₄ ) that holds the data D _x and is also the minimum value among the data held by the movable node, that is, S _x If =S ⁴ _min <S ³ _min , the voluntary destination selection device 22 transmits information to that effect to the archive instruction device 23. Then, the archive instruction device 23 transmits instruction information for archiving the data D _x to the spontaneous movement processing unit 130 (archive execution unit 134) at the node (node N ₄ ) that holds the data D _x . The archive execution unit 134 executes an archive to move the data D _x from hot storage to cold storage.

(Step 4) The sum of the "transfer energy (communication cost) for moving" and "transfer energy (communication cost) for data acquisition" when data _D Select the smallest optimal node.
For example, as a simple model, the amount of transfer energy is calculated as a value proportional to the data size and the number of communication hops. Specifically, in the example shown in _FIG . ₅ , when moving data D When the size of _x is DS _x , it becomes C×DS _x ×1 (C is a proportionality constant).

The transfer energy for data acquisition by the user terminal is calculated by calculating the difference from the case where the data D _x is not moved (the data D _x is held at the node N ₄ ).
[Beyond node N ₂ ]
From the user terminals accommodated by nodes beyond node N ₂ , it is expected that S _x × (A ¹ _t + A ² _t )/M (times) of accesses will be made in the future, and data D _x will be moved to node N ₃ . Since the distance is one hop closer, there is a reduction effect of C×DS _x ×1×(A ¹ _t +A ² _t )/M.
[Node N ₃ ]
From the user terminal accommodated by node N ₃ , it is expected that S _x × A ³ _t /M (times) of accesses will be made in the future, and by moving data D _x to node N ₃ , it will be one hop closer. There is a reduction effect of DS _x ×1 × A ³ _t /M.
[Node N ₄ ]
The user terminal accommodated by node N ₄ is expected to access S _x ×A ⁴ _t /M (times) in the future, and by moving data D _x to node N ₃ , it will move one hop away, so C × There is an increasing effect of DS _x ×1 × A ⁴ _t /M.
[Node N ₅ ]
The user terminal accommodated by node N ₅ is expected to access S _x × A ⁵ _t /M (times) in the future, and by moving data D _x to node N ₃ , it will move one hop away, so C × There is an increasing effect of DS _x ×1 × A ⁵ _t /M.
[Beyond node N ₆ ]
From user terminals accommodated by nodes beyond node N ₆ , it is expected that S _x × (A ⁶ _t + A ⁷ _t )/M (times) of accesses will be made in the future, and data D _x will be moved to node N ₃ . Since the distance is one hop away, there is an increasing effect of C×DS _x ×1×(A ⁶ _t +A ⁷ _t )/M.

Note that if nodes N 3 and N 4 are selected as holdable nodes in (step 3), the spontaneous destination _{selection device 22 selects nodes N 3 and} N ₄ as target nodes in (step ₄ ). becomes.
Then, in (step 4), the spontaneous movement destination selection device 22 calculates the sum of the transfer energy for movement and the transfer energy for data acquisition by the user terminal for each holdable node (here, nodes N ₃ , N ₄ ), and select the node with the minimum sum as the optimal node (voluntary movement destination node).
Furthermore, the spontaneous destination selection device 22 is not limited to executing all of the above-mentioned (Step 1) to (Step 4), but also when executing any one or a combination of these steps. good.
The spontaneous movement destination selection device 22 transmits information on the selected optimal node (voluntary movement destination node: for example, node N ₃ ) to the spontaneous movement processing unit 130 (voluntary movement execution unit 133).

Returning to FIG. 4, the spontaneous movement execution unit 133 transfers the data (here, data _D (Voluntary destination node: For example, node N ₃ ) A process of moving (voluntary movement) is executed.

Note that the archive instruction device 23 transmits instruction information to the data management device 1 to move the data with the lowest usefulness evaluation value among the data stored in the hot storage of the own node to the cold storage of the own node.
Then, as described above, if the validity evaluation value of the data to be processed is the minimum value among all the data held by the above-mentioned movable node, the archive execution unit 134 sends the spontaneous migration destination selection device 22 If it is determined, the instruction information from the archive instruction device 23 is received, and the data is archived from the hot storage (finite capacity storage unit 100) to the cold storage (infinite capacity storage unit 105).

Returning to FIG. 3, the data storage unit 140 receives data (for example, live data) from a device (Publisher) P that generates (collects) data, another data management device 1, etc. via the network. Then, upon receiving the data, the data storage unit 140 checks the capacity of the data. Then, the data storage unit 140 stores the data in the finite capacity storage unit 100 if there is free space in its own finite capacity storage unit 100 (hot storage) and the data can be stored.
On the other hand, if the data storage unit 140 has no free space in its own finite capacity storage unit 100 (hot storage), the data storage unit 140 selects the finite capacity storage unit 100 (hot storage) of any adjacent data management device 1 that can acquire status information. For example, the status monitoring unit 110 determines whether there is free space to store the data in the hot storage, and if there is free space, transfers the data to the data management device 1. do.
Then, if there is no free space in the finite capacity storage units 100 (hot storages) of all the data management devices 1 from which status information can be acquired, the data storage unit 140 forces information to that effect (information on the received data) It is transmitted to the movement target data selection device 31.

Note that the data storage unit 140 selects its own data management device 1 as the optimal node (voluntary movement destination node) by the above-described spontaneous movement destination selection device 22 of another data management system 1000, and the spontaneous movement processing unit 130 Data may also be received as a result of spontaneous movement. In this case, the data storage section 140 processes the spontaneously moved data because the adjacent data management devices 1 from which status information can be acquired are different in each data management device 1. is similarly performed in the data management device 1 after the spontaneous movement.

When the forced movement processing unit 150 receives data (for example, live data) and attempts to store it in the storage unit 12, the forced movement processing unit 150 uses the finite capacity storage units of all data management devices 1 (including itself) from which state information can be obtained. If there is no free space in 100 (hot storage), decide whether to perform forced migration or archiving. Regarding this decision, the forced migration processing unit 150 calculates a predetermined loss (“archive evaluation value” to be described later) when archiving the forced migration target data in its own infinite capacity storage unit 105 (cold storage), and the forced migration target data. This is done by comparing the loss ("forced migration evaluation value" to be described later) when the data is forcibly moved to the forced migration destination node.
Note that the forced migration target data is selected by the forced migration target data selection device 31 from among the data stored in the finite capacity storage unit 100 (hot storage) of its own data management device 1 with the lowest usefulness evaluation value. This is data selected as data to be forcibly moved. Further, the forced migration destination node is a node in which the forced migration destination selection device 32 selects the node (data management device 1) having data with the smallest usefulness evaluation value as the forced migration destination node.
Hereinafter, the functions of the forced movement processing unit 150 and the details of the device group related to forced movement will be explained with reference to FIG. 6 and the like.

FIG. 6 is a diagram for explaining the functions of the forced movement processing unit 150 and details of a group of devices related to forced movement according to the present embodiment.
As shown in FIG. 6, the forced migration processing section 150 includes an archive evaluation section 151, a forced migration evaluation section 152, a process determination section 153, an archive execution section 154, and a forced migration execution section 155.

Here, at time t, it is assumed that there is no free space in the finite capacity storage units 100 (hot storages) of all the data management devices 1 from which state information can be obtained. Further, the minimum data of the usefulness evaluation value of the node N _i is set as data D ⁱ _min . Then, as shown in FIG. 7, which will be described later, the explanation will be made assuming that the node N ₄ has received new data D _new .

First, when the forced migration target data selection device 31 acquires information indicating that new data D _new has been received from the data storage unit 140, it stores it in the limited capacity storage unit 100 (hot storage) of its own data management device 1. Among the data, the data with the smallest usefulness evaluation value (here, data D ⁴ _min ) is selected as the data to be forced to be moved.
The reason why the data with the minimum usefulness evaluation value (here, data D ⁴ _min ) is selected as the data to be forcibly moved is that the lower the usefulness evaluation value, the fewer future accesses, and This is because moving from 1) has little effect.
Note that the forced migration target data selection device 31 uses the usability evaluation unit 131 (see Fig. Similarly to 4), the usefulness evaluation value is calculated as an evaluation value that increases as the predicted value of the number of accesses in the future (from the present moment to the future) increases.
When the forced migration target data selection device 31 selects the forced migration target data, it transmits information about the forced migration target data to the forced migration processing unit 150 (archive evaluation unit 151) and the forced migration destination selection device 32.

The archive evaluation unit 151 calculates the archive evaluation value when the forced migration target data (here, data D ⁴ _min ) is archived in its own infinite capacity storage unit 105 (cold storage) without being moved to another node. calculate.
This archive evaluation value indicates the total amount of economic loss (predetermined loss) due to inability to obtain data in real time after archiving. Here, it is assumed that when archived data is accessed, an economic loss CP occurs as compensation for not being able to obtain data in real time.
For example, when the usefulness evaluation value when archiving the forced migration target data (data D ⁴ _min ) is S ⁴ _min , the archive evaluation unit 151 sets the archive evaluation value to S ⁴ _min ×CP (see FIG. 7). Code a).

The forced migration destination selection device 32 selects, as a forced migration destination node, a node that holds data with the smallest usefulness evaluation value from among the movable nodes for the forced migration target data.
Note that the movable node is the same as the explanation for the voluntary movement destination selection device 22, and among the nodes (candidate nodes) from which the node can acquire status information, the amount of links used to the candidate node and the data to be forcibly moved. (data D ⁴ _min ), excluding nodes that use links where the data cannot be moved due to the upper limit usage of each link.
Here, it is assumed that nodes N ₃ , N ₄ , N ₅ , N ₆ are movable nodes for the forced migration target data (data D ⁴ _min ) of node N ₄ , and the data with the minimum usefulness evaluation value (min ( The node N ₅ holding S ⁱ _min |3≦i≦6)=S ⁵ _min is selected as the forced migration destination node.
When the forced migration destination selection device 32 selects a forced migration destination node, it transmits information about the forced migration destination node to the forced migration processing unit 150 (forced migration evaluation unit 152).

The forced migration evaluation unit 152 calculates the sum of "data transfer energy (communication cost) due to forced migration" and "archive evaluation value of data archived at the forced migration destination node" as a forced migration evaluation value.
Specifically, the transfer energy generated by data movement of the forced movement target data (data D ⁴ _min ) is assumed to be TC (=C×DS ⁴ _min ×1) (symbol b in FIG. 7). Note that C is a proportionality constant.
Also, assume that capacity overflow occurs when node N ₅ receives data to be forcibly moved (data D ⁴ _min ) and data D ⁵ _min is archived, and its archive evaluation value is S ⁵ _min × CP (see Figure 7). Code c).
Thereby, the forced movement evaluation unit 152 calculates TC+S ⁵ _min ×CP as a forced movement evaluation value.

Returning to FIG. 6, the process determining unit 153 compares the archive evaluation value of the node that received the new data D _new with the forced migration evaluation value, and selects the smaller one as the processing target.
For example, if S ⁴ _min ×CP < TC+S ⁵ _min ×CP, the processing determining unit 153 determines to archive the data D ⁴ _min in its own infinite capacity storage unit 105 (cold storage).
On the other hand, if S ⁴ _min ×CP > TC+S ⁵ _min ×CP, the processing determining unit 153 determines to forcibly move the data D ⁴ _min to the forced movement destination node (node N ₅ ).

When the processing determining unit 153 determines to execute archiving, the archive execution unit 154 selects the data (data D ⁴ _min ) with the lowest usefulness evaluation value of the node (node N ₄ ) that received the new data D _new . , the archive is moved from the finite capacity storage unit 100 (hot storage) to the infinite capacity storage unit 105 (cold storage).

When the process determining unit 153 determines to perform a forced migration, the forced migration execution unit 155 selects the forced migration target data (data with the lowest usefulness evaluation value of the node (node N ₄ ) that has received the new data D _new ). The data D ⁴ _min ) is moved (forced) to the forced movement destination node (node N ₅ ).

Returning to FIG. 3, the generation interval adjustment device 40 adjusts the data generation interval to be short for a device (Publisher) with a high degree of real-time demand in order to keep the AoI of the latest data of that device small. Furthermore, the generation interval adjustment device 40 adjusts the data generation interval of a device with a low real-time requirement so as to increase the data generation interval in order to reduce the network load.

Here, the state monitoring unit 110 of the data management device 1 acquires information on the number of accesses to each data within a predetermined time after data generation, and stores it in the storage unit 12. Then, the generation interval adjustment device 40 refers to the information on the number of accesses stored in the storage unit 12, and calculates a value proportional to the number of accesses within a predetermined time as the real-time request degree for each data.
The generation interval adjustment device 40 generates instruction information for lengthening or shortening (adjusting) the data generation interval for a device that generates data according to the degree of real-time demand. Then, the generation interval adjustment device 40 adjusts the data generation interval on a device-by-device basis by transmitting the generated instruction information to the device that generates the data.

The data generation interval adjustment process by the generation interval adjustment device 40 is performed, for example, when the data management device 1 stores certain data stored in the finite capacity storage unit 100 (hot storage) or the infinite capacity storage unit 105 (cold storage). This may be performed when an acquisition request for a user terminal is received. Furthermore, the generation interval adjustment device 40 adjusts the data generation interval at predetermined time intervals for devices that generate each data stored in the finite capacity storage unit 100 (hot storage) and the infinite capacity storage unit 105 (cold storage). Adjustments may also be made.

<Modification of the configuration of this embodiment>
In the data management system 1000 according to the present embodiment described above, as shown in FIG. A device comprising a group of devices related to voluntary movement (voluntary movement condition determination device 21, voluntary movement destination selection device 22, archive instruction device 23) and a device group related to forced movement (forced movement target data selection device 31, forced movement destination selection device 32) It was explained as follows.
However, embodiments of the invention are not limited to the data management system 1000 shown in FIG. 3. For example, as shown in a data management system 1000A in FIG. 8, a configuration may be adopted in which the functions related to forced migration (the forced migration processing unit 150 and a group of devices related to forced migration) shown in FIG. 3 are not provided. The data management device 1A of the data management system 1000A is equipped with a finite capacity storage unit 100 (hot storage), an infinite capacity storage unit 105 (cold storage), a spontaneous movement processing unit 130, a group of devices related to spontaneous movement, and so on. Even if the storage capacity of the finite capacity storage unit 100 (hot storage) is insufficient, data can be stored without being deleted. Further, the spontaneous movement processing unit 130 can move data (voluntary movement target data) that satisfies predetermined spontaneous movement conditions based on the usefulness evaluation value and access ubiquity to the optimal node. Therefore, useful data that is likely to be accessed from a user terminal can be stored in the finite capacity storage unit 100 (hot storage) of the optimal node (data management device 1A) that requires less transfer energy.

Furthermore, as shown in a data management system 1000B in FIG. 9, a configuration may be adopted in which the functions related to spontaneous movement (the spontaneous movement processing unit 130 and the group of devices related to spontaneous movement) shown in FIG. 3 are not provided. The data management device 1B of the data management system 1000B is equipped with a finite capacity storage unit 100 (hot storage), an infinite capacity storage unit 105 (cold storage), a forced migration processing unit 150, a group of devices related to forced migration, etc. Even if the storage capacity of the limited capacity storage unit 100 (hot storage) is insufficient when attempting to store data, the data can be stored without being deleted. Furthermore, even if there is no free space in the finite capacity storage unit 100 (hot storage) of each data management device 1B, the forced migration processing unit 150 transfers the data to the finite capacity storage unit 100 (hot storage) of each data management device 1B. Among the stored data, the data with the lowest usefulness evaluation value (data to be forcibly moved) is forcibly moved to another data management device 1B, and the new data is stored in its own limited capacity storage unit 100 (hot storage). Can be memorized.

Furthermore, the data management systems 1000 (FIG. 3), 1000A (FIG. 8), and 1000B (FIG. 9) according to the present embodiment may be configured without the generation interval adjustment device 40.

<Processing of data management device>
Next, the processing executed by the data management system 1000 (see FIG. 3) will be explained. Here, each process related to spontaneous movement, forced movement, and generation interval adjustment will be specifically explained.
Note that similar processing is performed in the data management system 1000A shown in FIG. 8 and the data management system 1000B shown in FIG. 9.

≪Data placement control by spontaneous movement≫
First, data arrangement control by spontaneous movement of the data management system 1000 will be explained.
Note that here, the status monitoring unit 110 (FIG. 3) of the data management device 1 obtains the access source and the number of times of processing (number of accesses) for each data from other adjacent data management devices 1 from which status information can be obtained. It is assumed that information and state information such as the usage amount of each link are acquired in advance.
In addition, the spontaneous movement processing unit 130 (and a group of devices related to spontaneous movement) is configured such that, for example, the acquisition request processing unit 120 receives a data acquisition request (request), and the finite capacity storage unit 100 (hot storage ) may be used as an opportunity to perform the following data arrangement control using spontaneous movement for that data. In addition, the spontaneous movement processing unit 130 executes the following data arrangement process by spontaneous movement at predetermined time intervals for each data stored in the finite capacity storage unit 100 (hot storage) in the storage unit 12. You can. At this time, the spontaneous movement processing unit 130 may perform data arrangement processing by spontaneous movement, for example, in order of the data that has been accessed the most.

FIG. 10 is a flowchart showing the flow of data arrangement control by spontaneous movement of the data management system 1000 according to this embodiment.
First, the usefulness evaluation unit 131 (see FIG. 4) of the data management device 1 (self-movement processing unit 130) calculates a usefulness evaluation value based on the prediction of the future number of accesses to the target data (target data). (Step S10).
Then, the usefulness evaluation unit 131 transmits the calculated usefulness evaluation value to the spontaneous movement condition determination device 21.

Next, the access uneven distribution calculation unit 132 of the spontaneous movement processing unit 130 calculates the access uneven distribution that indicates the bias regarding the adjacent nodes of the access source (step S11).
The access uneven distribution calculation unit 132 transmits the calculated access uneven distribution information to the spontaneous movement condition determination device 21.

Subsequently, the spontaneous movement condition determination device 21 determines whether the target data satisfies predetermined spontaneous movement conditions regarding the usefulness evaluation value and the omnipresence of access (step S12).
The predetermined spontaneous movement condition is, for example, that the usefulness evaluation value is greater than or equal to a predetermined threshold (a predetermined first threshold), and the access ubiquity is greater than or equal to a predetermined probability (a predetermined second threshold). It is a condition.
Here, if the predetermined spontaneous movement condition is not satisfied (step S12→No), the process ends. On the other hand, if the predetermined spontaneous movement condition is satisfied (step S12 → Yes), the spontaneous movement condition determination device 21 determines that the data is data to be subjected to spontaneous movement (voluntary movement target data), and The information is transmitted to the destination selection device 22, and the process proceeds to the next step S13.

In step S13, the spontaneous movement destination selection device 22 selects the optimal node (voluntary movement destination node) to which the spontaneous movement target data is to be moved. Details of this optimal node selection process will be described later (see FIG. 11). Then, the spontaneous movement destination selection device 22 transmits information on the selected optimal node (voluntary movement destination node) to the spontaneous movement processing unit 130 (voluntary movement execution unit 133).

Next, the spontaneous movement execution unit 135 executes a process of moving (spontaneous movement) the data to be moved spontaneously to the data management device 1 that becomes the optimal node (voluntary movement destination node) selected in step S13 (step S14). .

By doing so, the data management device 1 (voluntary movement processing unit 130) of the data management system 1000 can select data (voluntary movement target data) that satisfies predetermined spontaneous movement conditions based on the usefulness evaluation value and access ubiquity. , it can be moved to the optimal node. Therefore, useful data that is likely to be accessed from the user terminal can be stored in the finite capacity storage unit 100 (hot storage) of the optimal node (data management device 1) that requires less transfer energy.

[Optimal node selection process]
FIG. 11 is a flowchart showing the flow of optimal node selection processing of the data management system 1000 according to this embodiment.
This optimal node selection process is a process executed by the spontaneous movement destination selection device 22 in step S13 in the data arrangement control by spontaneous movement shown in FIG.

First, the voluntary movement destination selection device 22 (FIG. 3) identifies a node (candidate node) from which state information can be acquired among the nodes (the data management device 1 connected to itself) that store the voluntary movement target data. Then, the spontaneous destination selection device 22 removes nodes located in directions other than those with high access unevenness (step S131: procedure 1).

Next, the spontaneous movement destination selection device 22 selects a movable node based on the link usage status to the candidate node (step S132: procedure 2).

Subsequently, the spontaneous migration destination selection device 22 refers to the storage unit 12 of the data management device 1 to confirm the minimum value of the usefulness evaluation value of data held by each movable node. Then, when the confirmed usefulness evaluation value is smaller than the usefulness evaluation value of the spontaneous movement target data, the spontaneous movement destination selection device 22 selects a movable node that holds the data of the confirmed usefulness evaluation value. , is selected as a holdable node (step S133: procedure 3).

Then, the spontaneous movement destination selection device 22 calculates the "transfer energy for movement (communication cost)" when moving the data to be spontaneously moved to a holdable node and holds it, and the "transfer energy for data acquisition" by the user terminal. (communication cost)" is selected as the optimal node from among the holdable nodes (step S134: procedure 4).
In this way, the spontaneous movement destination selection device 22 can select the optimal node regarding the spontaneous movement target data.

≪Data placement control by forced movement≫
Next, data arrangement control by forced migration of the data management system 1000 will be explained.
Note that here, the status monitoring unit 110 (FIG. 3) of the data management device 1 checks the link usage status (link usage) between each data management device 1 from other adjacent data management devices 1 from which status information can be acquired. It is assumed that information (status information) such as the free capacity of the finite capacity storage unit 100 (hot storage) of each data management device 1 and the data with the minimum usefulness evaluation value is acquired.
Further, it is assumed that there is no free space in the finite capacity storage units (hot storages) of all the data management devices 1 that can acquire state information.
The forced movement processing unit 150 and the group of devices related to forced movement start processing when the data storage unit 140 receives new data.

FIG. 12 is a flowchart showing the flow of data arrangement control by forced migration in the data management system 1000 according to the present embodiment.
First, upon receiving new data, the data storage unit 140 of the data management device 1 checks the capacity of the data. Then, the data storage unit 140 determines whether or not there is enough free space in its own limited capacity storage unit 100 (hot storage) to store new data (step S20).
Then, if there is free space (step S20→Yes), the data storage unit 140 stores the new data in the limited capacity storage unit 100 (hot storage) (step S21). On the other hand, if there is no free space (step S20→No), the data storage unit 140 transmits information on the new data to the forced migration target data selection device 31, and proceeds to the next step S22.

In step S22, the forced migration target data selection device 31 calculates a usefulness evaluation value for each data stored in the finite capacity storage unit 100 (hot storage) of the data management device 1 connected to itself, and evaluates the usefulness. Select the data with the minimum value as the data to be forcibly moved.
When the forced migration target data selection device 31 selects the forced migration target data, it transmits information about the forced migration target data to the forced migration processing unit 150 (archive evaluation unit 151) and the forced migration destination selection device 32.

Next, the archive evaluation unit 151 calculates an archive evaluation value when the forced migration target data is archived in its own infinite capacity storage unit 105 (cold storage) (step S23).

Subsequently, the forced migration destination selection device 32 selects, as the forced migration destination node, a node that holds data with the smallest usefulness evaluation value from among the movable nodes for the forced migration target data (step S24).
The forced migration destination selection device 32 transmits information on the selected forced migration destination node to the forced migration processing unit 150 (forced migration evaluation unit 152).

Then, the forced migration evaluation unit 152 calculates the "data transfer energy (communication cost) due to forced migration" when the forced migration target data is moved to the forced migration destination node, and the usability evaluation value stored in the forced migration destination node. The sum of the archive evaluation value when the smallest amount of data is archived ("archive evaluation value of data archived at the forced migration destination node") is calculated as the forced migration evaluation value (step S25).

Next, the processing determining unit 153 compares the archive evaluation value of the node that received the new data calculated in step S23 with the forced migration evaluation value calculated in step S25, and selects the smaller one as the processing target. Determine (step S26).
Here, if the archive evaluation value of the node that has received the new data is smaller than the forced migration evaluation value, the process determining unit 153 determines to execute the archive process. On the other hand, if the forced migration evaluation value is smaller than the archive evaluation value of the node that has received the new data, the process determining unit 153 determines to execute the forced migration.
Note that if the archive evaluation value of the node that has received new data and the forced migration evaluation value are the same, the processing determining unit 153 may execute a predetermined process, or may perform a random process. Either process may be determined.

Next, when the processing determining unit 153 determines to execute the archiving process in step S26, the archiving executing unit 154 selects the data (forced migration target data) with the lowest usefulness evaluation value of the node that received the new data. is archived from the finite capacity storage unit 100 (hot storage) to the infinite capacity storage unit 105 (cold storage) (step S27). Then, the process returns to step S20.

On the other hand, when the process determining unit 153 determines to execute forced migration in step S26, the forced migration executing unit 155 selects data (forced migration target data) with the lowest usefulness evaluation value of the node that received the new data. is forcibly moved to the destination node (step S28). Then, the process returns to step S20.

By doing this, the data management device 1 (forced migration processing unit 150) of the data management system 1000 can perform the , among the data stored in the finite capacity storage unit 100 (hot storage) of each data management device 1, the data with the lowest usefulness evaluation value (data to be forced to be moved) is forcibly moved to another data management device 1. , new data can be stored in its own finite capacity storage unit 100 (hot storage).

≪Data generation interval adjustment according to real-time requests≫
Next, data generation interval adjustment in response to a real-time request by the generation interval adjustment device 40 (FIG. 3) of the data management system 1000 will be described.
Here, it is assumed that information on the number of accesses to each data is stored in the storage unit 12 via the status monitoring unit 110.

FIG. 13 is a flowchart showing the flow of data generation interval adjustment in the data management system 1000 according to this embodiment.
First, the acquisition request processing unit 120 of the data management device 1 receives a data acquisition request from a user terminal (not shown) accommodated in its own data management device 1 or another data management device 1, and stores the data. The data stored in the finite capacity storage unit 100 (hot storage) or the infinite capacity storage unit 105 (cold storage) in the unit 12 is returned as response information (step S30).

Next, the generation interval adjustment device 40 calculates a value proportional to the number of accesses within a predetermined time as a real-time request degree for the data for which the acquisition request has been received (step S31).
Note that the generation interval adjustment device 40 may monitor the acquisition request processing unit 120 of the data management device 1 and obtain information about the data for which an acquisition request has been received, or may obtain information that the acquisition request processing unit 120 has returned the data. By setting the information to be sent to the generation interval adjustment device 40, information regarding the data may be acquired.
Subsequently, the generation interval adjustment device 40 generates instruction information for adjusting the data generation interval to be shorter as the real-time request degree is higher, and transmits it to the device (Publisher) that generated the data (step S32).

By doing so, the generation interval adjustment device 40 of the data management system 1000 can adjust the AoI of the latest data of a device with high real-time demand to a smaller value, that is, to shorten the data generation interval. On the other hand, the generation interval adjustment device 40 can adjust the AoI to be larger, that is, to lengthen the data generation interval, for the latest data of a device with a low real-time requirement. Therefore, it is possible to reduce the network load for moving the data.

<Hardware configuration>
The data management device 1 (1A, 1B) and each device (voluntary migration condition determination device 21, voluntary migration destination selection device 22, archive instruction device 23, forced migration target data selection device 31, forced migration destination selection device) according to this embodiment 32, generation interval adjustment device 40) is realized, for example, by a computer 900 having a configuration as shown in FIG.
FIG. 14 is a hardware configuration diagram showing an example of a computer 900 that implements the functions of the data management device 1 (1A, 1B) and each device according to the present embodiment. The computer 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM 903, an HDD (Hard Disk Drive) 904, an input/output I/F (Interface) 905, a communication I/F 906, and a media I/F 907. have

The CPU 901 operates based on a program stored in the ROM 902 or the HDD 904, and is controlled by the control unit 10 (FIGS. 3, 8, 9, etc.). The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 is started, programs related to the hardware of the computer 900, and the like.

The CPU 901 controls an input device 910 such as a mouse and a keyboard, and an output device 911 such as a display and a printer via an input/output I/F 905. The CPU 901 acquires data from the input device 910 via the input/output I/F 905 and outputs the generated data to the output device 911. Note that a GPU (Graphics Processing Unit) or the like may be used in addition to the CPU 901 as the processor.

The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I/F 906 receives data from other devices via a communication network (for example, NW (Network) 920) and outputs it to the CPU 901, and also sends data generated by the CPU 901 to other devices via the communication network. Send to device.

The media I/F 907 (data reading/writing module) reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads a program related to target processing from the recording medium 912 onto the RAM 903 via the media I/F 907, and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto Optical disk), a magnetic recording medium, a semiconductor memory, or the like.

For example, when the computer 900 functions as the data management device 1 (1A, 1B) of the present invention or each of the above devices, the CPU 901 of the computer 900 executes a program loaded on the RAM 903, thereby controlling the data management device 1 ( 1A, 1B) and the functions of each device. Furthermore, data in the RAM 903 is stored in the HDD 904 . The CPU 901 reads a program related to target processing from the recording medium 912 and executes it. In addition, the CPU 901 may read a program related to target processing from another device via a communication network (NW 920).

<Effect>
The effects of the data management system 1000 (1000B) etc. according to the present invention will be explained below.
A data management system according to the present invention is a data management system 1000 (1000B) that includes a data management device 1 (1B) that is a node in a storage network that includes a plurality of nodes that function as storage devices. The system 1000 includes a data management device 1, and a forced migration target data selection device 31 and a forced migration destination selection device 32, which are connected to the data management device 1 as its own node. a storage unit 12 having hot storage and cold storage for storing data, and a status monitoring unit 110 that acquires status information including the number of accesses to data stored in the hot storage and cold storage, and a status monitoring unit 110 for selecting data to be forcibly moved. When the data management device 1 receives data to be stored in the storage unit 12, the device 31 calculates the predicted future number of accesses based on the number of accesses up to the present time for each of the data and the data to be stored in the hot storage. , calculates a usefulness evaluation value, which is an evaluation value in which the larger the value, the higher the value, and selects data with the smallest usefulness evaluation value as data to be forcibly moved. Using the information of the minimum value of the usefulness evaluation value of data held in the hot storage of each node other than the own node, obtained by the state monitoring unit 110 of the node, the minimum value of the usefulness evaluation value of each other node is Among them, the other node that holds the data with the smallest usefulness evaluation value is selected as the forced migration destination node of the forced migration target data, and the data management device 1 performs a process of moving the forced migration target data to the forced migration destination node. If there is no free space in the hot storage, archiving, which is a process of moving the data stored in the hot storage to the cold storage, is performed.

In this way, the data management system 1000 (1000B) includes cold storage as well as hot storage in the storage unit 12 of the data management device 1 (1B), so that when there is no free space in the hot storage, the cold storage is used. Data can be moved and stored.
In addition, the forced migration target data selection device 31 forcibly migrates the data with the lowest usefulness evaluation value indicating that it is likely to be accessed in the future among the data stored in the hot storage of the node that received the data. Can be selected as target data.
Further, the forced migration destination selection device 32 can select, as the forced migration destination node, a node that holds data with the smallest usefulness evaluation value among other nodes.
As a result, the data management device 1 moves (forced migration) target data that will have fewer future accesses and will have less impact even if moved, to the forced migration destination node that holds data with the lowest usefulness evaluation value. ) can be done.

In addition, in the data management system 1000 (1000B), a value proportional to the number of accesses of data stored in hot storage within a predetermined time is calculated as the real-time demand level, and the higher the real-time demand level, the shorter the data generation interval is adjusted. The present invention is characterized in that it further includes a generation interval adjustment device 40 that generates instruction information to be used and outputs the instruction information to a device that generates the data.

By doing so, the generation interval adjustment device 40 of the data management system 1000 (1000B) can reduce the AoI by shortening the data generation interval of devices with high real-time demands. Furthermore, the generation interval adjustment device 40 can reduce the network load when moving data by lengthening the data generation interval for devices with low real-time requirements.

In addition, the data management device is a data management device 1 (1B) of a data management system 1000 (1000B) including a data management device 1 (1B) that is a node in a storage network composed of a plurality of nodes that function as storage devices. The data management system 1000 includes a data management device 1, a forced migration target data selection device 31, and a forced migration destination selection device 32, and the data management device 1 has hot storage and cold storage for storing data. A storage unit 12 having storage, a status monitoring unit 110 that acquires status information including the number of accesses to data stored in hot storage and cold storage, and a forced migration target data selected by a forced migration target data selection device 31, A forced migration execution unit 155 executes forced migration, which is a process of moving to a forced migration destination node selected by the forced migration destination selection device 32, and when there is no free space in the hot storage, the data stored in the hot storage is It is characterized by comprising an archive execution unit 154 that executes archiving, which is a process of moving to cold storage.

In this way, the data management device 1 (1B) includes cold storage as well as hot storage in the storage unit 12, so that when there is no free space in the hot storage, data is moved to and stored in the cold storage. be able to.
Furthermore, the data management device 1 can move the forced migration target data selected by the forced migration target data selection device 31 to the forced migration destination node selected by the forced migration destination selection device 32. Therefore, the data management device 1 can move data that will have fewer future accesses and will have less impact even if moved, to a node that holds data with the lowest usefulness evaluation value.

In addition, the forced migration target data selection device selects forced migration target data of a data management system 1000 (1000B) including a data management device 1 (1B) which is a node in a storage network configured of a plurality of nodes functioning as storage devices. The selection device 31, the data management system 1000, acquires status information including the storage unit 12 having hot storage and cold storage for storing data, and the number of accesses to the data stored in the hot storage and cold storage. The data management device 1 has a state monitoring unit 110 that performs data management, and a data selection device 31 for forced migration. When received, for each of the data and the data stored in hot storage, a usefulness evaluation value is calculated, which is an evaluation value in which the predicted future number of accesses based on the number of accesses to date is larger, the higher the value. , the data with the lowest usefulness evaluation value is selected as the data to be forcibly moved.

In this way, the forced migration target data selection device 31 selects the data with the lowest usefulness evaluation value, which indicates that it is likely to be accessed in the future, among the data stored in the hot storage of the node that received the data. Can be selected as data subject to forced movement.
Therefore, the forced migration target data selection device 31 can select, as forced migration target data, data that will have fewer accesses in the future and will have less impact even if moved.

In addition, the forced migration destination selection device is a forced migration destination selection device for a data management system 1000 (1000B) including a data management device 1 (1B) that is a node in a storage network consisting of a plurality of nodes functioning as storage devices. 32, the data management system 1000 has a storage unit 12 having hot storage and cold storage for storing data, and data held in the hot storage of each node other than the data management device 1, which is the own node. , a status monitoring unit 110 that acquires status information including information on the minimum value of the usefulness evaluation value, which is an evaluation value in which the larger the predicted future number of accesses based on the number of accesses up to the present moment, the higher the value. a forced migration target data selection device 31 and a forced migration destination selection device 32 for selecting forced migration target data, and the forced migration destination selection device 32 is a state monitoring unit of the data management device 1 Among the minimum values of the usefulness evaluation values of other nodes obtained by 110, the other node holding the data with the smallest usefulness evaluation value is selected as the forced migration destination node of the data to be forced migration. do.

In this way, the forced migration destination selection device 32 can select, as the forced migration destination node, a node that holds data with the smallest usefulness evaluation value among other nodes.
Therefore, the forced migration destination selection device 32 can move (forced migration) to a node that holds data with a minimum usefulness evaluation value that will have a small number of accesses in the future and will have little impact even if moved to cold storage.

Note that the present invention is not limited to the embodiments described above, and many modifications can be made within the technical idea of the present invention by those having ordinary knowledge in this field.
For example, as shown in the data management device 1C in FIG. 15, a group of devices related to spontaneous movement (voluntary movement condition determination device 21, spontaneous movement destination selection device 22, archive instruction device 23) in the data management system 1000 shown in FIG. , a device group related to forced migration (forced migration target data selection device 31, forced migration destination selection device 32), and the generation interval adjustment device 40 are incorporated as a data management device 1C, and can also be used as a single housing device. good. In this case, the functions of the spontaneous movement condition determination device 21, the spontaneous movement destination selection device 22, and the archive instruction device 23 are incorporated as the spontaneous movement processing section 130C. The functions of the forced migration target data selection device 31 and the forced migration destination selection device 32 are incorporated as a forced migration processing section 150C. Further, the function of the generation interval adjustment device 40 is incorporated into the control section 10 as a generation interval adjustment section 160.
Even in this case, the same effects as the data management system 1000 according to this embodiment can be achieved. Furthermore, the data management device 1C shown in FIG. 15 may be configured without any one of the voluntary movement processing section 130C, the forced movement processing section 150C, and the generation interval adjustment section 160.

1, 1A, 1B Data management device 10 Control section 11 Input/output section 12 Storage section (storage means)
21 Voluntary migration condition determination device 22 Voluntary migration destination selection device 23 Archive instruction device 31 Forced migration target data selection device 32 Forced migration destination selection device 40 Generation interval adjustment device 100 Limited capacity storage section (hot storage)
105 Infinite capacity storage unit (cold storage)
110 Status monitoring unit 120 Acquisition request processing unit 130 Voluntary movement processing unit 131 Usability evaluation unit 132 Access omnipresence calculation unit 133 Voluntary movement execution unit 134 Archive execution unit 140 Data storage unit 150 Forced movement processing unit 151 Archive evaluation unit 152 Forced movement Evaluation unit 153 Processing determination unit 154 Archive execution unit 155 Forced migration execution unit 1000, 1000A, 1000B Data management system

Claims (3)

A data management system including a data management device that is a node in a storage network configured with a plurality of nodes functioning as storage devices, the system comprising:
The data management system includes the data management device, and a forced migration target data selection device and a forced migration destination selection device connected to the data management device as its own node,
The data management device includes:
a storage unit having hot storage and cold storage for storing data;
a status monitoring unit that acquires status information including the number of accesses to data stored in the hot storage and the cold storage,
The forced movement target data selection device includes:
When the data management device receives data to be stored in the storage unit, the number of future accesses predicted based on the number of accesses up to the present time is large for each of the data and the data to be stored in the hot storage. calculate a usefulness evaluation value, which is an evaluation value that becomes a higher value, and select data with the minimum usefulness evaluation value as data to be forcibly moved;
The forced movement destination selection device includes:
Evaluate the usefulness of the other nodes using the information of the minimum value of the usefulness evaluation values of the data held in the hot storage of each of the other nodes other than the own node, which is obtained by the state monitoring unit of the data management device. Selecting another node holding data with the minimum usefulness evaluation value among the minimum values as the forced migration destination node of the forced migration target data,
The data management device includes:
Executing forced migration, which is a process of moving the forced migration target data to the forced migration destination node,
A data management system characterized by executing archiving which is a process of moving data stored in the hot storage to the cold storage when there is no free space in the hot storage. A value proportional to the number of accesses of the data stored in the hot storage within a predetermined time is calculated as the real-time request degree, and the higher the real-time request degree is, the instruction information for adjusting the data generation interval to be shorter is generated, and the data generation interval is adjusted to be shorter. The data management system according to claim 1, further comprising a generation interval adjustment device that outputs the instruction information to a device that generates the instruction information. A data management method for a data management system including a data management device which is a node in a storage network configured with a plurality of nodes functioning as storage devices, the method comprising:
The data management system includes the data management device, and a forced migration target data selection device and a forced migration destination selection device connected to the data management device as its own node,
The data management device includes:
Equipped with a storage unit that has hot storage and cold storage for storing data,
obtaining state information including the number of accesses to data stored in the hot storage and the cold storage;
The forced movement target data selection device includes:
When the data management device receives data to be stored in the storage unit, the number of future accesses predicted based on the number of accesses up to the present time is large for each of the data and the data to be stored in the hot storage. Calculate a usefulness evaluation value, which is an evaluation value that becomes a higher value, and select data with the minimum usefulness evaluation value as forced migration target data,
The forced movement destination selection device includes:
The minimum value of the usefulness evaluation value of the other nodes is determined using the information of the minimum value of the usefulness evaluation value of the data held in the hot storage of each of the other nodes other than the own node, which is obtained by the data management device. Selecting the other node holding the data with the minimum usefulness evaluation value as the forced migration destination node of the forced migration target data,
The data management device includes:
Executing forced migration, which is a process of moving the forced migration target data to the forced migration destination node,
A data management method comprising: executing archiving, which is a process of moving data stored in the hot storage to the cold storage, when there is no free space in the hot storage.

Images