JP7174372B2 - Data management method, device and program in distributed storage network - Google Patents

Description

The present invention relates to distributed storage network technology for distributing and storing data in a plurality of storages on a network.

In recent years, due to the rapid development of IoT (Internet of Things) technology, a large amount of data generated by IoT devices is flowing into networks. An example of such a large amount of data is 10 fps surveillance camera video data. It is expected that these data will not only be processed in real time, but will also be accumulated and stored in the storage of servers for effective utilization, such as use as learning data for AI, for example. Therefore, a platform called OpenIoT has been proposed for making such accumulated data open for the purpose of enhancing services.

By the way, in the conventional communication architecture, in order to obtain the data stored in the storage, it is necessary to obtain the address information of the server in which the data is stored and to access this server. On the other hand, in recent years, an architecture called Information-Centric Networking (ICN) has been proposed, which uses an identifier (such as a content name) that indicates the data itself, and acquires data from the network by specifying this identifier. (See Non-Patent Document 1).

Such an architecture is based on the idea of accommodating all data in one virtual storage (=virtual storage network). Non-Patent Document 1 proposes a technology called Mobility First.

In MobilityFirst, routers, which are nodes constituting a network, each have a storage. When a data transmission terminal called Publisher inputs data into the network, the data is stored in the storage of any node of the network. When acquiring data, a data requesting terminal called a Subscriber designates the identifier of the data and inquires of the management server about the location information of the data, and acquires the data from the nodes in the network using the acquired location information. . Here, each node on the communication path of data provided to the data requesting terminal in the network relays the data and stores the data as a cache in its own storage. Each node manages the cache stored in its own storage by a cache management algorithm such as LRU (Least Recently Used). With such a configuration, a large number of identical data are distributed and stored in the virtual storage network. Since there is a high possibility that certain data is cached in a node that is close in terms of network to the data requesting terminal that uses the data with high frequency, low delay can be realized.

As for the technique of distributing and storing the same data in a plurality of nodes, a technique called Web proxy cache described in Non-Patent Document 2 is also known. A Web proxy cache constructs a route via explicitly or implicitly specified storages, and places caches in all the storages on the route. This allows the data requesting terminal to acquire data from a server near itself.

Dipankar Raychaudhuri, et al., "MobilityFirst: a robust and trustworthy mobility-centric architecture for the future Internet", ACM SIGMOBILE Mobile Computing and Communications Review, 2012. Nikolaos Laoutaris, 2 others, "Meta Algorithms for Hierarchical Web Caches", [online], [searched on January 8, 2020], Internet<URL: http://cgi.di.uoa.gr/~laoutar /lcdt.pdf>

Although the number and capacity of data to be stored in a virtual storage network is increasing, in the conventional technology, as described above, data caches are stored in all nodes on the data communication path. There is a problem that the utilization efficiency of the storage is low.

That is, in the conventional technology, even data that is not popular and is not expected to be accessed in the future is cached in all nodes on the data communication path. This effectively reduces the area for storing data that is popular and expected to be accessed in the future. Moreover, cache management in each node does not consider the number of identical data stored in the entire virtual storage network for certain data. This data with a large number of storages is considered to be useless data that substantially wastes the storage capacity. Therefore, the presence of this large amount of wasted data effectively reduces the area for storing data that is popular and expected to be accessed in the future. Then there is the problem that a larger storage capacity is required to maintain a cache hit rate for data that is popular and expected to be accessed in the future.

By the way, with the increase in the number and capacity of data stored in the virtual storage network, there are two types of storage: storage with fast response speed but relatively small capacity and high cost, and storage with slow response speed but relatively large capacity. It is being considered to use it together with a storage whose cost is also low. The former corresponds to SSDs (Solid State Drives) and high-speed HDDs (Hard Disk Drives), for example. The latter corresponds to, for example, slow HDDs, tapes, optical discs, and the like. In this specification, the former is called hot storage, and the latter is called cold storage.

When such a storage configuration is adopted in the conventional technology, each node uses the hot storage as a cache and stores data to be deleted from the hot storage in the cold storage by cache management such as LRU. Here, access to data stored in cold storage has a slower response time than access to data stored in hot storage. Therefore, if data is stored in the cold storage of any node, but is not stored in the hot storage of any node, it is necessary to access the cold storage. For this reason, not only is the response time slow, but it is also possible that data cannot be acquired within the allowable time, resulting in financial loss as compensation.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and its object is to provide a data management method, apparatus, and program with good storage utilization efficiency and response speed in a distributed storage network.

In order to achieve the above object, the present invention provides a first storage, a second storage having a response speed lower than that of the first storage, and data to be stored in the first storage and the second storage. A data management method in a distributed storage network in which a plurality of information processing devices are interconnected via a network to form a virtual storage, the method comprising data placement control means for controlling the placement of data in the distributed storage network data arrangement control means of a plurality of information processing devices on a communication path from the user terminal to the information processing device in which the data is stored, in response to access from the user terminal to the plurality of information processing devices on the communication route a cache storage step of dividing an information processing device into a plurality of clusters and storing the data as a cache in a first storage of any one of the information processing devices belonging to the cluster in each cluster; and data of each information processing device. The arrangement control means calculates an evaluation value for each data stored in the first storage based on a predetermined rule, and the same data as the data with the lowest evaluation value is distributed to the other information processing devices in the distributed storage network. If the data with the lowest evaluation value is stored in the first storage, the data with the lowest evaluation value is deleted from the first storage, and the same data as the data with the lowest evaluation value is stored in the first storage of another information processing device in the distributed storage network. and a data arrangement step of moving the data with the lowest evaluation value from the first storage to the second storage or the first storage of another information processing apparatus if it is not stored in the storage. .

Further, the present invention provides a first storage, a second storage whose response speed is lower than that of the first storage, and data for controlling the arrangement of data to be stored in the first storage and the second storage. an information processing device in a distributed storage network in which a virtual storage is formed by interconnecting a plurality of information processing devices, each of which includes an arrangement control means; division processing means for dividing a plurality of information processing devices on a communication path from the user terminal to an information processing device storing the data into a plurality of clusters in response to access to data from the user terminal; determines one of the information processing devices belonging to the cluster to store the data as a cache, and if it determines to store the data in itself, stores the data as a cache in the first storage. allocation destination determination processing means; evaluation value calculation means for calculating an evaluation value based on a predetermined rule for each data stored in the first storage; If the data with the lowest evaluation value is stored in the first storage of another information processing device, the data with the lowest evaluation value is deleted from the first storage, and the same data as the data with the lowest evaluation value is stored in another information processing device in the distributed storage network. data placement processing means for moving the data with the lowest evaluation value from the first storage to a second storage or to a first storage of another information processing device if it is not stored in the first storage of the information processing device; characterized by comprising

According to the present invention, data accessed by a user terminal is divided into a plurality of clusters by the information processing apparatuses on the communication path, not by all the information processing apparatuses on the communication path, and one cluster in each cluster. Since it is stored as a cache in the storage of the information processing device, the utilization efficiency of the storage is improved. Further, according to the present invention, data that exists in only one piece in all first storages of the distributed storage network (hereinafter referred to as "last one data") is cached in the information processing device that stores the data. Even if the management evaluation value is low, it is moved to the first storage of another information processing apparatus under a predetermined condition. This reduces the chances of accessing the second storage, which has a slow response speed, and improves the response speed as a whole.

System configuration diagram of the prerequisite virtual storage network A diagram explaining the outline of the data management method in the virtual storage network Correlation graph of number of caches and cost Flowchart for explaining operations related to data placement Diagram explaining route node clustering Flowchart explaining the operation of the modified LRU Diagram explaining modified LRU Diagram explaining modified LRU Functional block diagram of an information processing apparatus according to an embodiment System configuration diagram of a virtual storage network according to another embodiment Functional block diagram of an information processing apparatus according to an example of another embodiment

A virtual storage network according to one embodiment of the present invention will be described. In the present invention, the goal is to determine a data storage destination that provides optimal cost/convenience performance, and the following four points are used as performance indicators (however, network impact, which will be described later, is a constraint).

(1) Communication cost for data transfer between storages (data transfer communication cost)
This is the communication cost according to the communication distance and size, which is incurred for moving data between physical storages when changing the data arrangement as described later. Note that this data mobile communication cost is also a kind of power cost.

(2) Subscriber data acquisition communication cost (data acquisition communication cost)
This is the communication cost corresponding to the communication distance and size that is incurred for transferring data from the hot storage to the subscriber when the hot storage is accessed. Note that this data acquisition communication cost is also a kind of power cost.

(3) Economic loss when data cannot be obtained within the allowable time (cold penalty)
This is an economic loss assuming that data cannot be obtained within the allowable time when cold storage with a slow response speed is accessed, and that it is assumed to occur as compensation. Note that this cold penalty is a kind of operating cost.

(4) Network impact caused by traffic when data allocation is changed (network impact)
Assuming that the network is shared with other services, this has adverse effects on the network, such as degradation of communication quality of background traffic and insufficient network line capacity.

A system configuration of a virtual storage network, which is a premise in this embodiment, will be described with reference to FIG. FIG. 1 is a system configuration diagram of a premise virtual storage network. The virtual storage network in this embodiment adopts the ICN architecture.

As shown in FIG. 1, network 10 includes a plurality of information processing devices 100 as nodes. Each information processing device 100 comprises a hot storage 210 and a cold storage 220 . In the present invention, data is stored in the hot storage 210 or the cold storage 220 of any one or a plurality of information processing apparatuses 100 among the plurality of information processing apparatuses 100 . That is, the network 10 functions as one virtual storage. Note that the network 10 can be shared with other services.

The information processing device 100 is communicably connected to other information processing devices 100 via network lines (links). The network topology related to the form of connection of the information processing apparatus 100 in the network 10 is irrelevant. The information processing device 100 can be implemented as a router that relays data in the network 10 . A specific configuration example of the information processing apparatus 100 will be described later as an embodiment.

When a data transmission terminal 1 called a publisher stores data in the network 10 , the data is stored in the hot storage 210 of any information processing device 100 on the network 10 .

Within the virtual storage network, data is stored in a plurality of information processing apparatuses 100 by predetermined data placement control processing. Here, the original data stored by the data transmission terminal 1 and the one or more pieces of data (also referred to as replicas) replicated as a cache in the data allocation control process are the same, and the two are indistinguishable. Data stored in the information processing apparatus 100 can be moved to another information processing apparatus 100 by a predetermined data arrangement control process. Correspondence information as to which information processing apparatus 100 in the network 10 stores data is managed by a management server 3 called GNRS (Global Name Resolution Service).

In order for the data requesting terminal 2 called a Subscriber to acquire data, first, it specifies the identifier (ID) of the data and inquires of the management server 3 about the location information (locations) of the data. The management server 3 replies with the location information of the data. The data requesting terminal 2 retrieves desired data by accessing the network 10 with reference to the location information. Here, when data is stored in a plurality of information processing apparatuses 100 within a virtual storage network, the management server 3 replies to the data request terminal 2 with location information about each data. The data requesting terminal 2 can select arbitrary position information from a plurality of position information and obtain data by referring to the position information. For example, the data requesting terminal 2 can acquire data from the information processing device 100 with the closest network distance.

Next, the outline of the present invention will be described with reference to FIG. FIG. 2 is a diagram for explaining the outline of the data management method in the virtual storage network according to the present invention. In each figure, a subscript is added to the end of the reference numeral to individually identify a plurality of identical constituent elements, such as an information processing device. Reference signs with subscripts removed shall be used. Also, the last one data among the data stored in the hot storage is hatched.

With conventional technology, when accessed from a distance, a large amount of cache is created on the path. do not have. The present invention pays attention to this point, and (1) does not put caches in the storage of all nodes on the path, but spreads the nodes on the path according to the access frequency, etc. (2) Last one data is a virtual storage network It has two features: it manages the hot storage as a whole, manages other caches in each storage, and makes it difficult for raster one data to be demoted to cold storage.

The aforementioned characteristic point (1) will be described with reference to FIG. 2 as an example. Here, it is assumed that the data request terminal 2 acquires data stored in the hot storage 210a of the information processing device 100a. The data to be acquired may be last-one data or other data. A plurality of information processing devices 100b to 100e exist on the communication path between the data request terminal 2 and the information processing device 100a.

In the example of FIG. 2, when the data requested by the data requesting terminal 2 stored in the hot storage 210a of the information processing device 100a is relayed by the information processing devices 100b to 100e, all the hot storages 210b on the route 210e are treated as two virtual storages, and the data is stored as a cache in each virtual storage. In other words, the plurality of information processing apparatuses 100b to 100e on the communication path are divided into a plurality of clusters 400a and 400b, and the information processing apparatuses 100b to 100d belonging to the clusters 400a and 400b are divided into the clusters 400a and 400b. The data is stored as a cache in one of the hot storages 210b or 210c and the hot storage 210d or 210e.

The characteristic point (2) described above will be described with reference to FIG. 2 as an example. In each information processing apparatus 100, all data stored in the hot storage 210 are held in a list structure arranged in descending order based on evaluation values regarding usefulness, which will be described later. Then, in each information processing apparatus 100, the last one data is prioritized over other data having the same data. Last-one data is demoted to the cold storage 220 if it is expelled from its own hot storage 210 and cannot be stored in another hot storage 210 .

The features of the present invention as described above and the differences between the conventional techniques will be described. MobilityFirst does not distinguish between hot storage and cold storage, and emphasizes "how close the cache is to the user." On the other hand, the virtual storage network premised in the present invention has hot storage and cold storage, and it is important to consider in which storage the last-one data resides, which is significantly different in cost and convenience. In other words, since the last-one data cannot be easily demoted to cold storage, it will be costly if a new cache is installed as easily as in the conventional technology. The correlation between the number of caches in hot storage and the cost roughly exhibits a curve as shown in FIG. The present invention aims at the vicinity of the bottom of the curve in FIG. It should be noted that the cost means not only the expense cost but also the total cost including communication cost, power cost, and the like.

Next, the features of the present invention described above will be described in more detail. First, route node clustering, which is a specific example of feature point (1), will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 is a flow chart for explaining operations related to data arrangement, and FIG. 5 is a diagram for explaining on-route node clustering.

In the present invention, _first , the number of clusters is determined according to the frequency of access to certain data _xn (x1, ... _xn , ... _xN is the same data stored in the hot storage 210 in the virtual storage network) (Fig. 4 step S1). A calculation example of the number of clusters is the following formula (1).

1/(ω ₁ * frequency of access to entire data x + ω ₂ * frequency of access to data x _n ) Equation (1)
where ω ₁ and ω ₂ are constants. In the example of FIG. 5, the number of clusters is two.

Next, cluster division points are set in ascending order of the link bandwidth between the information processing apparatuses 100, and division is performed by the determined number of clusters (step S2 in FIG. 4). In the example of FIG. 5, the link band between the information processing device 100c and the information processing device 100d is assumed to be smaller than the link band between the other information processing devices 100. In the example of FIG. In this case, the division point is between the information processing device 100c and the information processing device 100d. As a result, a cluster 400a consisting of the information processing devices 100b and 100c and a cluster 400b consisting of the information processing devices 100d and 100d are set.

Next, in each cluster 400, the evaluation value P1(x) is calculated for each information processing apparatus 100 belonging to the cluster 400, and the hot storage 210 of the information processing apparatus 100 having the maximum evaluation value P1(x) is calculated. (step S3 in FIG. 4). The evaluation value P1(x) can be calculated based on various attribute information of the data. For example, the evaluation value P1(x) is calculated by the following equation (2).

P1(x)=ω3* ₍ access frequency)/( _average access distance from users)+ω4*remaining storage capacity Equation (2)
where ω3 and _{ω4 are} constants. In the example of FIG. 5, data for the cluster 400a is stored in the hot storage 210c of the information processing device 100c, and data for the cluster 400b is stored in the hot storage 210e of the information processing device 100e.

Next, a modified LRU, which is a specific example of feature (2), will be described with reference to FIGS. 6 to 8. FIG. FIG. 6 is a flowchart for explaining the operation of the modified LRU, and FIGS. 7 and 8 are diagrams for explaining the modified LRU.

In the modified URL according to the present invention, the data stored in the hot storage 210 is held in a list structure arranged in descending order based on the evaluation value P2(x) regarding usefulness. This evaluation value P2(x) is a value that is sequentially updated, and means a point of data x for determining deletion, forced movement, or the like. The evaluation value P2(x) can be calculated based on various attribute information of the data. For example, the evaluation value P2(x) is calculated by the following equation.

P2(x)= _ω5 /(number of identical data)+ω6*(access frequency)/(average access distance from user) _−ω7 (current access time ₋ last access time) Equation (3)
where ω ₅ , ω ₆ and ω ₇ are constants.

The information processing apparatus 100 calculates an evaluation value P2(x) when storing data new to the information processing apparatus 100 in the hot storage 210, and performs hot data processing so that the order is based on the evaluation value P2(x). It saves in the storage 210 (steps S11 and S13 in FIG. 6, FIG. 7(a)). That is, in order to insert new data in the middle of the modified LRU, the pointers at the appropriate positions in the list structure are spliced according to the usefulness of the new data. Here, if the hot storage 210 does not have enough free space for storing new data, free space is secured by the following algorithm (steps S12, S14 to S18 in FIG. 6).

If the data with the lowest evaluation value P2(x) among the data stored in the hot storage 210 is not the last one data, the information processing apparatus 100 stores the data in the hot storage of another information processing apparatus 100. Therefore, it deletes it from its own hot storage 210 (steps S14 and S15 in FIG. 6, FIG. 7(b)).

On the other hand, if the data with the lowest evaluation value P2(x) is the last-one data, the information processing apparatus 100 determines whether to forcibly migrate the entire hot storage 210 in the virtual storage network (see FIG. 6). step S16). The evaluation value P2(x) in each information processing apparatus 100 can be used to determine whether or not to forcibly move. Specifically, if data with an evaluation value P2(x) lower than the evaluation value P2(x) of the last-one data is stored in another information processing apparatus 100, the data is forcibly moved to the other information processing apparatus 100. Then judge. When the information processing apparatus 100 determines to forcibly move, the information processing apparatus 100 moves the last-one data to another information processing apparatus 100 (steps S16 and S18 in FIG. 6, FIG. 7(d)). Here, in the other information processing apparatus 100 of the moving destination, the data with the evaluation value P2(x) lower than the evaluation value P2(x) of the last one data is expelled to the cold storage 220 of the information processing apparatus 100 concerned. On the other hand, if it is determined not to forcibly move, the information processing apparatus 100 moves the last-one data to its own cold storage 220 (steps S16 and S7 in FIG. 6, FIG. 7(c)). Note that moving data from the hot storage 210 to the cold storage 220 is also called "demoting" or "archiving."

Each parameter necessary for calculating the number of cluster divisions, calculating the evaluation value P1(x) used in determining the data allocation destination in the cluster, and calculating the evaluation value P2(x) regarding the usefulness of the data in the hot storage 210 is , if it is owned by itself, in other words, if it is closed in its own node, the parameter is used. If each parameter is not closed to its own node, it is acquired by exchanging information between each node. The management server 3 manages which information processing apparatus 100 stores the data. Therefore, for example, in order to acquire the access frequency of data in the other hot storage 210, the location information of the data is first acquired by the management server 3, and based on this location information, the information processing device 100 storing the data is Please inquire.

An embodiment of the information processing apparatus 100 will be described with reference to FIG. FIG. 9 is a functional block diagram of the information processing device according to the embodiment.

The information processing apparatus 100 includes a data storage unit 200 having a hot storage 210 and a cold storage 220 and a data allocation control unit 300 .

The data storage unit 200 has a function of relaying data from another information processing device 100 to another information processing device 100 . The data storage unit 200 also has a function of transmitting the data to the data requesting terminal 2 when the data requesting terminal 2 requests access to data stored in its own hot storage 210 or cold storage 220 . The data storage unit 200 also has a function of storing relayed data in the hot storage 210 according to an instruction from the data placement control unit 300 .

The hot storage 210 is a nonvolatile storage medium that has a fast response speed, a relatively small capacity, and a high cost. Examples of the hot storage 210 include SSDs and high-speed HDDs. The cold storage 220 is a non-volatile storage medium with a slow response speed, relatively large capacity, and low cost. The cold storage 220 corresponds to, for example, a low-speed HDD, tape, or optical disk. Note that the cold storage 220 may have a configuration in which media can be automatically or manually detachable.

The data placement control unit 300 includes an access unevenness evaluation unit 310 , a modified LRU-based utility evaluation unit 320 , and a control operation execution unit 330 . The control operation execution unit 330 includes a cache allocation operation execution unit 331 , a forced migration operation execution unit 332 , a deletion operation execution unit 333 and an archive operation execution unit 334 .

The access maldistribution evaluation unit 310 acquires and evaluates access information regarding access from the data request terminal 2 to each data saved in the virtual storage network. The access information is collected by itself for access to its own information processing apparatus 100 and is collected from the other information processing apparatus 100 for access to another information processing apparatus 100 .

The modified LRU-based utility evaluation unit 320 calculates the evaluation value P2(x) regarding utility described above.

The cache placement operation execution unit 331 performs cache operation and management of data in the data storage unit 200, and has the functions of the features (1) and (2) described above. The cache placement operation execution unit 331 includes a cluster division unit 331a and an intra-cluster placement destination storage determination unit 331b.

The cluster dividing unit 331a performs processing for dividing the information processing apparatuses 100 on the route into a plurality of clusters 400 as described above. The intra-cluster placement destination storage determination unit 331b determines the information processing apparatus 100 in which data is to be placed in each cluster 400 divided as described above.

The forced migration operation execution unit 332 executes a process of forcibly migrating data from its own hot storage 210 to the hot storage 210 of another information processing apparatus 100 . The deletion operation execution unit 333 executes processing for deleting data from its own hot storage 210 . The archive operation execution unit 334 executes processing for moving data from its own hot storage 210 to the cold storage 220 .

The information processing device 100 is a well-known information processing device including a main processing unit, memory, network interface, and the like. Here, the implementation form of the information processing apparatus 100 does not matter. For example, each part of the information processing device 100 may be distributed and implemented by a plurality of devices. Also, each part of the information processing apparatus 100 may be implemented by installing a program in a general-purpose computer, may be implemented as dedicated hardware, or may be combined.

As described above, according to the present invention, the data accessed by the data requesting terminal 2 is distributed to the plurality of clusters 400 by the information processing apparatuses 100 on the communication path, not by all the information processing apparatuses 100 on the communication path. Since it is divided and stored as a cache in the hot storage 210 of one information processing apparatus 100 in each cluster 400, storage utilization efficiency is improved. Further, according to the present invention, even if the cache management evaluation value of the last-one data is low in the information processing apparatus 100 that stores the data, it can be transferred to another information processing apparatus 100 under a predetermined condition. Move to hot storage 210 . This reduces the chances of accessing the cold storage 220, which has a slow response speed, and improves the response speed as a whole.

Although one embodiment of the present invention has been described in detail above, the present invention is not limited to the above-described embodiment, and various improvements and modifications may be made without departing from the gist of the present invention. .

For example, in the above-described embodiment, information necessary for on-route node clustering and modified LRU processing was obtained from the own information processing device 100 or another information processing device 100, but such information is obtained from other information processing devices. Each information processing apparatus 100 may be configured to acquire the information from the information management apparatus while performing overall management by the management apparatus. In addition, part of the processes in on-route node clustering and modified LRU may be performed by another information management device. In this case, the information necessary for the partial processing may be acquired from each information processing apparatus 100 . The information management apparatus 10 may be installed at any position as long as it can communicate with each information processing apparatus 100, regardless of the network position. An example of such an embodiment will be described with reference to FIGS. 10 and 11. FIG.

In this embodiment, as shown in FIG. 10, a forced migration candidate determination device 5 corresponding to the information management device described above is arranged so as to be communicable with each information processing device 100 . The forced migration candidate determination device 5 comprehensively manages the ranking of the forced migration candidates in the hot storage 210 of each information processing device 100. It has a function to manage the points of data that are likely to be stolen. Each information processing device 100 can determine whether or not to forcibly migrate data to another information processing device 100 by inquiring of the forced migration candidate determination device 5 . FIG. 11 shows an example of an information processing apparatus 100 in such an embodiment.

Further, in each of the above embodiments, for example, the calculation of the number of cluster divisions, the calculation of the evaluation value P1(x) used in determining the data allocation destination in the cluster, and the evaluation value regarding the usefulness of the data in the hot storage 210 The formula used for calculating P2(x) is an example, and other formulas and parameters can be used. Parameters include, for example, link bandwidth, link utilization rate and QoS, distance (distance) from main path to storage, storage capacity, number of users in storage location, past history, history of making replicas before , Age of Information size, increase/decrease rate of number of replicas, number of Publishers connected to storage, throughput of original data stored in storage, data size, number of topology branches (number of adjacent nodes), cumulative (past) access count , estimated (future) number of accesses, access frequency, most recent access interval, ratio of redundantly arranged data to storage capacity, whether it is the latest data of the source publisher (whether AoI is less than the data generation interval) Various data attributes such as link status, data characteristics, cache location destination storage status, network topology, etc., such as where the same data is currently located when the data to be used is redundantly allocated data, can be used arbitrarily. can.

DESCRIPTION OF SYMBOLS 1... Data transmission terminal 2... Data request terminal 3... Management server 10... Network 100... Information processing apparatus 210... Hot storage 220... Cold storage 300... Data arrangement control part 400... Cluster

Claims (6)

a first storage; a second storage having a response speed slower than that of the first storage; and data placement control means for controlling the placement of data stored in the first storage and the second storage. A data management method in a distributed storage network in which a plurality of information processing devices are connected to each other via a network to form a virtual storage,
In response to access from a user terminal to data in a distributed storage network, data arrangement control means of a plurality of information processing devices on a communication path from the user terminal to the information processing device in which the data is stored performs the communication. a cache storage step of dividing a plurality of information processing apparatuses on a path into a plurality of clusters and storing the data as a cache in a first storage of any one of the information processing apparatuses belonging to the cluster in each cluster;
The data arrangement control means of each information processing device calculates an evaluation value for each data stored in the first storage based on a predetermined rule, and the same data as the data with the lowest evaluation value is distributed to other data in the distributed storage network. is stored in the first storage of the information processing device, the data with the lowest evaluation value is deleted from the first storage, and the same data as the data with the lowest evaluation value is stored in other information in the distributed storage network and a data placement step of moving the data with the lowest evaluation value from the first storage to a second storage or the first storage of another information processing device if it is not stored in the first storage of the processing device. A data management method in a distributed storage network, characterized by: The cache storage step comprises: calculating the division number of the cluster; and calculating the division position of the cluster based on link information between a plurality of information processing apparatuses on the communication path and the division number. The data management method in a distributed storage network of claim 1, comprising: In the data arrangement step, when data having an evaluation value lower than the lowest evaluation value is stored in a first storage of another information processing device, the data having the lowest evaluation value is transferred from the first storage to the other information processing device. If data with an evaluation value lower than the lowest evaluation value is not stored in the first storage of another information processing device, the data with the lowest evaluation value is transferred to the first storage of the information processing device. 3. The method of managing data in a distributed storage network according to claim 1 or 2, further comprising the step of moving the data from the first storage to the second storage. the data allocation control means, when data is moved from another information processing device, moving data with an evaluation value lower than the evaluation value of the data from the first storage to the second storage. 4. A data management method in a distributed storage network according to claim 3. a first storage; a second storage having a response speed slower than that of the first storage; and data placement control means for controlling the placement of data stored in the first storage and the second storage. An information processing device in a distributed storage network in which a plurality of information processing devices are interconnected via a network to form a virtual storage,
The data arrangement control means is
Division processing for dividing a plurality of information processing devices on a communication path from the user terminal to the information processing device storing the data into a plurality of clusters in response to access from the user terminal to data in the distributed storage network. means and
For each cluster, one of the information processing devices belonging to the cluster is determined to store the data as a cache, and when it is determined to store the data in itself, the data is stored in a first storage as a cache. placement destination determination processing means for storing;
Evaluation value calculation means for calculating an evaluation value based on a predetermined rule for each data stored in the first storage;
If the same data as the data with the lowest evaluation value is stored in the first storage of another information processing device in the distributed storage network, the data with the lowest evaluation value is deleted from the first storage, and the data with the lowest evaluation value is deleted. If the same data as the value data is not stored in the first storage of another information processing device in the distributed storage network, the data of the lowest evaluation value is transferred from the first storage to the second storage or other information An information processing device in a distributed storage network, comprising data arrangement processing means for moving data to a first storage of the processing device. A program that causes a computer to function as the data arrangement control means according to claim 5.

Images