JP7082373B2 - Data management device and data management method - Google Patents

Description

The present invention relates to a data management device and a data management method in a storage network to which a plurality of storage devices are connected.

Conventionally, a cache control technique is known as a technique for deploying data to a data storage location (storage) in a convenient location for a device or application (hereinafter referred to as "user") to access the data. The characteristics of the location (convenient location) selected as the data storage location include, for example, a location with good QoS (Quality of Service) defined by the presence or absence of delay, and a location with low data transfer energy such as being physically close. Can be mentioned.

The cache control technology provides a function to temporarily store the data used by the user near the user, assuming the characteristic of the locality of the user (mostly accessed from the same place). This temporarily stored data is called cache data and is a temporary copy of the original data. On the other hand, the storage capacity is finite, and cache data that seems unnecessary is erased. LRU (Least Recent Used) and TLRU (Time aware Least Recent Used), which are called replacement algorithms, are known as algorithms to be determined to be unnecessary (see Non-Patent Documents 1 to 3).

LRU is an algorithm in which the last part of a data string arranged in order from recently accessed data is deleted. Further, the TLRU is an LRU in consideration of TTL (Time To Live) and TTU (Time To Use). The TTL indicates the limit of time that the data can exist in the cache, so to speak, the expiration date of the data. Since the original data may change over time after the cache data is generated, the correctness of the data is guaranteed by setting the TTL. In addition, TTU indicates the limit of the time that the data can survive from the time when the data is generated, so to speak, the expiration date of the data.

Kai Cheng et al., “LRU-SP: A Size-Adjusted and Popularity-Aware LRU Replacement Algorithm for Web Caching”, Proceedings 24th Annual International Computer Software and Applications Conference. COMPSAC2000, IEEE, October 2000 Giovanni Neglia et al., “Access-time aware cache algorithms”, 2016 28th International Teletraffic Congress (ITC 28), IEEE, September 2016 Zhongxing Ming et al., “Age-based Cooperative Caching in Information-Centric Networks”, 2012 Proceedings IEEE INFOCOM Workshops, IEEE, March 2012

FIG. 7 is an explanatory diagram schematically showing cache control according to the prior art.
The storage device 50 has a capacity α, and data a, b, and c are stored at the time of reference numeral T1 in FIG. Since the storage device 50 has free space at the time of reference numeral T1, the new data d can be stored as it is. The data a, b, and c are stored in the storage device 50 in this order, and the new data d is stored next to the data c.

After that, at the time of reference numeral T2 in FIG. 7, data a, b, c, d, e, f, g, and h are stored in the storage device 50, and the free space is exhausted. In order to store new data i in this state, it is necessary to delete one of the data. In this case, as shown by reference numeral T3 in FIG. 7, the data a at the beginning of the data string is deleted, and the data i is stored at the end.

Further, when the data in the storage device 50 is accessed by the user, the data is moved to the end of the data string. For example, when the data b is accessed by the user with the reference numeral T3 in FIG. 7 (arrow 31 of the reference numeral T3), the data b is moved to the end of the data string as shown by the reference numeral T4 in FIG. When the new data j is stored in this state, the data c, which is the first data, is deleted.

In the above-mentioned prior art, recently stored data or recently accessed data is at the end of the data string (data that is difficult to delete). That is, new data and data with the latest access time are treated as important data. However, these indicators are not sufficient as indicators for measuring the importance of data.
Further, in the above-mentioned conventional technique, when the storage device is full, data can only be deleted from the storage device, which causes inconvenience, for example, when there is data that should not be deleted even if it becomes old.

The present invention has been made in view of such circumstances, and an object of the present invention is to store data without deleting it even when the storage capacity of the storage device is insufficient.

In order to achieve the above object, the invention according to claim 1 is a data management device in a storage network to which a plurality of storage devices are connected, and each of the storage devices has data stored in the storage device. When the save storage to be moved to is connected and the data management device in each of the storage devices runs out of storage capacity of the storage device, at least one of the data stored in the storage device is used. The data is provided with a data moving unit for moving the data to another storage device or the saved storage, and a scoring unit for assigning a score regarding the importance of the data to each data stored in the storage device. The moving unit moves the moving target data to the other storage device when the score of the moving target data is within the first range indicating that the importance is high, and the moving unit moves the moving target data to the score of the moving target data. The data management device is characterized in that the data to be moved is moved to the save storage when the data is in the second range indicating that the importance is low .

The invention according to claim 6 is a data management method for a data management device in a storage network to which a plurality of storage devices are connected, and the data stored in the storage device is moved to each of the storage devices. A scoring step in which the backup storage to be preceded is connected, and the data management device gives a score regarding the importance of the data to each data stored in the storage device in each of the storage devices, and When the storage capacity of the storage device is insufficient, the data movement step of moving at least one of the data stored in the storage device to another storage device or the backup storage is executed , and the data movement is performed. In the step, when the score of the movement target data is within the first range indicating that the importance is high, the movement target data is moved to the other storage device, and the score of the movement target data is When the data is in the second range indicating that the importance is low, the data management method is characterized in that the data to be moved is moved to the saved storage .

By doing so, even if the storage capacity of the storage device is insufficient, the data management device can be moved to another storage device or an evacuation storage without deleting the data.
Further, the data management device can move the high-importance data to another storage device and move the low-importance data to the evacuation storage based on the score indicating the importance of the data. Therefore, the data management device can move the data to an appropriate storage according to the importance of the data without deleting the data.

The invention according to claim 2 is characterized in that the score giving unit calculates the score using a value obtained by subtracting the time when the data is generated from the time when the data is accessed. The data management device according to 1 .

By doing so, the time from the generation of the data to the access can be reflected in the score, and the importance of the data can be reflected in the score more accurately.

2. The invention according to claim 3 is characterized in that the score giving unit calculates the value each time the data is accessed, and calculates the score based on the sum of the values. The data management device described in 1.

By doing so, the number of accesses to the data can be reflected in the score, and the importance of the data can be reflected in the score more accurately.

In the invention according to claim 4 , the data moving unit has data in which the score is within the first range, or the score is in the second range, based on the operating status of the other storage device. The data management device according to any one of claims 1 to 3 , wherein the data in the data is determined as the data to be moved.

By doing so, it is possible to select the data to be moved based on the operating status of the destination, and it is possible to efficiently move the data within the storage network.

In the invention according to claim 5 , the storage network has a hierarchical structure for users who use the data, and the data moving unit obtains data whose score is within the first range. When moving to another storage device, the movement target data is moved to the other storage device located higher than the position of the storage device in the hierarchical structure closer to the user. The data management device according to any one of claims 1 to 3 .

By doing so, it is possible to move highly important data to a storage device close to the user. Therefore, it is possible to improve the access efficiency of data in the storage network.

According to the present invention, even when the storage capacity of the storage device is insufficient, the data can be stored without being deleted.

It is a figure which shows the structure of the storage network which concerns on embodiment. It is a figure which illustrates the detailed configuration of each storage node. It is a figure which illustrates the detailed configuration of each storage node. It is a block diagram which shows the functional structure of a data management apparatus. It is explanatory drawing which shows typically the cache control in a data management apparatus. It is a flowchart which shows the processing procedure of a data management apparatus. It is explanatory drawing which shows typically the cache control which concerns on the prior art.

Hereinafter, preferred embodiments of the data management apparatus and data management method according to the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a diagram showing an example of the configuration of the storage network 10 according to the embodiment. In the storage network 10, a plurality of storage nodes 12 are connected to each other.
The storage network 10 illustrated in FIG. 1 forms a hierarchical structure with respect to the user 14 who uses the data. For example, based on the user 14A, the storage node 12A, the storage node 12B, the storage node 12C, and the storage node 12D are located closer to the user 14 in this order. In the present embodiment, the storage node 12 close to the user 14 is set as the upper level, and the storage node 12 far from the user 14 is set as the lower level. That is, based on the user 14A, the storage node 12A, the storage node 12B, the storage node 12C, and the storage node 12D are arranged in order from the upper storage node.

Each storage node 12 stores cache data (hereinafter, may be simply referred to as "data") that is a copy of data used by various devices and applications that are users 14. For example, when the user 14 accesses the true data β (original data) stored in the predetermined storage, the cache data β'which is a copy (replica data) of the true data β is placed in the storage node 12 closer to the user. Will be done. When the user 14 needs to access the data β from the next time onward, the user 14 accesses the cache data β'.
The configuration of the storage network 10 is not limited to the one having the hierarchical structure illustrated in FIG. For example, each storage node 12 may be connected to each other by communication to form a cloud on the network.

As shown in FIG. 2, for example, each storage node 12 is a hot storage device (hereinafter, simply referred to as a hot storage device) which is a high-speed storage connected to a network via a switch device 120 and stores frequently accessed data (hot data). It is configured to include a cold storage device (hereinafter, referred to as “save storage 100A”) that stores data (cold data) that is infrequently accessed (referred to as “storage device 100”).

The cold storage device (save storage 100A) is a storage device that stores data having a lower score indicating the importance described later as compared with the hot storage device (storage device 100). This cold storage device (save storage 100A) may be connected to the network via the switch device 120 as shown in FIG. 2, or may be locally hot without being connected to the network as shown in FIG. It may be connected to a storage device (storage device 100).

Further, the cold storage device (save storage 100A) is distinguished from the hot storage device by, for example, a predetermined condition as shown below.
Compared to hot storage devices, it takes longer to access (access delay is large), and information (data) is stored in removable media (for example, discs and tapes such as DVD (registered trademark) and Blu-ray (registered trademark)). A storage device satisfying predetermined conditions such as no communication cost is installed in advance by the administrator of the storage network 10 as a cold device (save storage 100A).

Here, each storage device 100 (or each storage node 12) includes a data management device 20 (data management function) that manages the data stored by itself. The data management device 20 may be included as a device having one housing in the storage node 12, or may be realized as having a function (data management function) of the data management device 20 in the storage device 100. You may. Further, the data management device 20 may be configured to include the storage device 100.

FIG. 4 is a block diagram showing a functional configuration of the data management device 20.
The data management device 20 includes a new data receiving unit 202, a data storage unit 204, a capacity monitoring unit 206, a score giving unit 208, and a data moving unit 210.
The new data receiving unit 202 receives new data to be stored in the storage device 100 from another storage device, the user 14, or the like.
The data storage unit 204 stores new data in the storage device 100.
The capacity monitoring unit 206 monitors the free space of the storage device 100.

The scoring unit 208 assigns a score regarding the importance of the data to each data stored in the storage device 100. In addition to the conventional element of "time when data was accessed (more important for recently accessed data)", the score assigning unit 208 "time from data generation to access (data generation). A score indicating the importance of the data is calculated using the elements of "data accessed earlier is more important" and "access count (data with more access is more important)". Then, the score giving unit 208 sorts the data (data string) based on the calculated score.

More specifically, the score giving unit 208 calculates the score of each data using the following formula (1).
Score = Σ {Expiration date set in the data TTU- (Time when the data was accessed-Time when the data was generated)} + Time when the data was last accessed ... (1)

That is, the score giving unit 208 calculates the score using a value obtained by subtracting the time when the data is generated from the time when the data is accessed (hereinafter referred to as “data access timing value”). As a result, the elapsed time (AoI: Age of Information) from the generation of the data to the access can be reflected in the score. Further, the score giving unit 208 calculates the data access timing value every time the data is accessed, and calculates the score based on the sum of the data access timing values. As a result, the access frequency (total amount of access) to the data can be reflected in the score.

A more specific example will be given for explanation.
It is assumed that the storage device 100 holds three data A, B, and C. The generation time of each data is data A: 10, data B: 5, and data C: 100. Here, for convenience of explanation, the time is expressed as, for example, "10", "5", "100" as a numerical value of the elapsed time (for example, "second") from a certain reference time. .. The TTU of each data is 500. In addition, it is assumed that each data has been accessed at the following times. A: 200 (1 access), B: 110,200 (2 accesses), C: 200 (1 access).

Under the above conditions, the score SX of each data (X is the data name (A, B, C)) is as follows.
SA = {500- (200-10)} + 200 = 510
SB = {500- (110-5)} + {500- (200-5)} + 200 = 900
SC = {500- (200-100)} + 200 = 600
Therefore, it can be estimated that the data are important in the order of data B, C, and A in descending order of score. That is, in the present invention, a large score value is given to the data accessed at a stage where AoI is small, such as data B, or the data accessed a plurality of times, and it is judged that the importance is high.
The score calculation method by the score giving unit 208 is not limited to using the equation (1), and may be any method that can reflect the data access timing (data access timing value) and the number of times the data is accessed. ..

For comparison, the score calculated by the method related to the conventional technique is also examined.
In the conventional TLRU, the score is calculated by, for example, the following equation (2).
Score = {(data generation time + TTU) -current time} + last accessed time ... (2)
In the above equation (2), {(data generation time + TTU) -current time} corresponds to the remaining value of the TTU of the data.

For the data A, B, and C, the score SX'(X is the data name (A, B, C)) calculated by using the above formula (2) is as follows. The current time is 300.
SA'= {(10 + 500) -300} + 200 = 410
SB'= {(5 + 500) -300} + 200 = 405
SC'= {(100 + 500) -300} + 200 = 500
In this case, the data C, A, and B are in the order of increasing score.
As described above, in the conventional technique, it is determined that the importance is low even if the data is accessed when the AoI is small like the data B and is accessed a plurality of times.

The data moving unit 210 moves at least one of the data stored in the storage device 100 to another storage device 100 (another storage node 12) or an evacuation storage 100A based on the score. When the data movement unit 210 is within the first range indicating that the score of the data to be moved (movement target data) is high, the data movement unit 210 other than itself (other) storage node 12 (FIG. 1). The data to be moved is moved to another storage device 100 provided in (see). At that time, when the storage network 10 has a hierarchical structure as shown in FIG. 1, it is desirable that the data moving unit 210 moves the data to be moved to a storage device higher than the current storage device 100. Further, when the score of the movement target data is in the second range indicating that the importance is low, the data movement unit 210 moves the movement target data to the evacuation storage 100A (see FIGS. 2 and 3).

That is, the data moving unit 210 moves the highly important data among the data stored in the storage device 100 to another storage device 100 (for example, a higher-level storage device), that is, the “process A”, which is less important. “Process B” may be performed to move the data to the save storage 100A. In addition, "Process C" for deleting less important data may be performed.
For example, when the storage device 100 is located at the storage node 12B in FIG. 1, the highly important data is moved to the storage node 12A. Further, the less important data is moved to the evacuation storage 100A (FIGS. 2 and 3) in the own storage node 12.

The first range or the second range may be defined by a specific score value, for example, "data of a predetermined number (or a predetermined capacity) in descending order of score". You may.
The data movement by the data movement unit 210 may be performed periodically at predetermined time intervals, for example, when it is desired to store new data in a state where the storage device 100 has no free space. This is because the data moving unit 210 does not delete the data as in the prior art, but moves the data. By moving the data periodically according to the importance, the data in the storage network 10 is moved. The placement can be optimized.

It should be noted that which of the above processes A and B may be performed by the data moving unit 210 may be determined, for example, according to the status of each storage device in the storage network 10. For example, when the storage device 100 is located at the storage node 12B in FIG. 1, it is easy to acquire the operating status of the storage devices at the storage node 12A and the storage node 12C, for example, the availability status and the transfer speed between the nodes, and perform data transfer. Select a node (for example, a node that has a storage device with sufficient free space, or a node that has a high transfer speed (no network delay)). As a result, for example, when a higher node (or a lower node) is selected, the above processing A is performed on the data having high importance. Further, when another storage device 100 satisfying a predetermined condition is not found based on the operating condition, the above process B may be performed on the data having low importance. In addition to that, when there is no movable upper node that satisfies the predetermined condition based on the operating status (when only the movable lower node exists), the above processing is performed for the less important data. B may be performed.
That is, the data movement unit 210 determines the data movement destination storage device based on the operating status of the other storage device. At that time, when the storage network 10 has a hierarchical structure as shown in FIG. 1, the data moving unit 210 may determine the moving target data based on the position in the hierarchical structure of the moving destination storage device. ..
By doing so, it is possible to select the data to be moved based on the operating status of the destination, and it is possible to efficiently move the data within the storage network 10.

FIG. 5 is an explanatory diagram schematically showing cache control in the data management device 20.
The storage device 100 has a capacity α, and data a, b, and c are stored in this order at the time of reference numeral T10 in FIG. Since the storage device 100 has free space at the time of reference numeral T10, the new data d can be stored as it is. The new data d is stored next to the data c.

Further, when the data in the storage device 100 is accessed by the user, the score giving unit 208 recalculates the score and sorts the data based on the score. For example, when the data a is accessed by the user at the time of the reference numeral T11 in FIG. 5 (arrow 111 of the reference numeral T11), the score of the data a is recalculated and rearranged. In the prior art, the accessed data is at the end of the data string, but in the present embodiment, it is not necessarily the end, and may be the second from the end, for example, as in reference numeral T12. In this case, the data h at the end is, for example, frequently accessed data.

Further, in order to store new data i in a state where the storage device 100 has no free space as at the time of reference numeral T13 in FIG. 5, the data moving unit 210 stores one of the data in the other storage device 100 or the saved storage. Move to 100A. For example, when the above process A is performed, the data h, which is the most important data, is moved to another storage device 100 (for example, a higher storage device) (arrow 131 of reference numeral T13). Further, when the above process B is performed, the data b, which is the least important data, is moved to the save storage 100A (arrow 132 of reference numeral T13). When performing the above process C, the data b, which is the least important data, is deleted.

FIG. 6 is a flowchart showing a processing procedure of the data management device 20.
The scoring unit 208 monitors access to each data in the storage device 100, calculates a score for sequential importance, and sorts the data based on the score (step S1).
Next, it is determined whether the new data receiving unit 202 has received the new data (step S2). Then, if the new data receiving unit 202 has not received the new data (step S2: No), the new data receiving unit 202 waits until it is received. On the other hand, when the new data receiving unit 202 receives the new data (step S2: Yes), the new data receiving unit 202 proceeds to step S3.
The score calculation and data rearrangement by the score giving unit 208 may be performed at predetermined time intervals in step S1 or that new data has been received in step S2 (step S2: Yes). You may go as an opportunity.

In step S3, the capacity monitoring unit 206 confirms the free capacity of the storage device 100. Then, when the capacity monitoring unit 206 determines that there is enough free space to store the new data (step S3: Yes), the data storage unit 204 stores the new data in the storage device 100 (step S4).
On the other hand, if the capacity monitoring unit 206 determines that there is not enough free space to store new data (step S3: No), the process proceeds to the next step S5.

In step S5, the data moving unit 210 refers to the score of each data stored in the storage device 100, and determines whether or not there is data within the first range indicating that the score is of high importance. do. Then, when it is determined that the data whose score is within the first range exists (step S5: Yes), the data moving unit 210 moves the data to another storage device 100 (for example, a higher storage device). (Step S6). Then, the process proceeds to the next step S7. On the other hand, if there is no data whose score is within the first range (step S5: No), the process proceeds to step S7.

In step S7, the data moving unit 210 refers to the score of each data stored in the storage device 100, and determines whether or not there is data in the second range indicating that the score is of low importance. do. Then, when it is determined that the data whose score is within the second range exists (step S7: Yes), the data moving unit 210 moves the data to the evacuation storage 100A (step S8). Then, the process proceeds to the next step S9. On the other hand, if there is no data whose score is within the second range (step S7: No), the process proceeds to step S9.
Either of the determination processes in steps S5 and S7 may be performed first. Further, in steps S5 and S7, when a plurality of data to be moved (moving target data) are found, at least one of the moving target data may be set to be moved. Further, the data having a low importance score found in step S7 may be set to be deleted.

As a result of executing the process of step S6 or step S8, the data is moved, and a free space for storing new data is formed in the storage device 100. After that, the data storage unit 204 stores new data in the storage device 100 (step S9).

As described above, according to the data management device 20 according to the embodiment, it is possible to move highly important data to another storage device 100 (for example, a higher-level storage device) without deleting the data. In addition, less important data can be moved to the save storage 100A without being deleted. Therefore, it is possible to reliably store the data that should not be deleted. Further, according to the data management device 20, the time from the generation of the data to the access and the number of accesses to the data can be reflected in the score indicating the importance. By determining the destination of data movement based on this score, it is possible to improve the efficiency of accessing data in the storage network 10.

10 Storage network 12,12A-12D Storage node 14,14A User 20 Data management device 100 Storage device 100A Saved storage 202 New data reception unit 204 Data storage unit 206 Capacity monitoring unit 208 Score assignment unit 210 Data movement unit

Claims (6)

A data management device in a storage network to which multiple storage devices are connected.
A backup storage to which data stored in the storage device is moved is connected to each of the storage devices.
In each of the storage devices, the data management device
A data moving unit that moves at least one of the data stored in the storage device to another storage device or the evacuation storage when the storage capacity of the storage device is insufficient.
Each data stored in the storage device is provided with a scoring unit that gives a score regarding the importance of the data.
When the score of the movement target data is within the first range indicating that the importance is high, the data movement unit moves the movement target data to the other storage device, and the movement target data of the movement target data. When the score is in the second range indicating that the importance is low, the move target data is moved to the save storage.
A data management device characterized by that. The score giving unit calculates the score using a value obtained by subtracting the time when the data was generated from the time when the data was accessed.
The data management device according to claim 1 . The score giving unit calculates the value each time the data is accessed, and calculates the score based on the sum of the values.
The data management device according to claim 2 . The data moving unit uses the data whose score is within the first range or the data whose score is within the second range to be moved target data based on the operating status of the other storage device. Decide as,
The data management device according to any one of claims 1 to 3 , wherein the data management device is characterized by the above. The storage network has a hierarchical structure for users who use data.
When moving data whose score is within the first range to the other storage device, the data moving unit is located higher than the position of the storage device in the hierarchical structure closer to the user. To move the data to be moved to the other storage device,
The data management device according to any one of claims 1 to 3, wherein the data management device is characterized by the above. It is a data management method of a data management device in a storage network to which multiple storage devices are connected.
A backup storage to which data stored in the storage device is moved is connected to each of the storage devices.
In each of the storage devices, the data management device
A scoring step for assigning a score regarding the importance of the data to each data stored in the storage device, and a scoring step.
When the storage capacity of the storage device is insufficient, a data movement step of moving at least one of the data stored in the storage device to another storage device or the evacuation storage is executed .
In the data movement step, if the score of the movement target data is within the first range indicating that the importance is high, the movement target data is moved to the other storage device, and the movement target data is transferred. When the score is in the second range indicating that the importance is low, the move target data is moved to the save storage.
A data management method characterized by that.

Images