KR101533340B1 - A method of data replication using data access frequency and erasure codes in cloud storage system - Google Patents

Abstract

The present invention relates to a method for replicating data by using data access frequency and erasure codes in a cloud storage system, wherein the method is configured to replicate data by using erasure codes for improving storage space efficiency and input/output performance in an SSD-based cloud storage system. More particularly, the method comprises the steps of: (a) chunking an input file with respect to a write command of the input file; (b) generating an access frequency parameter for the input file; (c) determining the type of the input file among three types according to the access frequency parameter; and (d) determining the number of replicas of the input file to be stored, and whether erasure coding has been performed or not, according to the determined type, and storing a replica of the input file or results of erasure coding. The method for replicating data can solve a problem of wasting a storage space and increasing input/output while maintaining the same recovery performance with a prior art which stores three replicas on original data in a cloud storage system.

Description

[0001] The present invention relates to a method for data replication using a data access frequency and an erase code in a cloud storage system,

The present invention relates to a data replication method using a data access frequency and an erase code in a cloud storage system for replicating data using Erasure Codes in order to improve storage space efficiency and input / output performance in an SSD-based cloud storage system .

In 2012, 2.8 ZB (Zeta Bytes, Zeta Byte) will be generated by the use of PCs, mobile and notebook devices worldwide. It is predicted that 40ZB will be generated by 2020, and big data processing is becoming a big issue as the data size rapidly increases. It takes a lot of storage space to store vast amounts of data generated from cloud storage.

To efficiently store and manage big data, cloud storage systems use a distributed file system to network multiple servers and nodes together. Through this, cloud storage systems provide high availability and high scalability [Non-Patent Document 1].

Among the distributed file systems, the open source Hadoop Distributed File System (HDFS), developed by the Apache Foundation, is widely used in cloud storage systems. For reliability of storage space, HDFS chunks data in block units of 64 MB, creates three replicas, and distributes the blocks to data nodes and stores them [Non-Patent Document 2]. FIG. 1 shows the problem of the file replication process of the HDFS. If 320MB of data is received, the data is chunked in 64MB block units, and then 3 replicas are created and distributed. Since this method requires three times as much storage space as the original data size, the wasted storage space becomes larger as the data size increases [Non-Patent Document 3].

In addition, most cloud storage systems use HDDs, which are less expensive per storage space, to save money. However, a HDD operating using a spindle motor has a low random I / O (I / O) performance, resulting in a decrease in speed [Non Patent Document 4]. Therefore, in a cloud environment where random I / O is frequently generated, the HDD becomes a bottleneck.

Solid state disk (SSD) is a storage device based on NAND flash memory. It has higher sequential and random I / O performance than HDD. Using SSD as cloud storage can reduce the bottleneck caused by using HDD. Because it is becoming cheaper in terms of cost, cloud storage products based on Flash Array are being developed.

[Non-Patent Document 1] D. J. Abadi, "Data Management in the Cloud: Limits and Opportunities," in Proc. of IEEE Conf. on Data Engineering, pp.1-10, Shanghai, China, March 2009. [Non-Patent Document 2] K. Shvachko, H. Kuang, S. Radia, "The Hadoop Distributed File System," in Proc. of IEEE Conf. on Mass Storage System and Technologies, pp. 1 to 10, Santa Clara, California, USA, May 2010. [Non-Patent Document 3] J. Wang, W. Gong, P. Varman, C. Xie, "Reducing Storage Overhead with Small Write Bottleneck Avoiding in Cloud RAID System, of ACM and IEEE Conf. on 13th Grid Computing, pp.174-183, Beijing, China, September 2012. [Non-Patent Document 4] B. Mao, H. Jiang, S. Wu, Y. Fu, L. Tian, "SAR: SSD Assisted Restore Optimization for Deduplication-based Storage System in the Cloud," in Proc. of IEEE Conf. on 7th Networking, Architecture, and Storage, pp. 328-337, Xiamen, China, June 2012. [Non-Patent Document 5] B. Fan, W. Tantisiriroj, L. Xiao, G. Gibson, "DiskReduce: RAID for Data-Intensive Scalable Computing," in Proc. of ACM Conf. on Supercomputing PDSW'09, pp. 6-10, Portland, Oregon, USA, November 2009. [Non-Patent Document 6] S. Plank, S Simmerman, CD Schuman, "Jerasure: A Library in C / C ++ Facilitating Erasure Coding for Storage Applications," Technical Report CS-08-627, University of Tennessee Department of Electrical Engineering and Computer Science, pp. 1-59, August 2008. [Non-Patent Document 7] JS Plank, "A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems," Software-Practice & Experience, Vol.27, No. 9, pp.995-1012, September 1997 . [Non-Patent Document 8] M. Blaum, J. Brandy, J. Bruck, M. Jai, "EVENODD: An Efficient Scheme for Tolerating Double Disk Failures in RAID Architectures," IEEE Transactions on Computers, Vol. , pp. 192-202, February 1995. [Non-Patent Document 9] L. H. James, "WEAVER Codes: Highly Fault Tolerant Erasure Codes for Storage Systems," in Proc. of ACM Conf. on FAST, pp.1-10, 2005. [Non-Patent Document 10] X. Lihao, J. Bruck, "X-code: MDS array codes with optimal encoding," IEEE Transactions on Information Theory, Vol. 45, No.1, pp. 272-276, January 1999. [Non-Patent Document 11] J-H. Jo, J-K. Kim, P. Mahdi, D-H. Kim, "Data Replication Method using Erasure Code in SSD based Cloud Storage System," in Proc. of IEEK Conf. on Summer Conference, Vol. 36, No. 1, pp. 1539-1542, Jeju, Korea, July 2013. [Non-Patent Document 12] J-K. Kim, J-H. Jo, P. Mahdi, D-H. Kim, "Unified De-duplication Method of Data and Parity Disks in SSD-based RAID Storage," in Proc. of IEEK Conf. on Summer Conference, Vol. 36, No. 1, pp. 1543-1546, Jeju, Korea, July 2013. [Non-Patent Document 13] D. Park, D. H. C. Du, " Hot Data Identification for Flash-based Storage Systems Using Multiple Bloom Filters, "in Proc. of IEEE on 27th Mass Storage Systems and Technologies (MSST), pp. 1-11, Denver, Colorado, USA, May 2011. [Non-Patent Document 14] J.-W. Hsieh, L.-P. Chang, T.-W. Kuo, "Efficient Online Identification of Hot Data for Flash-Memory Management," in Proc. of ACM on 20th Symposium on Applied Computing (SAC) pp.838-842, Santa Fe, New Mexico, March 2005. [Non-Patent Document 15] H.-S. Lee, H.-S. Yun, D.-H. Lee, "HFTL: Hybrid Flash Translation Layer based on Hot Data Identification for Flash Memory," IEEE Transactions on Consumer Electronics, Vol.55, No.4, pp.2005~2011, November 2009.

An object of the present invention is to solve the above-mentioned problems, and it is an object of the present invention to provide a cloud storage system in which data is replicated using Erasure Codes in order to improve storage space efficiency and input / output performance in an SSD- And a data replication method using the data access frequency and the erase code.

In particular, it is an object of the present invention to provide a data replication method using a data access frequency and an erasure code in a cloud storage system that maintains the same data recovery performance by using erasure codes even if the number of times of replication is reduced according to a data access frequency .

According to an aspect of the present invention, there is provided a data replication method using a data access frequency and an erase code in a cloud storage system, the method comprising the steps of: (a) Chunking; (b) generating an access frequency parameter for the input file; (c) determining the input file as any one of the three types according to the access frequency parameter; And (d) determining the number of replicas to be stored in the input file and whether to perform erasure coding according to the determined type, and storing the replica or erasure coding result of the input file.

According to another aspect of the present invention, there is provided a data replication method using a data access frequency and an erase code in a cloud storage system, the method comprising the steps of: (e) updating an access frequency parameter of the input file with respect to a read command of the input file; (f) updating the type of the input file according to an access frequency of the updated input file; (g) when the type of the input file is changed, a copy of the input file is deleted or an additional copy is generated in accordance with the number of copies according to the changed type, and an erasure coding result is generated according to whether the erasure is coded according to the changed type Deleting the original erasure coding result; And (h) reading the combined chunks of the input file.

Further, the present invention provides a data replication method using a data access frequency and an erase code in a cloud storage system, wherein the type is classified into a hot data type, a worm data type, and a cold data type, And if the access frequency parameter is equal to or greater than the first threshold value, the access frequency parameter is determined as a data type, and if the access frequency parameter is less than a predetermined second threshold value And if it is the same or larger, it is determined as a hot data type, and the secondary threshold value is larger than the primary threshold value.

The present invention also provides a data replication method using a data access frequency and an erase code in a cloud storage system, the method comprising: generating and storing three replicas if the input file is a hot data type; and if the input file is a worm data type, And generates and stores an erasure coding result. If the input file is a cold data type, one file and an erasure coding result are generated and stored.

According to another aspect of the present invention, there is provided a data replication method using a data access frequency and an erase code in a cloud storage system, wherein the erase coding result is a parity chunk.

As described above, according to the data replication method using the data access frequency and the erase code in the cloud storage system according to the present invention, the cloud storage system can store An effect of solving the problem of waste of space and increase of input / output can be obtained.

Particularly, through experiments of the present invention, by comparing the method of generating three copies of data and the method of generating only parity, the frequency of data access is divided into hot, warm and cold, The data replication method using Erasure Codes achieves 40% more storage space efficiency, 11% read performance and 10% write performance than HDFS.

1 shows an example of storage space usage of an HDFS according to the prior art;
2 is a block diagram of an entire system for implementing the present invention;
3 is a configuration diagram of a RAID system according to an embodiment of the present invention;
4 is an illustration of an example of a hot data type data replication method in accordance with one embodiment of the present invention.
5 is an illustration of an example of a data replication method for a data type according to one embodiment of the present invention.
Figure 6 is an illustration of an example of a cold data type data replication method in accordance with one embodiment of the present invention.
FIG. 7 is an exemplary diagram illustrating a data replication method using an erase code according to an embodiment of the present invention; FIG.
FIG. 8 is a flowchart illustrating a method of a write command using a data access frequency and an erase code in a cloud storage system according to an embodiment of the present invention. FIG.
FIG. 9 is a flowchart illustrating a data read method using a data access frequency and an erase code in a cloud storage system according to an embodiment of the present invention. FIG.
10 is an example of an algorithm implementing a data replication method using a data access frequency and an erase code in a cloud storage system according to an embodiment of the present invention.
11 is a table showing specification information of the SSD used in the experiment according to the experiment of the present invention.
12 is a table showing the experimental environment according to the experiment of the present invention.
Figure 13 is a table showing experimental performance comparisons of storage space efficiency according to the experiments of the present invention.
14 is a table showing experimental performance comparisons for data writing in accordance with the experiments of the present invention.
15 is a table showing experimental performance comparisons for reading data according to an experiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In the description of the present invention, the same parts are denoted by the same reference numerals, and repetitive description thereof will be omitted.

First, the entire system for implementing the present invention will be described with reference to FIG.

As shown in FIG. 2, the overall system for carrying out the present invention comprises a client 10, a data node 20, and a replication system 30.

The client 10 is a conventional computing terminal such as a PC, a notebook, a netbook, a PDA, or a mobile, and is a terminal requesting a service for storing data or reading data in the clone system 30.

The data node 20 is a server or a storage server for storing data on a network, and has an SSD storage or the like and controls the data to perform a data input / output service (or function). The SSD storage included in the data node 20 is a storage or storage device configured by a solid-state disk (SSD) disk. The SSD is a storage device configured by combining NAND flash memories in parallel.

The cloning system 30 receives the request from the client 10 and fetches the data stored in the data node 20 or stores the data received from the client 10 in the data node 20. Preferably, the cloning system 30 configures the data nodes 20 by the Hadoop Distributed File System (HDFS).

The cloning system 30 is connected to at least three data nodes 20 to distribute and store data. When duplicating data and storing two or more duplicate data, the duplicating system 30 duplicates and stores duplicate data in the different data nodes 20. In particular, the cloning system 30 stores the data in chunking units after chunking the input data.

Further, the clone system 30 can be configured in a RAID system manner. Preferably, the clone system 30 is configured by a Hadoop Distributed File System (HDFS) using RAID.

In this case, when parity information is stored using erasure codes, parity data is generated in units of chunks, and data and parity data are stored in units of chunks.

As shown in FIG. 3, data chunks and parity chunks are stored in respective data nodes 20 in a striped fashion. Preferably, the chunk size range is 32 to 128 KB. In RAID, the stripe method divides data into chunk size units, and stores chunks of the same number of RAIDs as one set.

At this time, as shown in FIG. 3, the chunks of the data set and the parity chunks are stored in the order of A1, A2, A3, and PA by the stripe method in RAID-5,6. In the next data set, B2, B3, PB, and B1 are stored in this order. This is to prevent reliability degradation when parity chunks are biased on one data node. If the parity chunk is biased to one data node, if the parity node fails, the parity information is not available and data recovery can not be performed.

On the other hand, when the data is lost due to the damage of the data node or the like, the cloning system 30 can use the data chunks copied to other data nodes or recover the data using the parity data.

Next, the Hadoop distributed file system (HDFS) of the clone system 30 according to an embodiment of the present invention will be described in detail.

The Hadoop Distributed File System (HDFS) is open source software that allows you to process large amounts of data. HDFS consists of one name node and a number of data nodes. The name node manages the namespace of HDFS and handles client file access requests. The data node of HDFS stores the data input / output request of the client in the basic 64 MB block unit. It also provides the ability to create replicas in computer clusters to automatically recover quickly from loss of some storage devices. The recovery function creates as many copies of the data as the user specifies, usually three replicas. The creation of these three replicas will increase storage usage as the data size grows. HDFS is well suited for the cloud environment, but it has the disadvantage of high storage space consumption by creating three replicas.

In addition, a Hadoop distributed file system (HDFS-RAID) using a RAID (Redundant Array of Independent Disks) will be described.

HDFS-RAID is a distributed file system that uses RAID instead of Data Replication Method in existing HDFS. HDFS-RAID is also referred to as Distributed RAID File System (DRFS) and has been studied by the Apache Foundation [Non-Patent Document 5].

For reliability, HDFS-RAID does not create a replica of the data, but creates RAID 6 stripes and stores the parity blocks together. When the data is lost, the original data is recovered through the data recovery process from the parity block. HDFS-RAID can recover the same two losses, but uses less storage space compared to creating three replicas of existing HDFS. Therefore, it is effective when applied to cloud storage systems with large data storage capacity.

For example, there has been an increase in storage space efficiency by applying HDFS-RAID on Facebook. Facebook is the world's largest social networking service with more than a billion users around the world, sharing an average of 2.5 billion content per day and handling more than 500 terabytes of data. As such, the vast amount of data generated on Facebook is accumulated every day and big data is formed. On Facebook, HDFS-RAID was introduced, which resulted in a capacity reduction of about 5PB. Facebook's case shows that storage space usage is efficient when applying erasure codes to data replication methods.

Next, the erasure coding method used in the present invention will be described in detail.

Erasure Coding (or Erase Coding) is a method of storing codes generated through data and encoding processes and recovering original data through a decoding process in case of data loss [Non-Patent Document 6]. Erasure Codes are used on systems where data reliability is important due to excellent error recovery performance and I / O performance. The parity generated by the erasure codes occupies less storage space than the data replica generation, thus providing reliability and storage space efficiency.

Examples of the types of erasure codes include a Reed-Solomon code (Non-Patent Document 7), an EVENODD code (Non-Patent Document 8), a Weaver code Document 9], and X-code [Non-Patent Document 10]. Techniques are being developed to use different algorithms for each erasure code and to improve recovery performance while reducing computational complexity.

Next, a data replication method according to data access frequency in a cloud storage system according to an embodiment of the present invention will be described with reference to FIG. 4 to FIG.

Prior art data replication methods in cloud storage create three replicas to maintain reliability from data loss. Therefore, the replication method according to the related art causes waste of storage space. In contrast, the method according to the present invention uses three different replication methods depending on data access frequency in order to increase storage space efficiency and input / output speed.

The method according to the present invention classifies data into hot data, warm data, and cold data according to the frequency of access.

Hot data is frequently used because of its high frequency of access. Hot data creates three replicas as shown in FIG. 4 to guarantee robust reliability in case of data loss [Non-Patent Document 11].

왐 Data (Warm Data) is data with a lower access frequency than Hot Data, but it is likely to become hot data due to frequent use of clients later. Therefore, as shown in FIG. 5, the Warm Data secures reliability by using two replica creation and erasure codes. Cold data is data that is not frequently used because it has the lowest frequency of access and is prepared for data loss using only erasure codes as shown in FIG.

Generally, since hot data occupies a relatively small proportion of the entire data, even if three replicas are created, the storage space efficiency does not decrease significantly. In the case of Warm data and Cold data, which have a lower access frequency than hot data, since the amount of data is large, parity is generated by applying erasure codes for storage space efficiency FIG. 5 and FIG. 6 show data sizes when erasure codes are applied.

Compared with FIG. 4, the replication method requires three times as much storage space as the data size. However, in case of Cold Data, applying Erasure Codes requires 1.4 times as much storage space as data. In addition, although the storage speed has a parity operation time, the write speed is also improved because it takes less time than generating three copies.

That is, when the erasure coding result is regarded as "data chunk of the input file + parity chunk", the hot data, the warm data, and the cold data can be summarized as follows.

The hot data is distributed and divided into three replicas or "input file + two replicas", and the worm data is divided into one replica and erasure coding results (input file chuning + parity chunning) and stored. In addition, the cold data is divided and stored in the erase coding result (input file chunking + parity chunking).

Next, a data replication method using erasure codes in a cloud storage system according to an embodiment of the present invention will be described in more detail with reference to FIG.

The data replication method for data (Warm Data) and cold data (Cold Data) stores parity information using original data and erasure codes.

FIG. 7 shows an example of using RAID-6 and erasure codes for data replication. RAID-6 stores two parity blocks so that two data blocks can be recovered if they are lost. In addition, Erasure Codes generate parity information through encoding, which is an operation process, on original data (Non-Patent Document 12).

The writing process of the data replication method using Erasure Codes first passes through the process of chunking input data, and then generates chunks of parity through an erasure coding process for each chunk as shown in FIG.

The generated parity chunks are combined with the data chunks to form a single stripe. Thus, data and parity chunks are combined in one stripe. Where each data and parity chunk must be allocated and stored in one data node. In addition, each stripe stores data nodes to which a parity chunk is to be arranged while changing them, thereby preventing a parity chunk from being pushed into one data node.

Next, a method of data replication using a data access frequency and an erase code in a cloud storage system according to an embodiment of the present invention will be described with reference to FIGS. 8 to 10. FIG.

8 and 9 are flowcharts of a data replication method using erasure codes according to an access frequency, and show flowcharts of a write command and a read command, respectively. FIG. 10 shows an algorithm implementing the data replication method according to the present invention.

First, when a write command is entered into the system, the data is chunked in blocks of 64 MB.

Next, the access frequency count value and the storage method of the data are stored in the metadata area for the data requested to be written.

The access frequency parameter is the accumulated number of accesses since the data was entered. And if there is no data access within a specific time range (e.g., 6 hours), reduce the access frequency parameter (e.g., by 1). This is for switching from hot data to cold data.

Since the file to be written first is cold data because there is no access frequency, erasure coding is applied to store the original file and parity information.

Then, when a read command is input to each file, the metadata is referred to, and it is checked how the data is stored and the address value is confirmed. Since the access frequency is increased, the access frequency count value is incremented by 1 in the metadata. It then collects the chunks that make up the file and provides the data requested to be read.

If the read command is frequently generated and the count value of the access frequency increases or becomes equal to or larger than the first threshold value, the data is stored as Warm Data, and a duplicate copy of the original file is stored for reliability.

If the access frequency count value further increases and becomes equal to or larger than the second threshold value, the hot data becomes a hot data and the replica is stored one more time.

On the other hand, if the read command for the file does not occur for a predetermined period of time, the access frequency count value is decreased and stored in an erasure coding method to improve storage space efficiency. When the access frequency count value becomes smaller than the second threshold value, one replica is deleted to convert hot data into warm data and parity information of erasure codes is generated. When it becomes smaller than the first threshold value, one more replica is deleted to replace the cold data with the cold data, and only the original data and the parity information are left.

Figure 7 shows how data chunks are stored in a data node. That is, data nodes 1 to 8 are displayed in Fig. The result of the replica or erase code is distributed and stored in the data node. It does not store all chunks in one data node. FIG. 7 shows that data chunks and parity chunks are striped and stored in a plurality of data nodes in the case of erasure coding.

The storage of the replica stores only the parity chunks in FIG. 7, and the data chunks are distributed and stored in the plurality of data nodes. That is, if the access frequency is equal to or higher than the first threshold value, the cold data is changed to the worm data, and at this time, the replica is stored one more time. If the access frequency is more than the second threshold, the data is changed from the worm data to the hot data.

Next, the effects of the present invention will be described through experimental results.

In order to evaluate the performance of the method according to the present invention, the workload data sizes were set at 256 MB, 512 MB, 1 GB, and 2 GB, respectively. Because the workload was small, the chunk size was chunked to 4KB.

The threshold value, which is the criterion for determining the data storage method, was randomly read command before each experiment to check the frequency count of each data.

The ratio of hot data, warm data, and cold data is obtained from a workload extracted from a real server [Non-patent Documents 13 to 15]. Each reference paper used the workloads of Financial1, MSR, Distilled, RealSSD, digital camera, and Linux O / S.

In each workload, the average rate of hot data was 10%, so in the experiment, the ratio of hot data was set at 10%. 왐 Data (Warm Data) is defined as 20% by referring to the fact that the ratio of hot data is increased up to 30% in the reference paper. In the experiment, the ratio of hot data is set to about 10% of the total data, and the worm data is about 20% of the total data and the cold data is about 70% . In the parity generation method, a parity bit is generated by applying a Reed-Solomon code. At the time of replication, three replicas were created and stored like the existing distributed file system.

The experimental environment is as follows. IOzone was used as a benchmark tool to generate workload data of various sizes, and we used three PCs with the minimum number of HDFSs. The specification of each PC is the same with Intel Core2 (2.4Ghz) and DDR2 2GB memory. The operating system used kernel version 3.8.13 on Linux CentOS 6.2. The storage device is equipped with two SAMSUNG SSD 64GB in each PC. The specifications of the SSD used are shown in the table of FIG. 11, and the performance and environment of the computer used in the experiment are shown in the table of FIG.

The experimental results are shown in Figs. 13, 14 and 15. The storage space efficiency and the write / read performance are compared with each other, and the HDFS using only the data replication method and the HDFS-RAID using only the parity method are compared with each other according to the present invention.

Experimental Results In the storage space efficiency of FIG. 13, the method according to the present invention uses about 40% less storage space than HDFS. HDFS consumes the most storage space because it makes 3 replicas, and HDFS-RAID occupies the most storage space because it uses only erasure codes. The method according to the present invention is located between the two and shows a value close to that of HDFS-RAID, indicating that storage space efficiency is improved.

14 and 15 are graphs comparing write performance and read performance, respectively. In the write performance, the method according to the present invention is improved by about 10% from HDFS and about 7% from HDFS-RAID. In the read performance, the method according to the present invention is improved by about 11% compared to HDFS and about 12% higher than HDFS-RAID. Experimental Results The method according to the present invention is divided into Hot, Warm, and Cold to divide data access frequency, thereby improving performance compared to methods of generating only a data copy or generating only parity. This suggests that it is more efficient to use the replication and parity methods together, considering access frequency rather than creating multiple copies or parity.

In the present invention, in order to solve the problem of waste of storage space and increase of input / output in a method of storing three replicas in original data in a cloud storage system, data replication using existing erasure codes according to data access frequency Method. The same recovery performance as that of generating three replicas was maintained by using existing erasure codes according to data access frequency. In the experiment, we compared the method of generating three data replicas and the method of generating only parity. The data replication technique using Erasure Codes by dividing the data access frequency into hot, warm and cold divides the storage space efficiency by 40%, read performance by 11% and write Performance was improved by 10%.

The invention made by the present inventors has been described concretely with reference to the embodiments. However, it is needless to say that the present invention is not limited to the embodiments, and that various changes can be made without departing from the gist of the present invention.

10: Client 20: Data node
30: Replication system

Claims (5)

1. A data replication method using a data access frequency and an erase code in a cloud storage system in which an input file is replicated and stored,
(a) chunking the input file with respect to a write command of the input file;
(b) generating an access frequency parameter for the input file;
(c) determining the input file as any one of the three types according to the access frequency parameter; And
(d) determining the number of replicas to be stored in the input file and whether to perform erasure coding according to the determined type, and storing a copy or erasure coding result of the input file;
(e) updating an access frequency parameter of the input file with respect to a read command of the input file;
(f) updating the type of the input file according to an access frequency of the updated input file;
(g) when the type of the input file is changed, a copy of the input file is deleted or an additional copy is generated in accordance with the number of copies according to the changed type, and an erasure coding result is generated according to whether the erasure is coded according to the changed type Deleting the original erasure coding result; And
(h) reading together the chunks of the input file,
The type is classified into a hot data type, a worm data type, and a cold data type, and if the access frequency parameter is smaller than a predetermined first threshold value, the type is determined as a cold data type, And if the access frequency parameter is equal to or greater than a predetermined second threshold value, it is determined to be a hot data type, and if the access frequency parameter is equal to or greater than a predetermined second threshold value,
Wherein the second threshold is greater than the first threshold,
Wherein the access frequency parameter accumulates the number of accesses and decreases if there is no data access within a predetermined time range,
If the input file is a hot data type, three replicas are created and stored,
If the input file is a worm data type, two replicas and an erasure coding result are generated and stored,
And generating and storing one file and an erasure coding result if the input file is a cold data type, and storing the generated file and the erase coding result in the cloud storage system.
delete delete delete The method according to claim 1,
Wherein the erasure coding result is a parity chunk. The data replication method using a data access frequency and an erasure code in a cloud storage system.

Images