KR101895895B1 - Data deduplication method and system - Google Patents

Abstract

The present invention discloses a data deduplication method and a system therefor. A method for data deduplication in a data deduplication system including a user client, a metadata server, and a cloud storage server according to an aspect of the present invention comprises the steps of requesting, at a user client, a check for duplication of data to be uploaded to a metadata server; checking, at the metadata server, whether the requested data is duplicated, and transmitting the result to the user client; generating, at the user client, ownership verification data through an XOR operation according to the received result and transmitting the generated ownership verification data to the metadata server; performing, at the metadata server, ownership verification using the received ownership verification data, and transmitting the result to the user client; transmitting, at the user client, encrypted data to the cloud storage server when receiving the data upload permission as a result of the verification; and receiving, at the cloud storage server, the encrypted data from the user client and updating the data.

Description

{DATA DEDUPLICATION METHOD AND SYSTEM}

The present invention relates to a data deduplication method, and more particularly, to a data deduplication method for performing duplication of encrypted data in a cloud storage environment by performing ownership verification through XOR operation, ≪ / RTI >

Recently, as computer network technology has developed, data storage environment has changed from using physical storage device to using cloud storage through network.

Cloud storage is replacing some of the existing physical storage media in that it is accessible anywhere on the network and can be stored and downloaded. The service using the cloud storage is generally provided by a company that provides contents on a network, and is used simultaneously by a plurality of users. In this case, various data are stored in a cloud storage used by a plurality of users, but a large number of stored data is stored in the storage of already stored data, thereby increasing the ratio of redundantly stored data, . As a result, the maintenance cost may increase to expand the cloud storage space, and the higher the amount of redundant data, the more the maintenance cost increases.

Various deduplication techniques have been studied to prevent duplication of data stored in the storage space of cloud storage. Among the various de-duplication techniques, the most basic de-duplication technique is to use a hash algorithm to convert data to original file units, and then to compare one-to-one with the same data already stored in the storage. At this time, the above-mentioned deduplication technique can be divided into file unit deduplication and block unit deduplication according to the data comparison unit.

First, in the file-based de-duplication technique, a hash value of a data source is compared on a one-to-one basis by defining a redundant unit of data in units of files. The comparison speed of data is very fast. However, It is recognized as different data and there is a problem that duplication is not removed. In order to overcome this problem, a block-based de-duplication technique has been developed. In the block-based de-duplication technique, a data source is divided into a predetermined size, and a hash value converted through a hash algorithm is used. Therefore, a plurality of data blocks are generated from one data source, If the part of the data source is different, the deduplication efficiency is improved because only the remaining blocks are deduplicated. However, since the block is removed in units of blocks, there arises a problem in that a large number of operations are generated in the processing as the number of data blocks increases.

On the other hand, cloud storage is a type of remote server, and remote servers have the possibility of data leakage due to internal or external factors. Therefore, even if data stored in the cloud storage is leaked, Should be encrypted so that the original contents of the data can not be known. However, these two technologies can not be applied at the same time because the data de-duplication technology and the data encryption technology have conflicting properties. Various security technologies have been applied to solve these problems, but there are problems in that additional security threats such as a large number of operations and counterfeit attacks are generated in the application process.

Korean Registered Patent No. 10-1590270 (published on February 02, 2016)

Disclosure of Invention Technical Problem [8] Accordingly, the present invention has been made to solve the above problems, and it is an object of the present invention to provide a method for removing duplication of encrypted data having a relatively small amount of computation by improving the ownership verification method included in the deduplication process It has its purpose.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by one embodiment of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

According to an aspect of the present invention, there is provided a method for removing data de-duplication in a data de-duplication system including a user client, a metadata server, and a cloud storage server, Requesting confirmation of duplication of data to be uploaded; Checking whether the metadata requested by the metadata server is duplicated, and transmitting the result to the user client; Generating the ownership verification data through the XOR operation according to the received result and transmitting the generated ownership verification data to the metadata server; Performing the ownership verification using the received ownership verification data and transmitting the result to the user client; Transmitting encrypted data to the cloud storage server when the user client receives an upload permission of data as a result of the verification; And the cloud storage server receiving the encrypted data from the user client and updating the data.

The step of requesting confirmation of the duplication of data may include: transmitting, by the user client, its identifier information to the metadata server; Confirming a user client by checking the identifier information of the received user client, issuing and transmitting a session key to the user client; And the user client receives the session key, divides the data to be uploaded into blocks, and generates an encryption key and encryption data using a CE (Convergent Encryption) technology. The generated encryption key and encryption data are hashed And performing an XOR operation to generate a data identifier and transmitting the data identifier to the metadata server.

Checking whether the requested data is duplicated, and transmitting the result to the user client, the metadata server comparing the received data identifier with a previously stored data identifier and determining whether to store the data identifier; And the metadata server configures a data list by reconstructing only a data identifier that is not stored as a result of the determination, and transmits data for generating ownership verification data, which is connected to the reconfigured data list and an identifier not included in the data list, To the client.

Generating the ownership verification data through the XOR operation according to the received result and transmitting the generated ownership verification data to the metadata server may include transmitting the reconstructed data list received from the metadata server and the data for generating ownership verification data XORing with a data source owned by the user to extract a secret value; And XORing the extracted secret value and the held data by the user client to generate ownership verification data, and transmitting the generated ownership verification data to the metadata server.

Wherein the step of performing the ownership verification using the received ownership verification data and transmitting the result to the user client includes the steps of: XORing the ownership verification data by the metadata server to perform ownership verification; Upon completion, the user client can issue and transmit ownership of data for allowing upload of data not stored in the user client.

And transmitting the encrypted data to the cloud storage server when receiving the upload permission of the data as a result of the verification, when the user client receives the ownership of the data from the metadata server, And the verification data generated using the generated secret value can be transmitted to the cloud storage server.

The step of receiving encrypted data from the user client and updating the data comprises: updating and storing the encrypted data received from the user client in the cloud storage server, storing the encrypted data in the stored list, And transmits the generated data for generating the ownership verification data to the metadata server to synchronize the updated data list with the metadata server.

According to another aspect of the present invention, there is provided a data deduplication system for requesting confirmation of duplication of data to be uploaded to a metadata server and performing an XOR operation according to an acknowledgment result received from the metadata server And transmits the encrypted data to the metadata server when receiving the result of the ownership verification performed by the metadata server, and when the data is permitted to be uploaded as a result of the verification, the encrypted data is transmitted to the cloud storage server User client; Receiving a request for confirming duplication of data from the user client, transmitting the result to the user client, and receiving ownership verification data from the user client, performing ownership verification using the ownership verification data, A metadata server for transmitting a result to the user client; And a cloud storage server for receiving the encrypted data from the user client and updating the data in the storage list.

According to an aspect of the present invention, in a process of determining whether data is duplicated and verifying ownership in a metadata server by separating a metadata server and a cloud storage server and transmitting only data that is not duplicated to a cloud storage server, , It is possible to reduce the number of hash operations compared to the conventional method using the hash tree (merge tree) method, and it is possible to reduce the total amount of computation generated in the system There is an effect that can be reduced.

Also, it is possible to guarantee the confidentiality of data by providing encrypted data and to prevent duplication of data stored in the cloud storage server, thereby improving storage efficiency of the storage space.

The effects obtained in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments of the invention and, together with the specific details for carrying out the invention, And shall not be construed as limited to the matters described.
1 is a schematic configuration diagram of a data de-duplication system according to an embodiment of the present invention;
FIGS. 2 to 5 illustrate operation scenarios of a data de-duplication system according to an embodiment of the present invention;
6 illustrates an encryption method using a Convergent Encryption (CE) encryption method according to an embodiment of the present invention;
7 is a flowchart illustrating a data de-duplication method according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, in which: There will be. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. In addition, the term "Quot; and " part " refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

Before describing the data deduplication system according to the present embodiment, the operation process of the deduplication system in the prior art will be described.

The user client transmits the identifier of the data to be uploaded to the cloud storage server. The cloud storage server uses the received identifier to determine whether to store the data, and then returns the result. At this time, when the data is already stored in the cloud storage server, the cloud storage server assigns ownership of the data to the user client. On the other hand, when the data is not stored in the cloud storage server, the user client uploads the password data of the data. In this process, if an attacker who does not own the original data can falsify the identifier of the data already stored in the cloud storage server, there is a problem that the ownership of the data can be acquired without the source of the data. In order to solve this problem, the cloud storage server has to perform a process of determining whether the user client has the original of the data requested to be uploaded. In addition, in this process, the cloud storage server must be able to know whether the user client has a data source without acquiring the data source owned by the user client. This process is called the process of ownership verification.

The ownership verification process is a way to determine ownership of data without exposing the original data. For this purpose, when the data owner acquires the verification information generated by using a part of the data, a method of determining the authenticity of the data by using the verification information is needed. To this end, the common ownership verification process uses a data structure called Merkle Tree (also known as a hash tree). A merge tree is a method of hashing each leaf node, which is a plurality of data, into a plurality of pieces of data, splicing the data into two pieces, and then re-hashing the pieces of data until the last one piece of data is generated. Thus, the last node thus generated is referred to as a root node, and the data used in the root node is treated as an identifier of the corresponding data. This identifier is used to determine whether to store data. The cloud storage server determines whether the data is stored using the identifier. If the data is already stored, the cloud storage server performs the ownership verification with the user to verify whether the data is owned or not. The cloud storage server requests the user a random leaf node. The user client sends a sibling path, which is a set of sibling nodes, from the leaf node and the leaf node requested from the cloud storage server to the root node to the cloud storage server do. Then, the cloud storage server can create a root 'node' using the leaf node and the sibling path. If the root node matches the root 'node', the cloud storage server determines that the user owns the original data and ends the ownership verification process. On the other hand, a number of operations occur during the ownership verification process using Merkle Tree. The ownership verification method creates a Merkle Tree for each data to be uploaded. At this time, the generation of each merge tree generates several hash operations, and the number of hash operations increases sharply as the number of data to be uploaded increases. For example, if 1 GB of data is deduplicated in units of 32 KB, 29,150 hash operations are generated when 8 leaf nodes are generated for each block. As the number of hash operations increases, a large amount of processing time and resources are consumed. Therefore, a method of deduplicating data in a data deduplication system for solving such a problem will be described with reference to FIG. 1 through FIG.

1 is a schematic block diagram of a data deduplication system according to an embodiment of the present invention.

Referring to FIG. 1, a data deduplication system according to an embodiment of the present invention includes a user client 110, a metadata server 130, and a cloud storage server 150.

The user client 110 is connected to the metadata server 130 and the cloud storage server 150 via a network to perform communication. The user client 110 may communicate with the metadata server 130 to compare the encrypted data and perform a duplicate removal confirmation request process. The user client 110 may communicate with the cloud storage server 150 to upload the deduplicated data. At this time, the uploaded data may be encrypted data. The user client 110 may be a terminal used by a user and may be a personal computer, a tablet PC, a notebook, a net-book, an e-reader, a personal digital assistant an assistant, a portable multimedia player (PMP), an MP3 player, an MP4 player, or a handheld device such as a mobile phone, a smart phone, .

The metadata server 130 receives a deduplication request for the encrypted data from the user client 110 and transmits the result to the user client 110. Upon receiving the ownership verification data from the user client 110, It can perform the ownership verification using the ownership verification data and transmit the result to the user client 110. [

The cloud storage server 150 may receive the encrypted data from the user client 110 and update the data in the storage list.

The ID and the public key of the user client 110, the metadata server 130, and the cloud storage server 150 may be distributed in advance. The cloud storage server 150 may generate and store a data identifier for the encrypted data received from the user client 110 and may synchronize the data identifier of the encrypted data with the metadata server 130.

Hereinafter, a data de-duplication method in the data de-duplication system will be described.

FIGS. 2 to 5 illustrate operation scenarios of a data de-duplication system according to an embodiment of the present invention. FIG. 6 illustrates an encryption method using a Convergent Encryption (CE) encryption method according to an embodiment of the present invention. Fig.

According to the present embodiment, a password data deduplication system that performs lightweighted ownership verification in a cloud storage environment can be roughly implemented by encrypting and deduplicating data, verifying ownership of encrypted data, Uploading and updating the list of stored data to store the deduplicated data in the cloud storage server 150. [ In describing the present embodiment, the symbols to be used are defined as follows.

<Symbol definition>

: 참여 객체(

: 사용자,

: 메타데이터 서버(130),

: 스토리지 서버)·

: Participating object (

: user,

: The metadata server 130,

: Storage server)

: 원본 파일·

: Original file

: 중복되지 않은

의 수·

: Not duplicated

Number of

: 중복된

의 수·

: Duplicate

Number of

:

로부터 생성된 모든

의 수·

:

All

Number of

: CE(Convergent Encryption)로 생성된

의 암호화 키·

: Generated by Convergent Encryption (CE)

Encryption key

:

로 암호화된

·

:

Encrypted with

:

의 집합·

:

Set of

: 중복제거 된

의 집합·

: Duplicate removed

Set of

: 사용자

의 식별자·

: user

Identifier

: 메타데이터 서버(130)와 사용자

간의 세션키·

: The metadata server 130 and the user

Session key between

: *의 공개키·

: * Public key

: *의 개인키·

: * Private key

:

의 노드 수·

:

Number of nodes in

: 암호학적 해시 함수를 이용한 해시화·

: Hashing with cryptographic hash function

:

로 부터 생성된

·

:

&Lt; / RTI &gt;

:

로 부터 생성된

·

:

&Lt; / RTI &gt;

:

를 최초 업로드 한 사용자가 생성한

·

:

Created by the first uploading user

:

를 최초 업로드 한 사용자가 생성한 비밀값·

:

The secret value generated by the first uploading user

:

의 식별자·

:

Identifier

: 최초 업로드 한 사용자가 생성한 비밀값·

: The secret value generated by the first uploaded user

Hereinafter, an encryption data duplication elimination method in the encryption data duplication elimination system according to the present embodiment will be described in detail.

First, referring to FIG. 2, a process of the user client 110 requesting data encryption and duplication removal will be described.

FIG. 2 illustrates a process in which the user client 110 identifies itself to the metadata server 130, issues a session key, and confirms whether the data to be uploaded to the cloud storage server 150 is duplicated. In the encrypted data deduplication system, since the confidentiality of data is required, it is necessary to encrypt the transmitted data. Therefore, it is necessary to distribute the session key between the metadata server 130 and the user client 110.

)을 해시화하여

를 생성하고,

를 재차 해시화하여

를 생성하고,

를 암호화하여

를 생성하고,

를 해시화하여

를 각각 생성하고,

를 계산한 뒤, 세션키로 암호화하여 메타데이터 서버(130)로 전송한다(230, 240). 메타데이터 서버(130)는 전송받은 데이터를 복호화하여

를 얻은 후, 메타데이터 서버(130)에 보관된 데이터 식별자 목록과 비교하여 각 식별자들의 저장 여부를 판단한다. 한편,

은 CE(Convergent Encryption) 암호화 방식을 이용하여 생성될 수 있다. 도 6을 참조하면, CE(Convergent Encryption) 암호화 방식은 원본 파일(f)의 데이터 블록(

)을 해시 알고리즘을 통해 해시화한 값(

)을 대칭키 암호화 키로 사용하는 방식일 수 있으며,

는 아래의 수학식 1을 통해 생성될 수 있다. First, the user client 110 transmits its identifier (UID) to the metadata server 130 to inform the identity (210). After confirming the identity of the user client 110, the metadata server 130 distributes the session key encrypted with the public key of the user client 110 to the user client 110 (220). The user client 110 sends a data block (&quot;

) Is hashized

Lt; / RTI &gt;

Is again re-hashed

Lt; / RTI &gt;

By encrypting

Lt; / RTI &gt;

Hash

Respectively,

And transmits the encrypted data to the metadata server 130 (230, 240). The metadata server 130 decodes the received data

And compares it with the data identifier list stored in the metadata server 130 to determine whether to store the respective identifiers. Meanwhile,

May be generated using a Convergent Encryption (CE) encryption scheme. Referring to FIG. 6, the CE (Convergent Encryption) encryption method encrypts data blocks of the original file f

) Hash value obtained by the hash algorithm (

) As a symmetric key encryption key,

Can be generated by the following equation (1).

Next, a process of performing ownership verification will be described with reference to FIG.

3, the metadata server 130 transmits a result of duplicate checking to the user client 110, and the user client 110 generates ownership verification data according to the received result, . In the encryption data deduplication system according to the present embodiment, the user client 110 checks the data duplication using the identifier of the data to be uploaded, and the user client 110 actually checks the duplication of the original data It is necessary to carry out a process of judging whether or not the user is doing.

를 생성한다(310). 또한,

에 포함되지 않은

와 연결되어 있는

를 사용자 클라이언트(110)로 전송한다(320). 사용자 클라이언트(110)는 전송 받은

와 자신이 보유한 데이터 원본(

)을 이용하여 소유권 검증 데이터 생성에 이용되는

를 생성한다(330).

의 생성 과정은 아래의 수학식 2와 같다.Accordingly, the metadata server 130 catalogs identifiers that are not stored in the list of identifiers transmitted by the user

(310). Also,

Not included in

Connected with

To the user client 110 (320). The user client 110 sends

And your own data source (

) To be used for generating ownership verification data

(330).

Is expressed by the following equation (2).

,

와 자신이 생성한

,

를 계산하여 소유권 검증 데이터(

)를 생성한다. 사용자 클라이언트(110)는 생성된 소유권 검증 데이터를 메타데이터 서버(130)로 전송한다(340). 메타데이터 서버(130)는 소유권 검증 데이터와 클라우드 스토리지 서버(150)에 저장된 데이터를 각각 연산하여 소유권 검증을 수행한다(350). 이때, 소유권 검증 과정은 아래의 수학식 3과 같다.The user client (110)

,

And yourself

,

To calculate ownership verification data (

). The user client 110 transmits the generated ownership verification data to the metadata server 130 (340). The metadata server 130 performs ownership verification by calculating the ownership verification data and the data stored in the cloud storage server 150 (350). At this time, the ownership verification process is expressed by Equation (3) below.

If Equation (3) is satisfied, the metadata server 130 determines that the user client 110 owns the original data, and issues the ownership of the data to the user client 110 (360). Then, the metadata server 130 may permit uploading of data not stored in the cloud storage server 150. [

Next, referring to FIG. 4, a description will be made of a process in which the user client 110 uploads the duplicated cipher data and generates data for generating ownership verification data.

4 is a flowchart illustrating a method of generating and uploading data for encrypting data blocks and data for generating ownership verification data by the user client 110 to the cloud storage server 150. The cloud storage server 150 stores the data blocks and generates ownership verification data The data stored in the cloud storage server 150 can be updated through transmission to the metadata server 130. [

After the ownership verification process of FIG. 3 is completed, the user client 110 performs a process of uploading data that is not stored in the cloud storage server 150. Through this process, the user client 110 becomes the first uploading user client 110 with respect to the data to be uploaded, and then generates data for generating ownership verification data for performing the ownership verification of the uploading user client 110 It is necessary to perform a process of transmitting to the cloud storage server 150. [ Accordingly, the user client 110 generates data to be uploaded and data for generating ownership verification data to be transmitted to the cloud storage server 150, and the cloud storage server 150 has a process of storing and modulating the data.

)을 생성하고,

와

를 이용하여 소유권 검증 데이터 생성용 데이터(

)를 생성한다(410). 이때,

는 아래의 수학식 4를 통해 생성될 수 있다.First, the user client 110 sends a secret value (

),

Wow

Data for generating ownership verification data (

(410). At this time,

Can be generated by the following equation (4).

와

를 계산하여

를 생성하고,

,

,

, UID와 함께 암호화하여 클라우드 스토리지 서버(150)로 전송한다(420). 클라우드 스토리지 서버(150)는

를 저장한 후, 재차 해시화하여 소유권 검증 데이터 생성을 위한 데이터(

)를 생성한다(430). 이때,

는 아래의 수학식 5를 통해 생성될 수 있다.The user client (110)

Wow

To calculate

Lt; / RTI &gt;

,

,

, And transmits it to the cloud storage server 150 by encrypting it together with the UID (420). The cloud storage server 150

And then re-hashes the data to generate ownership verification data (

(430). At this time,

Can be generated by the following equation (5).

Lastly, referring to FIG. 5, a description will be given of data transmission for generating ownership verification data and a stored data list birth process.

FIG. 5 shows a process of updating the data list after the uploading of data to the cloud storage server 150 is completed.

와 사용자 클라이언트(110)가 전송한

를 연산하여

를 생성하고, 사용자 클라이언트(110)가 전송한

,

, UID를 메타데이터 서버(130)로 전송하며, 메타데이터 서버(130)는 전송 받은 데이터를 저장한다(510, 520). 이때, 메타데이터 서버(130)로 전송되는 데이터는 업로드가 완료된 데이터의 식별자일 수도 있다. The cloud storage server (150)

And the user client 110

By the operation

And transmits the data transmitted by the user client 110

,

, And transmits the UID to the metadata server 130. The metadata server 130 stores the received data (510, 520). At this time, the data transmitted to the metadata server 130 may be an identifier of the uploaded data.

7 is a flowchart illustrating a data de-duplication method according to an embodiment of the present invention.

Referring to FIG. 7, first, the user client 110 requests confirmation of duplication of data to be uploaded to the metadata server 130 (S710). At this time, the user client 110 transmits its identifier information to the metadata server 130, and the metadata server 130 confirms the identifier information of the received user client 110, 110), and can issue a session key to the user client 110 and transmit the session key. Also, the user client 110 receives the session key, divides the data to be uploaded into blocks, and generates an encryption key and encryption data using a CE (Convergent Encryption) technology. The generated encryption key and encryption data And XOR operation to generate a data identifier and transmit the data identifier to the metadata server 130.

The metadata server 130 checks whether the requested data is duplicated and transmits the result to the user client 110 (S720). At this time, the metadata server 130 compares the received data identifier with a previously stored data identifier to determine whether to store the data identifier, and the metadata server 130 reconstructs only the data identifier that is not stored as a result of the determination, And transmits data for generating ownership verification data linked to the reconfigured data list and an identifier not included in the data list to the user client 110. [

The user client 110 generates ownership verification data by XOR operation according to the received result and transmits it to the metadata server 130 in operation S730. At this time, the user client 110 performs XOR operation on the reconstructed data list received from the metadata server 130 and data for generating ownership verification data with a data source owned by the user client 110 to extract a secret value, (110) generates ownership verification data by XORing the extracted secret value and the held data, and transmits the generated ownership verification data to the metadata server (130).

The metadata server 130 performs ownership verification using the received ownership verification data, and transmits the result to the user client 110 (S740). At this time, the metadata server 130 XORs the ownership verification data to perform ownership verification. Upon completion of the ownership verification, the metadata server 130 acquires the ownership of data that is permitted to upload data that is not stored in the user client 110 Can be issued and transmitted.

When the user client 110 receives the data upload permission as a result of the verification, the encrypted data is transmitted to the cloud storage server 150 in operation S750. In this case, when the user client 110 receives the ownership of the data from the metadata server 130, the user client 110 transmits the verification data generated using the encrypted data and the secret value generated by the user client 110 to the cloud storage server 150 Lt; / RTI &gt;

The cloud storage server 150 receives the encrypted data from the user client 110 and updates the data (S760). At this time, the cloud storage server 150 stores encrypted data received from the user client 110, generates data for generating ownership verification data through hashing and XOR operation of the encrypted data, Data for generating the generated ownership verification data may be transmitted to the data server 130, and the updated data list may be synchronized with the metadata server 130.

According to the present invention, the metadata server 130 and the cloud storage server 150 are distinguished from each other, and the metadata server 130 performs data redundancy determination and ownership verification, and the cloud storage server 150 includes redundant In the process of transmitting only the data that is not transmitted, the ownership verification generated in the data deduplication process is performed based on the XOR to remove the redundant data. As compared with the conventional method using the hash tree (merch tree) method, It is possible to reduce the total amount of computation generated in the system.

Also, it is possible to guarantee the confidentiality of the data by providing the encrypted data, and to prevent the data stored in the cloud storage server 150 from being duplicated, the storage efficiency of the storage space can be improved.

The methods according to embodiments of the present invention may be implemented in an application or implemented in the form of program instructions that may be executed through various computer components and recorded on a computer readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be ones that are specially designed and configured for the present invention and are known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the specification contains many features, such features should not be construed as limiting the scope of the invention or the scope of the claims. In addition, the features described in the individual embodiments herein may be combined and implemented in a single embodiment. On the contrary, the various features described in the singular embodiments may be individually implemented in various embodiments or properly combined.

Although the operations are described in a particular order in the figures, it should be understood that such operations are performed in a particular order as shown, or that all described operations are performed in a series of sequential orders, or to obtain the desired result. In certain circumstances, multitasking and parallel processing may be advantageous. It should also be understood that the division of various system components in the above embodiments does not require such distinction in all embodiments. The above-described application components and systems can generally be packaged into a single software product or multiple software products.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings.

110: User Client (User)
130: Metadata Server
150: Cloud Storage Server (Storage Server)

Claims (8)

delete delete delete A method for deduplicating data in a deduplication system including a user client, a metadata server, and a cloud storage server,
Requesting the user client to confirm duplication of data to be uploaded to the metadata server;
Checking whether the metadata requested by the metadata server is duplicated, and transmitting the result to the user client;
Generating the ownership verification data through the XOR operation according to the received result and transmitting the generated ownership verification data to the metadata server;
Performing the ownership verification using the received ownership verification data and transmitting the result to the user client;
Transmitting encrypted data to the cloud storage server when the user client receives an upload permission of data as a result of the verification; And
The cloud storage server receiving encrypted data from the user client and updating the data,
Wherein the step of requesting confirmation of the duplication of the data comprises:
Transmitting, by the user client, its identifier information to the metadata server;
Confirming a user client by checking the identifier information of the received user client, issuing and transmitting a session key to the user client; And
The user client receives the session key, divides the data to be uploaded into blocks, generates an encryption key and encryption data using CE (Convergent Encryption) technology, and hashizes the generated encryption key and encryption data Generating a data identifier by performing an XOR operation and transmitting the data identifier to the metadata server,
Checking whether the requested data is duplicated, and transmitting the result to the user client,
Comparing the received data identifier with a previously stored data identifier and determining whether to store the data identifier; And
The metadata server configures a data list by reconstructing only the data identifier that is not stored as a result of the determination, and transmits data for generating ownership verification data, which is connected to the reconfigured data list and the identifier not included in the data list, , The method comprising:
Generating the ownership verification data through the XOR operation according to the received result and transmitting the generated ownership verification data to the metadata server,
Extracting a secret value by XORing the reconstructed data list received from the metadata server and data for generating ownership verification data with a data source owned by the user client; And
The user client XORing the extracted secret value and the held data to generate ownership verification data, and transmitting the generated ownership verification data to the metadata server. 5. The method of claim 4,
Performing ownership verification using the received ownership verification data, and transmitting the result to the user client,
The metadata server performs XOR operation on the ownership verification data to perform ownership verification. When the ownership verification is completed, the meta data server issues ownership of data that is allowed to upload data that is not stored in the user client, Removal method. 6. The method of claim 5,
Wherein the step of transmitting the encrypted data to the cloud storage server comprises the steps of:
When the user client receives the ownership of data from the metadata server, transmits the verification data generated using the encrypted data and the secret value generated by the user client to the cloud storage server. The method according to claim 6,
Wherein the receiving the encrypted data from the user client and updating the data comprises:
The cloud storage server updates and stores the encrypted data received from the user client in a stored list and generates data for generating ownership verification data through hashing and XOR operation of the encrypted data, And transmitting the generated data for generating ownership verification data to the metadata server to synchronize the updated data list with the metadata server. delete

Images