KR20210109299A - Method for distributed de-identification of large graph data - Google Patents

Abstract

The present invention relates to distributed processing de-identification for large-scale graph data. More specifically, the provided distributed processing de-identification method for large-scale graph data based on distributed processing makes it impossible to know contents of personal information for large-capacity RDF graph data used for public data, personal medical data, and social relation data.

Description

{Method for distributed de-identification of large graph data}

The present invention relates to distributed processing de-identification of large-scale graph data, and more particularly, to prevent the content of personal information from being known about large-capacity RDF graph data used for public data, personal medical data, and social relation data. It relates to a distributed processing de-identification method for large-scale graph data based on distributed processing.

Recently, large-capacity data in RDF format has been actively created and distributed on the web. These data are widely used in the format of government public data, personal medical data, and social relation data, but distribution is impossible because personal information is included in the distribution. Therefore, for distribution, a de-identification process that removes an individual's identity from the personal information contained in the RDF is required.

For de-identification processing, models such as k-anonymity, l-diversity, and t-proximity have been studied. However, the only case of applying the de-identification model to RDF graph data is k-anonymity. Therefore, we want to protect personal information by focusing on the l-diversity model, which is stronger than k-anonymity. At this time, the Anatomy algorithm among the implementation algorithms of the de-identification model is widely used to implement the l-diversity model, and there is no generalization or categorization processing, so the usefulness of the data is also high. However, it has disadvantages that it is difficult to apply the Anatomy algorithm to RDF data with different data structures, and the time required to process large-scale data is very large.

US 10-2015-0120443 A

The present invention was devised to solve such a problem, and it is to provide a distributed processing de-identification method for large-scale graph data based on distributed processing by applying the l-diversity model using the Anatomy algorithm to RDF data. do.

In order to achieve the above object, there is provided a distributed processing de-identification method for large-scale graph data according to the present invention, comprising the steps of: (a) converting RDF data into an attribute table of RDB; (b) removing the attribute table of the identifier (ID) after classifying it into an identifier (ID), a quasi-identifier (QI), and a sensitive attribute (SA) in the attribute table converted in step (a); (c) classifying the quasi-identifier attribute data into a sensitive attribute bucket (SABucket) using an attribute key having l-diversity value of the sensitive attribute (SA) separated in step (b), and designating it as a group; and (d) combining the groups designated in step (c) after each of the groups are merged into the quasi-identifier (QI) table and the sensitive attribute table.

The method further includes dividing the RDF data into at least one partition before the step (a).

The designation as a group in step (c) includes: (c1) moving each quasi-identifier attribute data sequentially from the sensitive attribute bucket (SABucket) to a group; (c2) determining whether the quasi-identifier attribute data moved in step (c1) includes data greater than or equal to the l-diversity value; (c3) generating a next group when the quasi-identifier attribute data includes data greater than or equal to l-diversity value as a result of the determination; and (c4) if quasi-identifier attribute data less than the l-diversity value remains in the sensitive attribute bucket after repeating steps (c1) to (c2), the quasi-identifier attribute data less than the l-diversity value adding to an arbitrary group.

The quasi-identifier (QI) is an attribute that can identify a specific individual when combined with two or more other attribute values.

The sensitive attribute SA is a specific attribute value that can be a target for attack or identification.

Another aspect of the present invention for achieving the above object is a distributed processing de-identification apparatus for large-scale graph data, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions stored in the at least one memory are configured to: (a) convert RDF data into an attribute table of RDB; converting to; (b) removing the attribute table of the identifier (ID) after classifying it into an identifier (ID), a quasi-identifier (QI), and a sensitive attribute (SA) in the attribute table converted in step (a); (c) classifying the quasi-identifier attribute data into a sensitive attribute bucket (SABucket) using the attribute key having the l-diversity value of the sensitive attribute (SA) separated in step (b), and designating it as a group again ; and (d) the group designated in step (c) is combined into the quasi-identifier (QI) table and the sensitive attribute table, respectively, and then the step of combining is executed.

Another aspect of the present invention for achieving the above object is a computer program stored in a non-transitory storage medium for distributed processing de-identification of large-scale graph data, which is stored in the non-transitory storage medium, and is (a) converting the RDF data into an attribute table of the RDB; (b) removing the attribute table of the identifier (ID) after classifying it into an identifier (ID), a quasi-identifier (QI), and a sensitive attribute (SA) in the attribute table converted in step (a); (c) classifying the quasi-identifier attribute data into a sensitive attribute bucket (SABucket) using the attribute key having the l-diversity value of the sensitive attribute (SA) separated in step (b), and designating it as a group again ; and (d) the group designated in step (c) being combined into the quasi-identifier (QI) table and the sensitive attribute table, respectively, and then the combining step is executed.

According to the present invention, however, the Anatomy algorithm is applied to RDF data having different data structures, and the time required for large-scale data processing is reduced.

1 is a block diagram showing a distributed processing de-identification apparatus for large-scale graph data according to the present invention.
2 is a flowchart illustrating a distributed processing de-identification method for large-scale graph data according to the present invention.
3 is a flowchart illustrating a method of designating a group in the distributed processing de-identification method for large-scale graph data according to FIG. 2 .
FIG. 4 is a diagram for explaining step S110 in the distributed processing de-identification method for large-scale graph data according to FIG. 2 .
5 is a diagram illustrating steps S120 to S130 in the distributed processing de-identification method for large-scale graph data according to FIG. 2 .
6 is a view for explaining step S140 in the distributed processing de-identification method for large-scale graph data according to FIG. 2 .
7 is a view for explaining step S150 in the distributed processing de-identification method for large-scale graph data according to FIG. 2 .
8 to 10 are diagrams for explaining experimental evaluation for verifying distributed processing de-identification for a large-scale graph according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, the terms or words used in the present specification and claims should not be construed as being limited to conventional or dictionary meanings, and the inventor should properly understand the concept of the term in order to best describe his invention. Based on the principle that it can be defined, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the configuration shown in the embodiments and drawings described in the present specification is only the most preferred embodiment of the present invention and does not represent all of the technical spirit of the present invention, so at the time of the present application, various It should be understood that there may be equivalents and variations.

1 is a block diagram illustrating a distributed processing de-identification apparatus for large-scale graph data according to the present invention. Distributed processing de-identification apparatus 100 for large-scale graph data according to the present invention includes a processor 110, a non-volatile storage unit 120 for storing programs and data, a volatile memory 130 for storing a program being executed, It consists of a communication unit 140 for performing communication with other devices, and a bus that is an internal communication path between these devices. The running program may include a device driver, an operating system, and various applications. In the present invention, the de-identified application 310 is a computer device on which the running program is executed. And, although not shown, the computer device includes a power supply unit such as a battery.

2 is a flowchart illustrating a distributed processing de-identification method for large-scale graph data according to the present invention.

First, RDF data is divided into at least one partition for distributed processing (S100).

Then, the RDF data divided into at least one partition in step S100 is converted into an attribute table of the RDB (S110). RDF is basically described as a triple model of subject, predicate, and object. The subject means the data to be expressed, and the description means the description of the subject or the relationship between the subject and the purpose. And purpose means the content or value of the description, and each content can be described through URI.

FIG. 4 is a view for explaining step S110 according to FIG. 2 . The left side shows RDF data, and the right side shows the attribute table of the converted RDB. As shown in Fig. 4, in the left RDF data, the subject is 'patient', and the predicate is 'name', 'resident registration number', 'name', 'address', 'disease name', and object ) are 'Chul-Sik Moon', '630128-0050352', 'Male', 'Seongnam-si, Gyeonggi-do', and 'Diabetes', and the right side is converted into an attribute table for RDF data. Here, conversion of RDF data into attribute table of RDB can be done using Apache Jena Framework, and Jena in the present invention helps to convert triples of RDF into attribute table like RDB.

Next, the attribute table converted in step S110 is divided into identifiers (IDs), quasi-identifiers (QIs), and sensitive attribute (SA) tables (S120).

Then, the identifier (ID) table is removed from the identifier (ID), the quasi-identifier (QI), and the sensitive attribute (SA) table identified in step S120 (S130). 5 is a diagram to help explain steps S120 to S130. Attributes 'name' and 'resident registration number' are divided by identifiers (IDs), 'gender' and 'address' are quasi-identifiers (QI), and 'ill name' ' is classified as a private attribute (SA), and the identifier (ID) is removed from the attribute table. Here, an identifier (ID) is an attribute that can identify an individual and refers to all information that can be matched 1:1, for example, resident number, phone number, email, name, account number, NRI photo, genetic information, etc. , encrypted values are also classified as identifiers and must be deleted unconditionally in case of de-identification. And although a quasi-identifier (QI) is not an identifier by itself, it is a property that can be used to indirectly infer a specific individual through combination with other data. For example, it may include the name of the city where you live, your weight, your blood type, and your gender. Lastly, the sensitive attribute (SA) is an attribute that can reveal an individual's privacy, such as disease name, deposit balance, card payment amount, etc., and is a target attribute mainly measured when analyzing data. It is a value that preserves data values in , and is a specific attribute value that can be targeted for attack or identification.

Next, by using the attribute key having the l-diversity value of the sensitive attribute (SA) classified in step S120, the quasi-identifier attribute data is classified into the sensitive attribute bucket (SAbucket) classification, and this is designated as a group (S140). Here, FIG. 6 is a diagram to help explain step S140. The attribute keys having the l-diversity value are 'AIDS', 'osteoporosis', 'cancer', and 'diabetes' as shown, and having the l-diversity value. It shows that the quasi-identifier attribute data is divided into sensitive attribute buckets (SAbucket) according to the attribute key, and the quasi-identifier attribute data of the sensitive attribute bucket (SAbucket) is moved to group 1 and group 2. For reference, the process of designating a group according to step S140 will be described again with reference to FIG. 3 below.

Next, the group designated in step S140 is combined with the quasi-identifier table and the sensitive attribute table of step S120 (QIT, ST) and then combined (S150). Here, the groups are referred to as an equivalence class (EC). FIG. 7 is a diagram for explanation of step S150. Referring to FIG. 7, the equivalence class and group number are one quasi-identifier table (QIT; QI Table). Similarly, the group number and the number of sensitive attributes and the number of sensitive attributes in the group are combined into a sensitive attribute table (ST; SA Table), where the QIT created by combining the group with the quasi-identifier table and the group is sensitive Looking at the ST generated by combining with the attribute table, only the identifier (ID) of the RDF data in step S110 is lost, and unlike the de-identifier model, the QIT is like the original, which is not generalized or categorized. It cannot be identified, and there are two separate sensitive attribute l-diversity values in group 1, and three separate sensitive attribute l-diversity values exist in group 2 as shown in Fig. 6. Therefore, the condition of the l-diversity model is On the other hand, as shown in the lower part of Fig. 7, QIT and ST are distributed through a Turtle-type black node using the Apache Jena Framework, so that each group can be identified and referenced when using data later.

3 is a flowchart showing a group designation process according to the classification of the sensitive attribute bucket (SAbucket) in step S140 according to FIG. 2 .

Referring to FIG. 3 , first, one quasi-identifier attribute data is sequentially moved to a group in a sensitive attribute bucket (SAbucket) (S141). Then, it is determined whether the quasi-identifier attribute data moved in step S141 includes data greater than or equal to the l-diversity value (S142).

As a result of the determination (S142), when the quasi-identifier attribute data includes data greater than or equal to the l-diversity value, the next group is generated (S143).

Thereafter, after repeating steps S141 to S143, if quasi-identifier attribute data less than the l-diversity value remains in the sensitive attribute bucket, the quasi-identifier attribute data less than the l-diversity value is added to an arbitrary group (S144). ).

The following is an [Example] of the present invention, and Spark is used in the [Example], and to configure the Anatomy algorithm as in the present invention, it is similar to the existing Anatomy, but additionally RDD (Resilient Distributed Data) must be used, [ Example] shows the pseudo code of the algorithm implemented through this RDD.

[Example]

Data: HDFS RDF file, L value

Result: De-identified RDF file

// Lines 1-6 are the steps to convert RDF to tuple

1. RDF is divided into at least as many partitions as there are executors.

2. tripleList = List of tuples converted from RDF Triple.

3. tripleRDD = parallelized tripleList.

4. mapTripleRDD = Mapping ripleRDD to {SA, SA's Tuple}.

5. SASet = Set of SA sets in mapTripleRDD.

6. SABucket; bucketCnt = 0; groupCnt = 0.

// Lines 7-8 are the creation stage of buckets filled with SA.

7. For each loopSA in SASet

8. SABucket _bucketCnt = Filter and save the same value as loopSA among the keys of mapTripleRDD.

9. bucketCnt = bucketCnt + 1.

// Lines 10-19 are the creation stage of groups filled with tuples.

10. While If there are at least L non-empty SABucket

11. Sort SABuckets by the number of members of SABuckets.

12. new groupBucket

13. for idx=1 to L

14. tuple = Get value from idx index of SABucket.

15. Add the first value of the tuple and groupCnt to groupBucket.

16. Remove the value added to groupBucket from SAbucket.

17. groupCnt = groupCnt + 1.

18. For each bucket in non-empty SABuckets.

19. Add the first value of Bucket and a random numeric value within the range of groupCnt to groupBucket.

// Lines 20-22 are the split steps of QIT and ST.

20. allTuples = parallised groupBucket.

21. Create a QI partition by extracting the required attributes from QIT = allTuples, and then merge all partitions.

22. Create an ST partition by extracting the necessary attributes from ST = allTuples and then merge all partitions.

// Lines 23-24 are the final writing steps of the RDF consisting of QIT and ST.

23. For each cnt in groupCnt

24. Add QITcnt, SAcnt Triple through Jena.

As in the [Example] above, the file is divided into multiple partitions through one line so that multiple executors can access it by dividing it. And in lines 2-4, tripleRDD is created by combining Objects into one tuple as the property value of Predicate in "Professor" Subject. Then, the tuple creates a mapTripleRDD with the SA of tripleRDD as the Key value and the corresponding tuple as the Value. Then, in lines 5-9, we create a SAset that removes duplicates from the entire SA, and iterate over all elements of the SASet. In this iteration, each SA of SASet is called loopSA, and the values of loopSA matching the key of mapTripleRDD are stored in the bucketCnt index of SABucket in the form of a list. And bucketCnt is incremented by 1 until the iteration ends. The next line 10-19 is the step of dividing the values of SABucket into each group. First, lines 10-17 are repeated until the total number of tuples falls below the L value. Through this, all tuples of SABucket are sequentially grouped and stored by groupCnt index. The number of tuples per group is L. And in lines 18-19, undivided tuples are assigned to a random group. In lines 20-22, the divided groups are divided into QIT and ST based on the quasi-identifier, sensitive attribute, and group number and integrated into one table. After that, in lines 23-24, using Jena, RDF as a Blank Node ends the process.

8 to 10 are diagrams for explaining experimental evaluation for verifying distributed processing de-identification for a large-scale graph according to the present invention.

As an experimental evaluation, the results obtained from the Anatomy algorithm using large-scale RDF data were evaluated using both IneMemory and Spark, and the respective execution times were compared.

The RDF used for the experiment was created using the LUBM data generator to generate benchmark data. These generators can specify the number of universities that will go into the LUBM data. Therefore, in this experiment, LUBMs including 10, 50, 100, 300, and 700 universities were created in Turtle format. The size of the input data and the number of triples are shown in FIG. 8 .

It is a cluster system consisting of a total of 5 workers as hardware resources, and each worker is composed of 24Gb memory and Intel (R) Xeon (R) CPU E3-1220 V2 @ 3.10 GHz. Also, for the experiment, only data of professor type such as {fullProfessor, AssociateProfessor, assistantProfessor} information were used in the data. In the de-identification process, the professor's ID was deleted, and the properties actually used are { name, researchInterest, undergraduateDegreeFrom, masterDegreeFrom, doctorDegreeFrom }.

In the experiment, Java InMemory de-identification and Java Spark distributed cluster were compared. First, when the size of the input data LUBM is small, the working time of InMemory was faster than Spark. However, as the size of LUBM increases, the calculation speed of InMemory sharply decreases. Also, data processing is impossible from LUBM-300 due to lack of hardware resources. Therefore, by using Spark cluster, resource shortage of single hardware and degradation of processing speed were overcome. Spark consists of a driver that manages all processes and an executor that performs actual tasks. It is automatically classified among Workers when Spark is executed. Among them, the driver divides all tasks that occur during execution and allocates them to the executor, and this allocation time is disadvantageous compared to InMemory for low-capacity data. However, as the size of the data increases, the processing efficiency increases compared to the disadvantage of the division time, which is advantageous.

FIG. 9 is a diagram in which various options are added, including the number of cores of the CPU and the amount of memory to be used using Spark-submit. This means that as the size of the RDF file increases, only one executor is executed per worker to increase the amount of memory allocated to one executor as much as possible, and the driver uses all one worker. In addition, when processing data in Spark, files are stored in Hadoop's storage system HDFS, which is an HDD-based distributed processing platform, and then accessed. However, in the case of InMemory system, access time is not included in the result graph in order to make the comparison fair because access time is relatively longer because it accesses a normal file.

Figure 10 (a) compares Spark and InMemory using 4 workers. As can be seen from the graph, InMemory performs better on small data, but large files do not run despite using more than 10 Gb of Java heap memory. And Spark doesn't show stable results for small amounts of data. However, in (b), which processes data using Spark, it can be seen that the processing speed increases as the number of workers is added for large data. Although the speed does not decrease in direct proportion to the increased number of workers, it exceeds the performance of a single CPU system and can scale horizontally. In addition, faster de-identification of large RDF data can be supported by replacing the worker with a specification superior to that of the cluster used in the current experiment.

In this experimental evaluation, we proposed a platform through the implementation of the Anatomy algorithm of l-diversity de-identification model using Spark to use large-scale RDF. In addition, the experimental results showed that the Spark-based Anatomy algorithm showed significant advantages in the relatively large RDF data set, which showed that the execution time decreased even as the data size increased.

However, although the Anatomy algorithm has been used with the goal of de-identification and preserving the usefulness of the final data, it is still vulnerable to several attacks, including inference attacks. In order to solve this problem, we would like to proceed with a study to newly apply an algorithm that satisfies the t-proximity model and preserves usefulness as much as possible like Anatomy. You can also value l-diversity algorithms for data sets that need to be dynamically modified to conduct research.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited thereto, and the technical idea of the present invention and the following by those of ordinary skill in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalents of the claims to be described.

100: Distributed processing de-identification device for large-scale graph data
110: processor
120: storage
130: memory
140: communication department
310: non-identifying application

Claims (7)

As a distributed processing de-identification method for large-scale graph data,
(a) converting the RDF data into an attribute table of the RDB;
(b) removing the attribute table of the identifier (ID) after classifying it into an identifier (ID), a quasi-identifier (QI), and a sensitive attribute (SA) in the attribute table converted in step (a);
(c) classifying the quasi-identifier attribute data into a sensitive attribute bucket (SABucket) using the attribute key having the l-diversity value of the sensitive attribute (SA) separated in step (b), and designating it as a group again ; and
(d) combining the groups designated in step (c) after each of them are combined into the quasi-identifier (QI) table and the sensitive attribute table
Distributed processing de-identification method for large-scale graph data including The method according to claim 1,
before step (a)
Partitioning the RDF data into at least one partition
Distributed processing de-identification method for large-scale graph data comprising further. The method according to claim 1,
Designation as a group in step (c) is,
(c1) sequentially moving one quasi-identifier attribute data to a group in the sensitive attribute bucket (SABucket);
(c2) determining whether the quasi-identifier attribute data moved in step (c1) includes data greater than or equal to the l-diversity value;
(c3) generating a next group when the quasi-identifier attribute data includes data greater than or equal to l-diversity value as a result of the determination; and
(c4) If quasi-identifier attribute data less than l-diversity value remains in the sensitive attribute bucket after repeating steps (c1) to (c2), quasi-identifier attribute data less than l-diversity value Steps to add to any group
Distributed processing de-identification method for large-scale graph data, characterized in that it comprises a. The method according to claim 1,
The quasi-identifier (QI) is an attribute that can identify a specific individual when combined with two or more other attribute values
Distributed processing de-identification method for large-scale graph data, characterized by The method according to claim 1,
The sensitive attribute (SA) is a specific attribute value that can be a target for attack or identification
Distributed processing de-identification method for large-scale graph data, characterized by As a distributed processing de-identification device for large-scale graph data,
at least one processor; and
at least one memory for storing computer-executable instructions;
The computer-executable instructions stored in the at least one memory are executed by the at least one processor,
(a) converting the RDF data into an attribute table of the RDB;
(b) removing the attribute table of the identifier (ID) after classifying it into an identifier (ID), a quasi-identifier (QI), and a sensitive attribute (SA) in the attribute table converted in step (a);
(c) classifying the quasi-identifier attribute data into a sensitive attribute bucket (SABucket) using the attribute key having the l-diversity value of the sensitive attribute (SA) separated in step (b), and designating it as a group again ; and
(d) combining the groups designated in step (c) after each of them into the quasi-identifier (QI) table and the sensitive attribute table
Distributed processing de-identification device for large graph data that makes it run. A computer program stored in a non-transitory storage medium for distributed processing de-identification of large-scale graph data,
It is stored in a non-transitory storage medium, and is
(a) converting the RDF data into an attribute table of the RDB;
(b) removing the attribute table of the identifier (ID) after classifying it into an identifier (ID), a quasi-identifier (QI), and a sensitive attribute (SA) in the attribute table converted in step (a);
(c) classifying the quasi-identifier attribute data into a sensitive attribute bucket (SABucket) using the attribute key having the l-diversity value of the sensitive attribute (SA) separated in step (b), and designating it as a group again ; and
(d) combining the groups designated in step (c) after each of them are combined into the quasi-identifier (QI) table and the sensitive attribute table
A computer program stored in a non-transitory storage medium for distributed processing de-identification of large-scale graph data, including instructions to be executed.

Images