KR101441000B1 - A parallel change detection method for triple data - Google Patents

Abstract

The present invention relates to a method of detecting a change in triple data based on parallel processing in which a source / target triple data is searched for a triple added / deleted in a target triple based on source triple data. ; (b) aligning the triples in the triple group; (c) comparing the number of source triples and target triples in the triple group; (d) comparing the source triple and the target triple if the numbers are the same; (e) compares all of the source triples and the target triples in the group, if they are different, to compute the added triple list and the deleted triple list; (f) determining whether the number of source triples or the number of target triples is 0 if the numbers are different, and if not, performing the step (e); And (g) if the number of source triples or the number of target triples is zero, including a source triple or a target triple in the added triple list or the deleted triple list.
According to the change detection method for the triple data as described above, the RDF triple data comparison can be processed in parallel. Thus, it is possible to detect the RDF triple change even in a distributed environment composed of a plurality of computers, have.

Description

Technical Field [0001] The present invention relates to a parallel change detection method for triple data,

The present invention relates to a change detection method for triple data based on parallel processing for finding triples added and deleted triples on one side of two triple sets generated at different points in time.

Typically, a representative model of triple data is RDF data. Resource Description Framework (RDF) Triple indicates what information is described in three narrative units: Subject, Predicate, and Object.

For example, the following statement is expressed in the RDF scheme shown in FIG.

"The genre of singer 015B music is ballad."

The Subject indicates the subject to be described, and the Predicate indicates which attribute of the subject is to be described. The object (object) indicates the value of the attribute. This unit is usually referred to as a triple because it describes the subject, the predicate, and the object.

There are various ways of expressing the RDF model. The present invention deals with the N3 format. N3 or Notation 3 is one of the RDF (Resource Description Framework) notation schemes, which displays triples in a simpler form than the XML notation.

Figure 2 is part of the actual Korean DBpedia infobox.

Information of genre, activity period, and member of 015B is in the form of Subject, Predicate, and Object. Subject, Predicate, and Object are separated by tabs or whitespace, and each triple is separated by a period (.).

The problem of change detection is to find data added to the other side and deleted data based on either one of two arbitrary data. Therefore, the change detection problem for RDF triple data is the problem of finding triples added and deleted triples based on either RDF triple data.

Figure 3 shows an example of a change detection problem for an RDF triple.

As shown in FIG. 3, there are two types of input RDF triple data: a source and a target. If the two RDF triple data are input, the output of the RDF change detection is the "deleted triple list" and the "added triple list" shown in the right part of FIG. In the case of <Rejuvenation, Affiliation, and Chungcheongdo>, it is a triple to be deleted because it is a triple that exists in the source but does not exist in the target. Also, in the case of <acting, belonging to, and Sejong City> and <Sookwon, belonging to, and Sejong City>, it is a triple in the target but not in the source. So it is an added triple.

FIG. 4 shows the results of Structural diff and Semantic diff as a result of detecting changes in version A (version A) and version B (version B) of RDF triple data.

There are two types of change detection methods for RDF triple data. Structural diff-based schemes are a way to identify add / delete triples without taking into account property inference. The semantic diff-based method is a method of identifying addition / deletion triples by considering that triples that can be generated through inference by the property are present.

Structural diffs for version A (Version A) and version B (version B) The addition / deletion triple as a result of change detection can be defined as the difference set operation for the set as follows.

[Equation 1]

added (A, B) = B - A = B - (A? B)

removed (A, B) = A - B = A - (A? B)

Semantic diff In the case of change detection method, it is defined as follows.

&Quot; (2) &quot;

added (A, B) = inf (B) - inf (A)

removed (A, B) = inf (A) - inf (B)

In this case, inf (A) indicates another data set obtained by applying an inference rule to the version A (Version A).

For example, Inf (A) is a dataset with the following triple added to existing version A:

a rdfs: type d

b rdfs: type d

This is based on the transition rule "a -> c and c -> d if a -> d".

Thus, in a semantic diff-based scheme, the two triples are not in the added data set. Because it does not consider it to be added because it can be obtained through such reasoning.

Such an aggregation-based change detection method can be implemented using a memory, a database, and the like. For example, when there are two database tables such as a source and a target as shown in FIG. 5, the added triplet and deleted triplet using the SQL minus operation query, Can be obtained.

However, memory-based implementations can not handle large RDF triple data. This is because memory capacity is limited. Database-based implementations can handle large RDF triple data, but they are time consuming.

[1] Jeffrey Dean, Sanjay Ghemawat: MapReduce: Simplified Data Processing on Large Clusters. OSDI 2004: 137-150 [2] Apache Hadoop, http://hadoop.apache.org/ [3] SHA, http://en.wikipedia.org/wiki/SHA

It is an object of the present invention to solve the above problems and to provide a method and apparatus for parallel processing of RDF triple data comparison using a distributed environment composed of a plurality of computers in order to reduce change detection time for a large capacity RDF triple data, And to provide a change detection method for triple data based on parallel processing.

In particular, it is an object of the present invention to provide a method for detecting a change in triple data based on parallel processing that can reduce a comparison time by not comparing a RDF triple data in a specific case.

According to an aspect of the present invention, there is provided a method of detecting change in triple data based on source triple data and target triple data based on parallel processing based on triple and deleted triples added in a target triple based on source triple data and target triple data, (a) generating a triple group by subject from the triple data; (b) aligning the triples in the triple group; (c) comparing the number of source triples and target triples in the triple group; (d) comparing the source triple and the target triple if the numbers are the same; (e) compares all of the source triples and the target triples in the group, if they are different, to compute the added triple list and the deleted triple list; (f) determining whether the number of source triples or the number of target triples is 0 if the numbers are different, and if not, performing the step (e); And (g) including the source triple or target triple in the added triple list or the deleted triple list if the number of source triples or the number of target triples is zero, And the deleted triple list is included in the source triple list and is a list of triples included in the target triple list.

The method may further include: (e1) sequentially comparing a source triple and a target triple in the triple group; (e2) if the source triple is greater than the target triple, including a target triple in the added triple list and selecting a next target triple; (e3) if the target triple is larger than the source triple, including the source triple in the deleted triple list and selecting the next source triple; (e4) if the source triple and the target triple are the same, selecting the next source triple and the target triple; And (e5) repeating steps (e1) to (e4) until there are remaining source triples and target triples (S55).

The method may further include the step of converting all the source triples in the triple group into a string (hereinafter referred to as a source string), converting all the target triples into a string (target Character strings), and hash values of the source character string and the target character string are compared with the hash value.

Further, the present invention is a method for detecting change in triple data based on parallel processing, wherein the triple is composed of subject, predicate, and object.

According to another aspect of the present invention, there is provided a change detection method for triple data based on parallel processing, wherein the triples are arranged in order of subject, predicate, and object, and each subject, .

As described above, according to the change detection method for the triple data based on the parallel processing according to the present invention, the RDF triple data comparison can be performed in parallel, thereby enabling the RDF triple change detection even in a distributed environment composed of a plurality of computers Thus, the effect of processing large-scale RDF triple data is obtained.

In addition, according to the method for detecting a change in triple data based on parallel processing according to the present invention, compared to the RDF triple data, the comparison process is not performed in a specific case, and the comparison time can be significantly reduced.

Figure 1 is an example of an RDF triple according to the prior art.
2 is an example of a part of the Korean DBpedia infobox according to the prior art written in the N3 format.
3 is a diagram illustrating a change detection problem for an RDF triple according to the prior art.
FIG. 4 shows an example of change detection by structural difference and semantic difference according to the prior art.
FIG. 5 illustrates a conventional method of detecting change in RDF triple data using a database.
6 is a diagram showing a configuration of an overall system for carrying out the present invention.
7 is a flowchart illustrating a change detection method for triple data based on parallel processing according to an embodiment of the present invention.
FIG. 8 is a flowchart illustrating a method of comparing triple data to find added / deleted triples according to an embodiment of the present invention.
Figure 9 shows an example of an algorithm for the method of Figure 8;
FIG. 10 shows an example of finding a triple added / deleted by the method of FIG.
11 illustrates an example of finding a triple added / deleted by parallel processing of triple data according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In the description of the present invention, the same parts are denoted by the same reference numerals, and repetitive description thereof will be omitted.

First, examples of the configuration of the entire system for carrying out the present invention will be described with reference to Figs. 6A and 6B.

6A, the change detection method for triple data based on parallel processing according to the present invention receives source triple data 11 and target triple data 11 and generates target triple data 11 with respect to source triple data 11, To the computer terminal on the server 20 or to the program system on the server 20 to find the newly added triple and the deleted triple in the server 11. That is, the change detection method can be implemented by a program and installed in a computer terminal or server 20 and executed. A program installed in the computer terminal or the server 20 can operate as a single program system 30. [

Meanwhile, as another embodiment, the change detection method may be implemented by a single electronic circuit such as an ASIC (on-demand semiconductor) in addition to being operated by a general-purpose computer. Or dedicated computer terminal 20 dedicated to only detecting change detection of triple data. This is called a parallel processing based triple data change detection device. Other possible forms may also be practiced.

Next, a method for parallel processing triple change detection by a plurality of servers according to an embodiment of the present invention will be described with reference to FIG. 6B.

As shown in FIG. 6B, a plurality of servers S ₀ , S ₁ ,..., S _N are connected to each other through a network (not shown). A server S ₀ of one of the plurality of servers operates as a master computer, and the remaining servers S ₁ , ..., S _N serve as slave computers.

That is, the server S ₀ is a master computer storing an input file (source and target triple data), and the remaining servers are slave computers performing a comparison operation.

The master computer collects only the triples of the same subject in the triple list in the source and target and sends it to the slave computer. The triples that were in the source are marked with an S at the end and the triples that were in the target are marked with a T at the end.

On the other hand, when a comparison operation is performed by a slave computer, a del list and an add list can be obtained (the comparison operation will be described in detail below). At the end of the comparison, the slave computer sends the created del and list to the master computer.

The master computer combines each deleted (del) list and each added list into one. You can finally get the final deleted (del) list and the add list for the original source and target triple lists.

Next, a method for detecting a change in triple data based on parallel processing according to an embodiment of the present invention will be described with reference to FIG.

As shown in FIG. 7, the parallel processing-based triple data change detection method according to an embodiment of the present invention includes the steps of: (a) generating a triple group per subject (S10); (b) triple aligning step S20; (c) comparing the number of triples of the source / target (S30); (d) triplet comparison step (S40) of source / target if the numbers are the same; (e) comparing each triple if all are different (S50); (f) determining whether the number is 0 if the numbers are different (S60); And (g) if the number is 0, constructing a triple of adding / deleting triples of the source / target (S70).

First, the RDF triple data given as an input is divided by a subject (S10). It is the task of collecting triples having the same subject and creating a triple group by Subject. The triple group distinguishes whether the triples in the source and target belong to the source and whether they are from the target.

Next, the triples in each triple group are sorted (S20). Sorting sorts by the characters in the triple. When sorting, it ignores whether it is a source or a target and sorts only the subject, the predicate, and the object.

Next, the number of source and target triples in each triple group is compared (S30). Depending on the number, the next task is different.

First, if the number of source and target triples is equal, it is checked whether the entire contents of the source triple and the target triple are identical (S40).

In this case, the triple group having the same number of triples from the source to the target is divided into two cases. The first is when two triples are composed of exactly the same triple, and the second is when they are of the same number but of different triples.

First, the source triple list and the target triple list are converted into a string, respectively. Here is an example where there is a triple group (S means a triple from a source and T means a triple from a target).

<a p c S>

<b w t T>

<c p c S>

<d p q T>

At this time, it is converted into a string as follows.

Source: "apccpc"

Target: "bwtdpq"

Apply a hash function to each of these two strings.

The hash function converts an input string to another string. The output string of the hash function is called a hash value and consists of a number of characters smaller than the number of characters in the input string.

In general, if two input strings are equal, the hash values are the same, and if they are different, the hash values are also different. Therefore, when judging whether two large strings are equal, it is efficient to compare only the hash value without comparing each input string.

Even in the case of the present invention, if the hash value (or the hash value of the string) of the source triple group is the same as the hash value (or the hash value of the string) of the target triple group, (Target) Triple group means that it consists of the same triple. Therefore, this triple group is not identified as an added triple or a deleted triple.

If the two hash values are different, it means that there are other triples in the Source triple list and the Target triple list. In this case, the flow advances to step S50 in which the triple list is compared one by one.

Next, the number of triples from the source to the target and the number of triples from the target are described.

If the number of source and target triples is different, it is checked whether the number of source triples or target triples is 0 (S60).

If the number of source triples or target triples is zero, both the source triples or the target triples are included in the added triple list or the deleted triple list (S70).

That is, if the number of source triples is 0, the triples of the triple group are included in the added triple list. If the target triple count is zero, the triple group triples are included in the deleted triple list.

If both the number of source triples and the number of target triples are not 0, the process proceeds to step S50 in which the triple lists are compared one by one.

Finally, comparing the given two triple lists to find another triple (S50). The description of step (S50) will be described in more detail below.

Next, the step of comparing two triple lists according to an embodiment of the present invention to find other triples will be described in more detail with reference to FIGS. 8 to 10. FIG.

As shown in FIG. 8, the step S50 of comparing the source / target triple groups according to the present invention to the changed triplets S50 includes the steps of sequentially comparing the source triple and the target triple in the sorted list S51, ; (e2) if the source triple is larger than the target triple, including the target triple in the added triple list and selecting the next target triple (S52); (e3) if the target triple is larger than the source triple, including the source triple in the deleted triple list and selecting the next source triple (S53); (e4) selecting the next source / target triple if the source / target triple is the same (S54); And (e5) repeating (e1) to (e4) until there is a remaining source / target triple (S55).

Figure 9 is an algorithm implementation of the steps of Figure 8, and Figure 10 is an example of the steps of Figure 8.

Because it is a triple group, the subject parts are all the same. Predicate parts may be different, but assume that the predicate parts are all the same for the sake of explanation.

Since they are aligned in step (b) and step (S20), they are arranged in the alphabetical order of the object part. Also, since it is a triple group, the source triple and the target triple are arranged in the same order, but are separated.

As shown in FIGS. 9 and 10, the first triple (<SPE>) in the source is compared with the first triple (<SPE>) in the target (S51). Since E> A, <S P A> is included in the added triple list by the first if statement in the algorithm (S52). The only increment of t index is 1, and the next comparison is <S P E> and the second triple <S P D> of the target. Similarly, since E> D, <S P D> is included in the added triple list (S53). Then, comparing <S P E> and <S P J>, E <J, so the <S P E> by the second if statement is included in the deleted triple list.

Proceeding this way leads to the last part of Figure 8. Since all the triples of the source have been searched, the loop is terminated (S55). I quit the loop, but there are two triples in the target that have not been searched yet. These triples are all included in the added triple list.

The algorithm of FIG. 9 is available only when two lists are sorted, and is fast because it searches the list only once.

Next, a method of distributing and parallelizing RDF triple data according to an embodiment of the present invention will be described with reference to FIG.

The parallel processing method can use the MapReduce framework [Non-Patent Document 1]. The MapReduce framework is a type of parallel processing that is one of the ways that Google search engines process large amounts of web data. In the Map phase, data is segmented by specific rules, and in the Reduce phase, operations are performed on the segmented data.

Apache Hadoop [Non-Patent Document 2] is an open source library that implements the MapReduce framework in the Java language. In a distributed computing environment, an unexpected error may occur in an individual computer, and a method of monitoring and processing it is necessary. Apache Hadoop also provides distributed computing management features such as restarting Maple steps on specific individual computers or restarting them if they fail during the Reduce phase.

11 is a diagram showing the algorithm of the present invention by the MapReduce framework.

In the Map phase, all RDF triples in the Source and Target are divided by Subject. Task # 1 creates a task in the form of a triple with Subject A, and a triple with Subject D in Task # 2. If the number of machines is less than the number of jobs, it waits and sends the job that was waiting when the job is completed to the machine. In task # 1, the triple (<A p B> <A k D>) is a triple from the source, so it displays S at the end. Likewise, in task # 1, the triple (<A k B> <A p B>) is a triple from the target, and thus a T is displayed at the end.

In the Reduce step, the two triple lists (Source and Target) are compared. If the two triple lists are the same size, the hash function (SHA) is used to compare the hash values. If the hash values are the same, it means that the two triple lists are the same and are ignored. If it is not the same, compare the two triple lists to find the added and deleted triples.

A system is being actively developed to make data accessible to anyone by making the data public on the web in a standardized form. One of the Web data standards is LOD. LOD is an abbreviation of Linked Open Data, which indicates that data from public institutions are described in a standardized form (RDF triple).

LODs are often added or deleted over time. In order to utilize this LOD, it is possible to design an efficient LOD utilization system if the common triple between the old version LOD and the current version LOD can be distinguished from the common triple. Thus, the RDF triple change detection technique that identifies added and deleted triples in any given two triple lists is important.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

11: Source triple data 12: Target triple data
20: computer terminal 30: program system

Claims (5)

1. A change detection method for triple data based on parallel processing for finding triples and deleted triples added in a target triple based on source triple data with respect to source triple data and target triple data,
(a) generating a triple group by subject from the triple data;
(b) aligning the triples in the triple group;
(c) comparing the number of source triples and target triples in the triple group;
(d) comparing the triple data of the source triples of the corresponding number and the triple data of the target triples of the same number, respectively, if they are the same;
(e) if the triple data is different, comparing the source triple and the target triple in the group, respectively, to create an added triple list and a deleted triple list;
(f) determining whether the number of source triples or the number of target triples is 0 if the numbers are different, and if not, performing the step (e); And
(g) if the number of source triples or the number of target triples is zero, including a source triple or a target triple in the added triple list or the deleted triple list,
Wherein the added triple list is not included in the source triple list but is a list of triples included in the target triple list,
Wherein the deleted triple list is a list of triples included in the source triple list and included in the target triple list.
2. The method of claim 1, wherein step (e)
(e1) sequentially comparing the source triple and the target triple in the triple group by a string of the corresponding triple data;
(e2) if the source triple is greater than the target triple, including a target triple in the added triple list and selecting a next target triple;
(e3) if the target triple is larger than the source triple, including the source triple in the deleted triple list and selecting the next source triple;
(e4) if the source triple and the target triple are the same, selecting the next source triple and the target triple; And
(e5) repeating steps (e1) through (e5) until there are remaining source triples and target triples (step S55).
3. The method of claim 2,
In step (d), all the source triples in the triple group are converted into a string (hereinafter, referred to as a source string), all the target triples are converted into a string (target string), the source string and the target string are hashed, Value of the triple data based on the parallel processing.
The method according to claim 1,
Wherein the triple comprises a subject, a predicate, and an object in order.
5. The method of claim 4,
Wherein the arrangement of the triples is arranged by a string of a subject, a descriptor, and an object, and is arranged in order of a subject, a descriptor, and an object.

Images