KR102279109B1 - Distributed system and method for integrating knowledge - Google Patents

Abstract

A data integration system for accessing a plurality of remote data transformation systems for a distributed system for knowledge integration includes, according to an exemplary embodiment of the present invention, a job selector configured to receive a request from an administrator and decode the request, the administrator In response to the conversion request, the conversion management unit instructs at least one data conversion system to convert the source data, and receives the converted source data from the at least one data conversion system, and metadata used for knowledge integration. ), and in response to an administrator's request for integration, a data integration unit for integrating the transformed source data into the knowledge base based on a mapping table between knowledge domains stored in the metadata repository. have.

Description

DISTRIBUTED SYSTEM AND METHOD FOR INTEGRATING KNOWLEDGE

The technical idea of the present invention relates to knowledge data, and more particularly, to a distributed system and method for integrating scattered knowledge data.

The present invention is derived from research conducted and conducted by Saltlux Co., Ltd. as part of the SW computing source technology development project (SW) of the Ministry of Science, ICT and Future Planning. [Research period: 2019.01.01~2019.12.31, Research management institution: Information and Communication Technology Promotion Center, Research project name: WiseKB: Development of self-learning knowledge base and reasoning technology based on big data understanding, project identification number: 2013-0 -00109]

The knowledge data may be interspersed in various knowledge domains. For example, knowledge data may be respectively included in different knowledge domains according to fields, or may be included in different knowledge domains, respectively, according to a service method of the knowledge data. Also, knowledge domains may store and manage knowledge data in different ways. Such scattered knowledge data may make it difficult for a user to access the knowledge data, and duplicate knowledge data may be included in a plurality of knowledge domains. Accordingly, in order to improve the utilization and efficiency of knowledge data, a method for integrating scattered knowledge data, ie, knowledge integration, may be required.

The technical idea of the present invention provides a distributed system and method for knowledge integration using a plurality of data transformation systems.

In order to achieve the above object, according to an aspect of the technical idea of the present invention, a data integration system for accessing a plurality of remote data transformation systems for a distributed system for knowledge integration receives a request from an administrator and , a job selection unit for decoding the request, in response to the conversion request of the manager, instructing the conversion operation of the source data to at least one data conversion system, the conversion management unit for receiving the converted source data from the at least one data conversion system, and accessing a metadata repository that stores metadata used for knowledge integration, and in response to an integration request of an administrator, converts source data transformed based on a mapping table between knowledge domains stored in the metadata repository into knowledge. It may include a data integration unit for integrating into the base.

According to an exemplary embodiment of the present invention, the conversion request may include identification information of the knowledge domain including the source data and at least one identifier of the source data included in the knowledge domain.

According to an exemplary embodiment of the present invention, the data integrator includes a knowledge integrator that generates a source instance based on at least one first identifier of at least one knowledge entity included in the mapping table and the transformed source data, and the source It may include a knowledge application unit that refines the instance and integrates the refined source instance into the knowledge base.

According to an exemplary embodiment of the present invention, the knowledge integration unit searches for at least one first identifier in a mapping table, and a knowledge entity selector that extracts at least one knowledge entity from the knowledge base based on the search result, and at least and a source instance generator configured to generate a source instance from the converted source data by changing one first identifier to at least one second identifier of the extracted at least one knowledge entity.

According to an exemplary embodiment of the present invention, when the at least one first identifier is not found in the mapping table, the knowledge entity selector may extract at least one candidate knowledge entity from the knowledge base based on the converted source data. have.

According to an exemplary embodiment of the present invention, the knowledge application unit includes a source instance post-processing unit that post-processes a source instance based on the types of knowledge instances included in the knowledge base, and a post-processed source instance to the knowledge included in the knowledge base. It may include an instance comparison unit that selectively integrates the post-processed source instance into the knowledge base by comparing with the instances.

According to an exemplary embodiment of the present invention, the data integration system provides the data generated by the conversion management unit and the data integration unit to the manager in response to the manager's inquiry request, and in response to the manager's update request, metadata It may further include a data management unit for updating the.

A distributed system for knowledge integration according to an aspect of the inventive concept includes a plurality of data transformation systems, and transformed source data provided from a plurality of remote data transformation systems based on a mapping table between knowledge domains and a plurality of data transformation systems. may include a data integration system for integrating into the knowledge base, wherein each of the plurality of data transformation systems is included in the knowledge domain based on the structure of the knowledge instances included in the knowledge base in response to an instruction provided from the data integration system. It may include a plurality of transformation engines that transform the source data.

According to an exemplary embodiment of the present invention, each of the plurality of data transformation systems selects one transformation module of the plurality of transformation modules based on the instruction and provides the selected transformation module to at least one of the plurality of transformation engines It may include a conversion engine management unit that

According to an exemplary embodiment of the present invention, the indication may include at least one of identification information of a knowledge domain, at least one identifier of source data, and an index of a transformation module.

According to an exemplary embodiment of the present invention, each of the plurality of data transformation systems may include a transformation data store that stores transformed source data, and each of the plurality of transformation engines collects source data from the knowledge domain. and store the converted source data in the conversion data store.

According to the distributed system and method according to the technical idea of the present invention, data transformation and data integration required for knowledge integration are separated, and the data transformation is processed in parallel by a plurality of data transformation systems, so that vast knowledge Data can be distributed and processed, so that knowledge integration can be efficiently and easily achieved.

In addition, according to the distributed system and method according to the technical idea of the present invention, a knowledge base including integrated knowledge data can be easily implemented, and thus the usefulness of various services based on the knowledge base can be improved.

Effects that can be obtained in the embodiments of the present invention are not limited to the effects mentioned above, and other effects not mentioned are common in the art to which the present invention belongs from the description of the embodiments of the present invention below. It can be clearly derived and understood by those with knowledge. That is, unintended effects of carrying out the present invention may also be derived by those of ordinary skill in the art from the embodiments of the present invention.

1 is a block diagram illustrating a distributed system for knowledge integration according to an exemplary embodiment of the present invention.
Fig. 2 is a block diagram showing an example of a data integrator according to an exemplary embodiment of the present invention.
3 is a block diagram illustrating an example of a knowledge integration unit according to an exemplary embodiment of the present invention.
4A and 4B are diagrams illustrating examples in which source instances are created according to exemplary embodiments of the present invention.
5 is a flowchart illustrating an example of an operation of a knowledge object selection unit according to an exemplary embodiment of the present invention.
6 is a block diagram illustrating an example of a knowledge application unit according to an exemplary embodiment of the present invention.
Fig. 7 is a block diagram showing an example of a data conversion system according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art. Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals are used for like elements. In the accompanying drawings, the dimensions of the structures are enlarged or reduced than the actual size for clarity of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings in the context of the related art, and unless explicitly defined in the present application, they are not interpreted in an ideal or excessively formal meaning. .

In the drawings and description below, a component indicated or described as one block may be a hardware block or a software block. For example, each of the components may be an independent hardware block that sends and receives signals with each other, or may be a software block executed by one processor.

1 is a block diagram illustrating a distributed system for knowledge integration according to an exemplary embodiment of the present invention. The distributed system may include a data integration system 110 and a plurality of data transformation systems 120 , may communicate with an administrator 10 , and may have access to a metadata repository 130 and knowledge base 140 . can In some embodiments, the distributed system may include a metadata repository 130 and/or a knowledge base 140 . In this specification, the metadata repository 130 and the knowledge base 140 will be described as being external to the distributed system, but it is noted that the exemplary embodiments of the present invention are not limited thereto.

The distributed system may perform the operation of integrating the scattered knowledge data (which may be referred to herein as source data) into the knowledge base 140 . The manager 10 may be any entity that manages the integration of knowledge data by a distributed system (which may be referred to as knowledge integration herein). For example, the manager 10 may instruct the distributed system an operation for knowledge integration (eg, a transformation operation, an integration operation, etc. to be described later), or inquire about the knowledge data integration process performed by the distributed system. Also, it is possible to control the integration of the knowledge data by updating the data used for the integration of the knowledge data, for example, the metadata stored in the metadata storage 130 . The manager 10 may communicate with the data integration system 110 in a variety of ways. For example, the manager 10 may refer to any terminal that communicates with the data aggregation system 110 through a communication channel, and the terminal is the data aggregation system through the communication channel according to an input received from the manager 10 . The request may be provided to the 110 , and a response according to the request may be received from the data integration system 110 and provided to the manager 10 . In some embodiments, the communication channel between the manager 10 and the data aggregation system 110 may be established via a network, such as the Internet, or may be one-to-one direct communication.

The metadata storage 130 may store metadata used for integration of knowledge data. For example, the metadata repository 130 may store intermediate data (eg, 132, 136, etc. in FIG. 2 ) generated in the process of integrating knowledge data, and data referenced in the process of integrating knowledge data (eg, in FIG. 2 ). 134 of 2) can also be stored. In some embodiments, the administrator 10 may update a portion of the metadata stored in the metadata repository 130 via the data integration system 110 . The metadata repository 130 may communicate with the data aggregation system 110 through a network as an independent system (eg, a server), or may be a local storage connected to the data aggregation system 110 through a one-to-one communication channel. An example of the metadata repository 130 will be described below with reference to FIG. 2 and the like.

The knowledge base 140 may refer to a system including integrated knowledge data. The knowledge base 140 may store knowledge data according to a predefined structure, and may provide the knowledge data to the outside according to a request (eg, a query). For example, the knowledge base 140 may include knowledge instances expressed in a Resource Description Framework (RDF) structure, and a set of knowledge instances may be referred to as a knowledge graph. In the present specification, a knowledge instance may refer to a unit representing knowledge, for example, "Sun-sin Yi's birthday is April 28, 1545". In the RDF structure, a knowledge instance may be expressed as a triple. A triple may be expressed as a subject, a predicate, and an object, for example, "Sun-Shin Yi / Birthday / 1545-04-28". "Sun-Shin Yi" may be referred to as a knowledge entity or simply an entity, and may have a unique identifier in the knowledge base 140 (which may be referred to as a second identifier herein), such as a Uniform Resource Identifier (URI). . Also, “birthday” may be referred to as an attribute indicating a relationship between a subject and an object. In the triple, the object may correspond to a value such as "April 28, 1545" above, or may correspond to an entity different from the subject when a predicate indicates a relationship between entities, such as "Sun-Shin Yi / Job / Soldier". A knowledge graph may be implemented in a knowledge domain different from the knowledge base 140 , and thus, knowledge integration may be referred to as knowledge graph integration.

The knowledge data scattered in the knowledge domain may be integrated into the knowledge base 140 by a distributed system. The integration of knowledge data includes a data transformation operation for transforming distributed knowledge data according to the structure of the knowledge base 140 and data integration for integrating the transformed knowledge data with knowledge data included in the knowledge base 140 , that is, knowledge instances. It can include actions. In a distributed system, a data transformation operation may be performed in parallel by a plurality of data transformation systems 120 , while a data integration operation may be performed by the data integration system 110 . In this way, the distributed system separates the integration of knowledge data into a data transformation operation and a data integration operation, and by parallel processing the data transformation operation, a large amount of knowledge data can be distributed and processed, and the integration of knowledge data is efficiently and easily achieved. can be In addition, since the knowledge base 140 including the integrated knowledge data is easily implemented by a distributed system, various services based on the knowledge base 140 may be available, and the usefulness of such services may be remarkably improved.

The data integration system 110 may communicate with the manager 10 and may communicate with a plurality of data conversion systems 120 , as shown in FIG. 1 . In some embodiments, the data aggregation system 110 may communicate with a plurality of data conversion systems 120 via a network, such as the Internet. For example, the plurality of data conversion systems 120 may be located at remote locations from the data integration system 110 , and may also be remote between the plurality of data conversion systems 120 . Additionally, the data integration system 110 may communicate with the metadata repository 130 and the knowledge base 140 . As shown in FIG. 1 , the data integration system 110 may include a job selection unit 112 , a conversion management unit 114 , a data integration unit 116 , and a data management unit 118 .

The task selection unit 112 may receive a request from the manager 10 and may decode the received request. For example, the job selection unit 112 may receive a conversion request from the manager 10, and by decoding the conversion request, an indication of the conversion operation and information for the conversion operation may be provided to the conversion management unit 114. . In addition, the task selection unit 112 may receive the integration request from the manager 10 , and provide the data integration unit 116 with information for an indication of the integration request and an integration operation by decoding the integration request.

The conversion manager 114 may communicate with the plurality of data conversion systems 120 . The conversion management unit 114 may instruct a conversion operation of the source data to at least one data conversion system among the plurality of data conversion systems 120 in response to the conversion request of the manager 10 , and at least one data conversion system It is possible to receive the converted source data from the system. For example, the conversion management unit 114 may instruct the conversion operation to the first data conversion system 120_1 and/or the k-th data conversion system 120_k in response to the conversion request, and the first data conversion system ( 120_1) and/or may receive the converted source data from the k-th conversion system 120_k. In some embodiments, the conversion request provided by the manager 10 includes at least identification information of a knowledge domain including source data to be converted and source data to be converted among source data included in the knowledge domain. It may include one identifier (which may be referred to as a first identifier herein). In some embodiments, the conversion management unit 114 may select at least one data conversion system in consideration of the loads of each of the plurality of data conversion systems 120 , and perform a conversion operation on the selected at least one data conversion system You can provide the information included in the instructions and conversion requests. The conversion management unit 114 may provide the converted source data received from at least one data conversion system to the data integration unit 116 .

The data integration unit 116 may receive the converted source data from the conversion management unit 114 , and may access the metadata repository 130 and the knowledge base 140 . In response to the integration request of the manager 10 , the data integration unit 116 integrates the converted source data provided from the conversion management unit 114 into the knowledge base 140 by referring to data stored in the metadata storage 130 . can The data integration unit 116 may process data stored in the metadata storage 130 during the integration process, and may store the processed data in the metadata storage 130 . An example of the data integrator 116 will be described later with reference to FIG. 2 and the like.

The data management unit 118 may include at least a portion of the metadata stored in the metadata storage 130 in response to the inquiry request of the manager 10 , for example, data generated by the conversion management unit 114 and the data integration unit 116 ( For example, 132 and 136 of FIG. 2 may be provided to the manager 10 through the task selection unit 112 , and at least the metadata stored in the metadata storage 130 in response to the update request of the manager 10 . Some (eg, 134 in FIG. 2 ) may be updated. That is, the data management unit 118 may support the curation of the manager 10 for the distributed system.

Each of the plurality of data conversion systems 120 may convert source data, ie, knowledge data included in the knowledge domain, based on an instruction of the conversion management unit 114 of the data integration system 110 . For example, a knowledge domain may store knowledge data, ie, source data, in a unique way. For example, the knowledge domain may store knowledge data according to an RDF structure, similar to the knowledge base 140 , or may store knowledge data according to another arbitrary structure. In addition, the knowledge domain may store knowledge data according to a unique format such as CSV, TXT, Linked Open Data (LOD), and the like. Accordingly, in order for the data integration system 110 to process the knowledge data included in the knowledge domain, the plurality of data conversion systems 120 may convert the knowledge data of the knowledge domain into a common structure and format. In this specification, data in which the source data of the knowledge domain is converted by the plurality of data conversion systems 120 may be referred to as converted source data. As shown in FIG. 1 , the first data transformation system 120_1 may include a transformation data store 122_1 and a plurality of transformation engines 124_1 , and the k-th data transformation system 120_k is a transformation data store 122_k and a plurality of transformation engines 124_k.

Each of the plurality of data transformation systems 120 may include an independent transformation data store, and thus may support large-capacity processing of source data. Unlike shown in FIG. 1 , when data conversion and data integration are performed in a single system (or a single server), processing of massive source data may be inefficient. Each of the plurality of data conversion systems 120 may include a plurality of conversion engines capable of independently performing a conversion operation as well as a conversion data store, and thus the conversion operation is performed in parallel even within the data conversion system can be In some embodiments, one data transformation system may transform source data included in one knowledge domain. Also, in some embodiments, two or more data transformation systems may each transform different source data included in one knowledge domain. Such conversion operations by the plurality of data conversion systems 120 may be scheduled by the conversion management unit 114 as described above.

Fig. 2 is a block diagram showing an example of a data integrator according to an exemplary embodiment of the present invention. Specifically, the block diagram of FIG. 2 shows the data integrator 200 and the metadata repository 130 and the knowledge base 140 accessed by the data integrator 200 . As described above with reference to FIG. 1 , the data integration unit 200 may be included in the data integration system 110 of FIG. 1 , and in response to the integration request of the manager 10 , the conversion provided from the conversion management unit 114 . source data may be integrated into the knowledge base 140 . As shown in FIG. 2 , the data integrator 200 may include a knowledge integrator 220 and a knowledge application unit 240 , and the metadata repository 130 includes source instances 132 and a mapping table. 134 and candidate knowledge entities 136 , and the knowledge base 140 may include knowledge instances 142 . Hereinafter, FIG. 2 will be described with reference to FIG. 1 .

The knowledge integrator 220 may generate a source instance based on at least one first identifier of at least one knowledge entity included in the transformed source data. The transformed source data may include at least one entity having a unique identifier in a knowledge domain including the source data. For example, in the above-described example “Sun-Sin Yi / Profession / Soldier”, the entities “Sun-Sin Yi”, “Occupation” and “Soldier” may each have a unique identifier in the knowledge domain. The mapping table 134 may define a mapping relationship between the identifier of the knowledge entity included in the source data and the identifier of the knowledge entity included in the knowledge instances 142 of the knowledge base 140 . Accordingly, the knowledge integrator 220 may generate the source instances 132 by referring to the mapping table 134 from the converted source data, and may store the source instances 132 in the metadata storage 130 . Also, when the identifier of the entity included in the source data does not exist in the mapping table 134 , the knowledge integrator 220 may search for the candidate knowledge entities 136 from the knowledge base 140 , It may be stored in the data storage 130 . An example of the knowledge integrator 220 will be described later with reference to FIG. 3 .

The knowledge application unit 240 may refine the source instances 132 generated by the knowledge integration unit 220 , and may integrate the refined source instances into the knowledge base 140 . For example, the knowledge applicator 240 may verify the source instances 132 , and compare the source instances 132 with at least one knowledge instance of the knowledge base 140 . It may be determined whether the source instances 132 are added. An example of the knowledge application unit 240 will be described later with reference to FIG. 6 .

3 is a block diagram illustrating an example of a knowledge integration unit according to an exemplary embodiment of the present invention, and FIGS. 4A and 4B are diagrams illustrating examples of generating source instances according to exemplary embodiments of the present invention. As described above with reference to FIG. 2 , the knowledge integrator 300 may be included in the data integrator 200 of FIG. 2 (or 116 of FIG. 1 ), and the converted source is referred to by referring to the mapping table 134 . You can create source instances from data. Hereinafter, content overlapping with the description of FIG. 2 among the description of FIG. 2 in FIGS. 3, 4A and 4B will be omitted.

Referring to FIG. 3 , the knowledge integrator 300 may include a knowledge object selector 320 and a source instance generator 340 . The knowledge object selector 320 may search the mapping table 134 for an identifier of a knowledge object included in the converted source data, and extract at least one knowledge object from the knowledge base 140 based on the search result. can For example, as shown on the right side of FIG. 4A , the knowledge object selection unit 320 identifies the knowledge objects "Sun-Shin Yi" and "Soldier" in the example 42 "Sun-Shin Yi / Occupation / Soldier" of the converted source data. As , "wdata: 43" and "wdata: 30" can be obtained, respectively. The knowledge object selection unit 320 may search for identifiers “wdata: 43” and “wdata: 30” in the mapping table 41 shown on the left side of FIG. 4A , and as shown in FIG. 4A , identifiers “wdata” : 43" and identifiers "addr: 0102" and "addr; 0001" corresponding to "wdata: 30" can be obtained. Although not shown in FIG. 4A , the predicate P may also have an identifier as one knowledge entity in the knowledge domain, and the mapping table 41 may also include a mapping relationship between identifiers of the predicate P. Similarly, as shown on the right side of FIG. 4B , the knowledge entity selection unit 320 as an identifier of the knowledge entity “Sun-Shin Yi” in the example 44 “Sun-Shin Yi / Birthday / 1545-04-28” of the converted source data. “wdata: 43” may be obtained, and an identifier “addr: 0102” corresponding to “wdata: 43” may be obtained as an identifier with reference to the mapping table 41 .

Unlike the examples shown in FIGS. 4A and 4B , when the identifier is not searched for in the mapping table 134 of FIG. 3 , the knowledge entity selector 320 may access the knowledge base 140 , and Candidate knowledge entities 136 may be generated by extracting knowledge entities included in the knowledge instances of the base 140 . When the identifier is not found in the mapping table 134 of FIG. 3 , an example of the operation of the knowledge entity selector 320 will be described with reference to FIG. 5 .

The source instance generator 340 may generate the source instances 132 by changing the identifier of the knowledge entity included in the converted source data to the identifier provided by the knowledge entity selector 320 . For example, as shown on the right side of FIG. 4A , the source instance generator 340 is an example 42 of converted source data including identifiers “wdata: 43” and “wdata: 30” of the knowledge domain. A source instance 43 including identifiers “addr: 0102” and “addr: 0001” of a knowledge entity included in the knowledge base 140 may be created from the . Similarly, as shown on the right side of FIG. 4B , the source instance creation unit 340 stores the knowledge base 140 from the example 44 of the converted source data including the identifiers “wdata: 43” of the knowledge domain. It is possible to create a source instance 45 containing the identifier "addr: 0102" of the included knowledge entity. In some embodiments, as shown in FIG. 4B , the source instance generator 340 may generate the source instance 45 including the value “1545-04-28” of the example 44 of the source data as it is. can

5 is a flowchart illustrating an example of an operation of a knowledge object selection unit according to an exemplary embodiment of the present invention. As described above with reference to FIG. 3 , the knowledge object selector 320 of FIG. 3 may search the mapping table 134 for the identifier of the knowledge object included in the converted source data, and based on the search result, the knowledge object selector 320 may search for the knowledge object identifier included in the converted source data. At least one knowledge entity may be extracted from the base 140 . As shown in FIG. 5 , the operation of the knowledge object selection unit 320 may include a plurality of steps S51 to S59 . Hereinafter, FIG. 5 will be described with reference to FIGS. 3, 4A and 4B.

In operation S51 , an operation of searching for the first identifier of the source data in the mapping table 134 may be performed. For example, the knowledge object selector 320 may map the first identifier (eg, "wdata: 43", "wdata: 30" in FIGS. 4A and 4B) of the knowledge object included in the converted source data to the mapping table ( 134) can be found.

In step S52, it may be determined whether the search for the first identifier is successful. For example, when the first identifier of step S51 exists in the mapping table 134 , the knowledge object selector 320 may determine the search success, otherwise, it may determine the search failure. As shown in Fig. 5 , when it is determined that the search succeeds, step S59 may be performed subsequently, while when it is determined that the search fails, step S53 may be performed subsequently.

When it is determined that the search for the first identifier fails, an operation for searching for a candidate knowledge entity in the knowledge base 140 may be performed in step S53 . For example, the knowledge object selection unit 320 selects at least one candidate knowledge object from the knowledge base 140 to determine the knowledge object included in the knowledge base 140 that corresponds to the knowledge object having the first identifier. You can search. In some embodiments, such as the "similarity calculation unit" described in Korean Patent Application No. 10-2018-0151222 filed by the same applicant as the present application and incorporated herein by reference in its entirety, the knowledge object selection unit 320 ) may search the knowledge base 140 for knowledge instances similar to the source instance, and extract at least one candidate knowledge entity based on similarities of the found knowledge instances.

In step S54, it may be determined whether the candidate knowledge entity search succeeds. For example, the knowledge object selection unit 320 may determine a search failure when a knowledge object having a similarity greater than or equal to a predefined threshold value is not searched in the knowledge base 140 in step S53, while in step S53, the knowledge object selection unit 320 may determine the search failure in advance. When at least one knowledge entity having a similarity greater than or equal to a defined threshold is found in the knowledge base 140 , the search success may be determined. As shown in Fig. 5 , when it is determined that the search succeeds, step S57 may be performed subsequently, while when it is determined that the search fails, step S55 may be performed subsequently.

When it is determined that the search for the candidate knowledge entity fails, an operation for generating a new identifier may be performed in step S55. For example, when a knowledge object similar to the knowledge object corresponding to the first identifier is not found in the knowledge base 140 , the knowledge object selector 320 may determine that the knowledge object corresponding to the first identifier is the knowledge base 140 . ) can be determined as a new knowledge entity that is not in Accordingly, the knowledge entity selector 320 may generate a new identifier corresponding to the first identifier with reference to the identifier system of the knowledge base 140 .

On the other hand, if the search success of the candidate knowledge entity is determined, it may be determined in step S57 whether a single candidate knowledge entity has been searched. For example, the knowledge entity selector 320 may determine whether only one knowledge entity having a similarity greater than or equal to a predefined threshold is found. In some embodiments, when two or more candidate knowledge entities having similarity greater than or equal to a predefined threshold are searched for, the knowledge entity selector 320 may be configured to determine between the candidate knowledge entity having the highest similarity and the candidate knowledge entity having the second highest similarity. When the similarity difference is greater than or equal to a predefined reference value, it may be determined that the candidate knowledge entity having the highest similarity is independently searched. As shown in FIG. 5 , when a single candidate knowledge entity is searched, step S56 may be subsequently performed, while if two or more candidate knowledge entities are searched, step S58 may be subsequently performed.

When two or more candidate knowledge entities are found, an operation of storing the candidate knowledge entities may be performed in step S58. For example, the knowledge entity selector 320 may store candidate knowledge entities in the metadata repository 130 , and accordingly, as described above with reference to FIG. 2 , the metadata repository 130 may store candidate knowledge entities. (136). When it is difficult to automatically determine the knowledge entity of the knowledge base 140 corresponding to the knowledge entity of the first identifier through the similarity, the candidate knowledge entities 136 may be stored in the metadata storage 130 , and the metadata The candidate knowledge objects 136 stored in the storage 130 may be provided to the manager 10 through the data management unit 118 and the task selection unit 112 of FIG. 1 . The manager 10 may select a knowledge entity corresponding to the knowledge entity of the first identifier from among the candidate knowledge entities 136 through curation, and the data management unit 118 according to the selected knowledge entity generates the mapping table 134 . ) can be updated. 5 , step S51 may be performed again following step S58, and accordingly, the identifier of another knowledge entity included in the converted source data may be retrieved from the mapping table 134 as the first identifier. have.

In step S56, an operation of updating the mapping table 134 may be performed. For example, the knowledge entity selector 320 maps the new identifier generated in step S55 or the identifier of the single candidate knowledge entity searched for in step S53 with the first identifier as the second identifier, and adds it to the mapping table 134 . can do. Accordingly, the updated mapping table 134 may include a first identifier and a second identifier corresponding to the first identifier.

In step S59, an operation of obtaining a second identifier corresponding to the first identifier may be performed. For example, the knowledge object selection unit 320 may perform a successful search for the first identifier in step S52 (eg, examples of FIGS. 4A and 4B ) or after the mapping table 134 is updated in step S56, A second identifier corresponding to the first identifier may be obtained from the mapping table 134 . The knowledge entity selector 320 may provide the obtained second identifier to the source instance generator 340 .

6 is a block diagram illustrating an example of a knowledge application unit according to an exemplary embodiment of the present invention. As described above with reference to FIG. 2 , the knowledge application unit 600 may refine the source instances 132 generated by the knowledge integrator (eg, 220 of FIG. 2 ), and convert the refined source instances into knowledge. It can be integrated into the base 140 . As shown in FIG. 6 , the knowledge application unit 600 may include a source instance post-processing unit 620 and an instance comparison unit 640 , and FIG. 6 will be described below with reference to FIG. 2 .

The source instance post-processing unit 620 may post-process the source instances 132 based on the types of knowledge instances included in the knowledge base 140 . For example, in the example of FIG. 4B , in the source instance 45 corresponding to "Sun-Shin Yi / Birthday / 1545-04-28", "1545-04-28" indicating time information as the object of the triple is in the source domain. May conform to the format "YYYY-MM-DD" for representing a date. The source instance post-processing unit 620 may materialize time information by post-processing “1545-04-28”. For example, the source instance post-processing unit 620 may run from “1545-04-28” to “tm:1545-04-28”, “tm:1545-04-28 year 1545”, “tm:1545-04-28” month 04", "tm:1545-04-28 day 28". As a non-limiting example, when the knowledge base 140 has "MON DD, YYYY" as a format for indicating the date in the knowledge instance, the source instance post-processing unit 620 may display "1545" based on the specified time information. The source instance 45 can be post-processed by changing "-04-28" to "APR 28, 1545". In some embodiments, the source instance post-processing unit 620 post-processes the source instances 132 by referencing a plurality of post-processing modules including, for example, a post-processing module for converting a date format as in the example described above. can do. Herein, post-processed source instances may be referred to as refined source instances.

The instance comparison unit 640 may receive the post-processed (or refined) source instance from the source instance post-processor 620 , and compare the post-processed source instance with knowledge instances included in the knowledge base 140 . can The instance comparison unit 640 may determine whether to add the knowledge corresponding to the post-processed source instance to the knowledge base 140 according to the comparison result, and selectively integrate the post-processed source instance into the knowledge base 140 . can In some embodiments, like the "similarity calculator" described in Korean Patent Application No. 10-2018-0151222, the instance comparison unit 640 compares knowledge instances similar to the post-processed source instances in the knowledge base 140 . may be searched, and it may be determined whether to add a post-processed source instance based on similarities of the retrieved knowledge instances. For example, the instance comparison unit 640 may ignore the post-processed source instance when a knowledge instance having a similarity greater than or equal to a predefined threshold is searched for in the knowledge base 140 , while having a similarity less than the threshold. If only the knowledge instances are retrieved from the knowledge base 140 , the post-processed source instance may be added to the knowledge base 140 .

Fig. 7 is a block diagram showing an example of a data conversion system according to an exemplary embodiment of the present invention. As described above with reference to FIG. 1 , the distributed system may include a plurality of data conversion systems, and the data conversion system 700 of FIG. 7 as one of the plurality of data conversion systems is the data integration system ( 110 ′), source data, that is, knowledge data included in the knowledge domain 150 may be converted. As shown in FIG. 7 , the data conversion system 700 may include a conversion engine manager 720 , a plurality of conversion modules 740 , a conversion data store 760 , and a plurality of conversion engines 780 . have.

Each of the plurality of transformation modules 740 may define a required transformation rule according to the knowledge domain 150 or a group of knowledge data included in the knowledge domain 150 . In some embodiments, the data transformation system 700 may perform a transformation operation without being limited to a specific knowledge domain, and for this purpose, it may include a plurality of transformation modules 740 corresponding to a plurality of knowledge domains. can Further, in some embodiments, the data transformation system 700 may be limited to transforming knowledge data included in a single knowledge domain, and a plurality of transformations each corresponding to groups of knowledge data included in a single knowledge domain. Modules 740 may be included. In some embodiments, the plurality of transformation modules 740 may be provided from the data integration system 110 ′, and may be stored in the data transformation system 700 by the transformation engine manager 720 .

The transformation engine management unit 720 may receive an instruction from the data integration system 110 ′, and may control a data transformation operation based on information included in the instruction. In some embodiments, the indication from data integration system 110 ′ may include identifying information (eg, URL) for knowledge domain 150 . The transformation engine manager 720 may select at least one transformation module from among the plurality of transformation modules 740 based on the identification information of the knowledge domain 150 included in the instruction, and convert the selected transformation module to a plurality of transformation engines. It may be provided to at least one of 780 and trigger a conversion operation. Also, in some embodiments, the instruction from the data integration system 110 ′ may include identifiers of source data indicating a range of source data to be converted among source data included in the knowledge domain 150 . . The transformation engine management unit 720 may recognize a group of source data including source data based on the identifiers of the source data, and select at least one transformation module among the plurality of transformation modules 740 according to the recognized group. may be selected, and may provide the selected transformation module to at least one of the plurality of transformation engines 780 and trigger a transformation operation. The conversion engine management unit 720 may provide the converted source data stored in the conversion data storage 760 to the data integration system 110 ′. In addition, the transformation engine manager 720 may schedule tasks of the plurality of transformation engines 780 .

Each of the plurality of transformation engines 780 may access the knowledge domain 150 and may transform source data included in the knowledge domain 150 based on a transformation module provided from the transformation engine manager 720 . A plurality of transformation engines 780 may perform transformation in parallel, and for this purpose, the transformation engine manager 720 operates a plurality of transformation engines 780 such that different source data is transformed by different transformation engines, respectively. can be controlled In some embodiments, the knowledge domain 150 may be a remote system (eg, a server), and each of the plurality of transformation engines 780 is configured through a network based on an identifier list provided from the transformation engine manager 720 . By accessing the knowledge domain 150, source data can be collected and transformed. Also, in some embodiments, the knowledge domain 150 may be a local storage of the data transformation system 700 that stores source data collected from a remote system, and the transformation engine manager 720 stores areas of the local storage. Each of the plurality of conversion engines 780 may be designated. The plurality of transformation engines 780 may store the transformed source data in the transformation data storage 760 .

Exemplary embodiments have been disclosed in the drawings and specification as described above. Although the embodiments have been described using specific terms in the present specification, these are only used for the purpose of explaining the technical idea of the present invention and not used to limit the meaning or the scope of the present invention described in the claims . Therefore, it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be defined by the technical spirit of the appended claims.

Claims (11)

A data integration system configured to access a plurality of remote data transformation systems for a distributed system for knowledge integration comprising:
a job selection unit configured to receive a request from a manager and to decode the request;
a conversion management unit configured to instruct the at least one data conversion system to convert the source data in response to the conversion request of the manager, and to receive the converted source data from the at least one data conversion system; and
configured to access a metadata repository that stores metadata used for knowledge integration, and in response to an integration request of the administrator, the transformed source data based on a mapping table between knowledge domains stored in the metadata repository a data integration unit configured to integrate the
The data integrator generates a source instance based on the knowledge entity included in the mapping table and the transformed source data, extracts at least one knowledge entity from the knowledge base based on the knowledge entity, and and integrating the source instance into the knowledge base based on a similarity between the extracted knowledge entity and the at least one knowledge entity included in the source data. The method according to claim 1,
The conversion request, the data integration system, characterized in that it includes identification information of the knowledge domain including the source data and at least one identifier of the source data included in the knowledge domain. The method according to claim 1,
The data integration unit,
a knowledge integrator configured to generate the source instance based on at least one first identifier of at least one knowledge entity included in the mapping table and the transformed source data; and
and a knowledge application unit configured to refine the source instance and integrate the refined source instance into the knowledge base. 4. The method according to claim 3,
The knowledge integration unit,
a knowledge entity selector configured to search for the at least one first identifier in the mapping table and extract the at least one knowledge entity based on a search result; and
and a source instance generator configured to generate a source instance from the transformed source data by changing the at least one first identifier to the at least one second identifier of the at least one knowledge entity extracted. system. 5. The method according to claim 4,
The knowledge entity selection unit is configured to extract at least one candidate knowledge entity from the knowledge base based on the converted source data when the at least one first identifier is not found in the mapping table. integrated system. 4. The method according to claim 3,
The knowledge application unit,
a source instance post-processing unit configured to post-process the source instance based on the types of knowledge instances included in the knowledge base; and
and an instance comparison unit configured to selectively integrate the post-processed source instance into the knowledge base by comparing the post-processed source instance with knowledge instances included in the knowledge base. The method according to claim 1,
The data integration system is
A data management unit configured to provide the data generated by the conversion management unit and the data integration unit to the manager in response to the manager's inquiry request, and to update the metadata in response to the manager's update request Data integration system, characterized in that. As a distributed system for knowledge integration,
a plurality of data conversion systems; and
a data integration system configured to integrate transformed source data provided from the plurality of remote data transformation systems into a knowledge base based on the inter-knowledge domain mapping table;
Each of the plurality of data conversion systems,
a plurality of transformation engines configured to transform source data included in a knowledge domain based on a structure of knowledge instances included in the knowledge base in response to an instruction provided from the data integration system; and
an engine management unit configured to select one transformation module from among a plurality of transformation modules based on the instruction and to provide the selected transformation module to at least one of the plurality of transformation engines;
Each of the plurality of transformation modules defines a transformation rule required according to a knowledge domain or a group of knowledge data included in the knowledge domain. delete 9. The method of claim 8,
The instructions are
The distributed system for knowledge integration, comprising at least one of identification information of the knowledge domain, at least one identifier of source data, and an index of a transformation module. 9. The method of claim 8,
each of the plurality of data transformation systems includes a transformation data store configured to store the transformed source data;
and each of the plurality of transformation engines is configured to collect source data from the knowledge domain and store the transformed source data in the transformation data store.

Images