KR20210079203A - System for multi-layered knowledge base and processing method thereof - Google Patents

Abstract

Provided are a multi-layered knowledge base system and a processing method thereof. In the knowledge base system, semantic space learning of converting learning data into a semantic vector, which is a vector of semantic space, by learning a transformation function based on a plurality of learning data is performed. Then, relational knowledge learning of acquiring a relation between the semantic vectors by learning a relation function based on the semantic vectors obtained by the semantic space learning is performed. The acquired relation is converted into graph knowledge and stored. The graph knowledge uses the relation as an edge and the semantic vector corresponding to the relation as a node. Therefore, various services are made possible with one knowledge base.

Description

A multi-layered knowledge base system and a processing method thereof

The present disclosure relates to a knowledge base, and more particularly, to a multi-layered knowledge base system and a processing method thereof.

The knowledge base is one of the components of an expert system, and is a database in which expert knowledge accumulated through intellectual activities and experiences of an expert in a specific field or facts and rules necessary for problem solving are stored. For this knowledge base, the text-oriented knowledge is stored in the form of a knowledge graph or ontology, and the answer is found to a query through reasoning about it, or the knowledge base is grown through the knowledge base completion technology. . However, this method is mainly suitable for data with clear knowledge such as language-oriented or graph-type data of a text-oriented corpus, that is, learning data having an explicit knowledge expression. As multi-modal learning data such as images/videos/voices are converted into texts by specific learning and then converted into knowledge bases, the data inherent in multi-modal learning data cannot be fully utilized and limited to specific tasks or domains. Only information extracted from learning can be used. In addition, there are disadvantages such as loss of information and knowledge because it is not possible to utilize inherent knowledge through knowledge exchange or convergence between learning data.

In addition, from the viewpoint of continuously adding and growing knowledge to the knowledge base for given learning data, if the knowledge to be added about the previous learning knowledge and the new learning data is fragmentarily composed only in text form, when trying to converge knowledge between them, Only information extracted from learning limited to a specific task or domain can be utilized. In addition, there are disadvantages in that more knowledge cannot be extracted by mutually utilizing information and knowledge between learning data, and effective knowledge fusion is not achieved due to a lack of mutual correlation.

The problem to be solved by the present disclosure is to provide a multi-layered knowledge base system that stores multi-layered knowledge information ranging from intrinsic knowledge to explicit knowledge, and a method for searching and extracting knowledge through interaction and connection of each layer. will be.

In addition, an object of the present disclosure is to provide a method capable of inferring and growing knowledge at high speed from a multi-layered knowledge base.

According to an embodiment, a processing method in a knowledge base system is provided. The processing method may include: learning a transformation function based on a plurality of learning data and performing semantic space learning to transform the learning data into a semantic vector that is a vector of a semantic space; performing relational knowledge learning for acquiring a relation between the semantic vectors by learning a relational function based on the semantic vectors obtained according to the semantic space learning; and converting the obtained relation into graph knowledge, wherein the graph knowledge has the relation as an edge and a semantic vector corresponding to the relation as a node, wherein the knowledge base system includes the learning data A learning data layer that stores and manages , a semantic space layer that stores and manages semantic space knowledge that is a semantic vector obtained according to the semantic space learning, and a graph knowledge that stores and manages the graph knowledge acquired according to the relational knowledge learning includes layers.

In one embodiment, the processing method includes: generating semantic space knowledge metadata that maps the learning data used for the semantic space learning, a transformation function, and the acquired semantic vector; and generating graph knowledge metadata for mapping the semantic vector used for the relational knowledge learning, the relational function, and the related graph knowledge, the metadata related to the learning data and the semantic space knowledge metadata; Based on the graph knowledge metadata, search and association between the learning data layer, the semantic space layer, and the graph knowledge layer may be made.

In an embodiment, the learning data may be multimodal data including at least one of an image and a voice.

In an embodiment, the processing method may further include performing inference for each layer of the knowledge base system according to the inference request. In this case, the performing of the inference may include: performing first inference based on graph knowledge included in the graph knowledge layer; performing second inference based on semantic space knowledge included in the semantic space layer; and performing a third inference based on the training data included in the training data layer.

In one embodiment, the reliability for the first inference, the reliability for the second inference, and the reliability for the third inference are given differently, respectively, and based on the different confidence levels, the result of the first inference, the second The method may further include obtaining a final reasoning result based on the result of the inference and the result of the third reasoning.

In an embodiment, in the performing of the inference, the inference may be performed in parallel for each of a plurality of domains.

In an embodiment, the performing of the speculation may include selectively performing at least one of performing the first speculation, performing the second speculation, and performing the third speculation according to a set speculation depth. can be done

In an embodiment, the performing of the speculation may include selectively performing at least one of performing the first speculation, performing the second speculation, and performing the third speculation according to a set time. can do.

In one embodiment, the performing of the inference may include generating new knowledge through multi-layered knowledge inference based on input learning data or input fact information.

In one embodiment, the generating of the new knowledge includes finding similar training data in the training data layer for input training data, and a semantic vector corresponding to the similar training data in the semantic space layer and transformation thereof extracting a function; inferring a new relation by performing predictive inference based on the relation function with respect to the extracted semantic vector; and generating graph knowledge for the inferred new relationship and storing the graph knowledge in the graph knowledge layer.

In one embodiment, the generating of the new knowledge may include: verifying a previous prediction inference result based on input fact information; modifying a relational function corresponding to the inference result based on the verification result; inferring a new relationship by performing predictive inference based on the modified relationship function; and generating graph knowledge for the inferred new relationship and storing the graph knowledge in the graph knowledge layer.

According to another embodiment, a multi-layered knowledge base system is provided. The multi-layered knowledge base system may include: an interface device configured to receive data; a storage device configured to store knowledge information; and a processor configured to form a knowledge base based on the data, wherein the processor learns a transformation function based on a plurality of learning data input through the interface device to convert the learning data into a semantic vector that is a vector of a semantic space. a semantic space learning unit configured to perform transforming semantic space learning; and a relational knowledge learning unit configured to learn a relational knowledge based on the semantic vectors obtained according to the semantic space learning to obtain a relation between the semantic vectors, wherein the relational knowledge learning unit further comprises: and transform the obtained relationship into graph knowledge, wherein the graph knowledge has the relationship as an edge and a semantic vector corresponding to the relationship as a node.

In one embodiment, the storage device comprises: a training data storage unit configured to store a training data layer for storing and managing the training data; a semantic space knowledge storage unit configured to store a semantic space layer for storing and managing semantic space knowledge, which is a semantic vector obtained according to the semantic space learning; and a graph knowledge storage unit configured to store a graph knowledge layer that stores and manages graph knowledge acquired according to the relational knowledge learning.

In an embodiment, the processor generates learning data metadata related to the learning data, and generates semantic space knowledge metadata that maps the learning data used for the semantic space learning, a transformation function, and the obtained semantic vector, The method may further include a knowledge element manager configured to generate graph knowledge metadata for mapping the semantic vector used for learning the relational knowledge, the relational function, and the related graph knowledge. In this case, based on the learning data metadata, the semantic space knowledge metadata, and the graph knowledge metadata, a search and linkage between the learning data layer, the semantic space layer, and the graph knowledge layer stored in the storage device is performed. can

In one embodiment, the processor may further include an inference processing unit configured to perform inference for each layer of the knowledge base system according to the inference request. The reasoning processing unit may include: performing first reasoning based on graph knowledge included in the graph knowledge layer; performing second inference based on semantic space knowledge included in the semantic space layer; And it may be configured to perform at least one of the operation of performing a third inference based on the learning data included in the learning data layer.

In one embodiment, the inference processing unit additionally grants different confidence levels for the first inference, the second inference, and the third inference, and the first inference based on the different confidence levels. As a result of , the operation of obtaining a final reasoning result based on the result of the second reasoning and the result of the third reasoning may be performed.

In an embodiment, the inference processing unit may include a plurality of prediction speculators configured to perform speculation for each domain, and the plurality of prediction speculators may be configured to perform speculation in parallel.

In an embodiment, the inference processing unit is configured to perform the first inference, the second inference, and the third inference according to at least one of a set speculation depth and a set time. It may be configured to selectively perform at least one.

In an embodiment, the inference processing unit may be configured to generate new knowledge through multi-layered knowledge inference based on input learning data or input fact information.

In an embodiment, the inference processing unit is configured to infer new knowledge by performing inference on a plurality of relational functions according to the first inference and weighting the results when performing the operation of performing the first inference, or , or, when performing the operation of performing the second inference, inferring new knowledge with a semantic vector similar to the semantic vector of valid knowledge in the semantic space according to the second inference and weighting the result, or , or when performing the operation of performing the third inference, inferring new knowledge by applying a weight to the result by performing inference using a relational function connected to similar learning data using the degree of similarity between learning data according to the third inference can be configured.

According to the embodiments, the inherent information of multi-modal learning data such as images/videos/voices as well as text-based data such as language or graphs and their interrelated information are extracted, and based on this, information is added to existing knowledge Without loss of knowledge, not only explicit knowledge but also intrinsic knowledge can be fused together to form a knowledge base to grow the knowledge base.

In addition, it is possible to speed up the knowledge base by limiting the degree to which intrinsic information is used to infer and grow knowledge at high speed. Therefore, since the multimodal learning data is continuously used for learning, it is possible to grow knowledge while reducing the loss of knowledge and information about previously learned knowledge even with respect to heterogeneous learning data and learning tasks. That is, it is possible to effectively converge and make knowledge of learning data sets having heterogeneous domains and tasks while reducing knowledge loss.

In addition, more information is accumulated in the knowledge base by applying more effective and diverse learning tasks and learning data in multi-modal knowledge inference, so that various services are possible with one knowledge base. In addition, since knowledge transfer occurs between heterogeneous domains and tasks, performance improvement such as accuracy can be expected for each individual task. In addition to knowledge inference, the knowledge generated in this way can be used for various services such as multi-modal data management and search, retrieval, conversation generation, knowledge extraction, and the like.

1 is a diagram illustrating a structure of a multi-layered knowledge base system according to an embodiment of the present disclosure.
2 is a schematic diagram illustrating semantic spatial knowledge metadata management according to an embodiment of the present disclosure.
3 is a schematic diagram illustrating graph knowledge metadata management according to an embodiment of the present disclosure.
4 is a diagram illustrating the concept of multi-layered knowledge management according to an embodiment of the present disclosure.
5 is an exemplary diagram illustrating a multi-layered knowledge search according to an embodiment of the present disclosure.
6 is an exemplary diagram illustrating a similarity measurement of learning data according to an embodiment of the present disclosure.
7A to 7C are diagrams illustrating examples of performing inference according to an embodiment of the present disclosure.
8 to 11 are exemplary views illustrating metadata according to an embodiment of the present disclosure.
12 is a diagram illustrating an example of functionally classifying a multi-layered knowledge base system according to an embodiment of the present disclosure.
13 and 14 are flowcharts of a processing method according to an embodiment of the present disclosure.
15 and 16 are exemplary diagrams illustrating knowledge growth based on input of new fact information, according to an embodiment of the present disclosure.
17 is a diagram illustrating an example of inferring knowledge generated for each domain in parallel and fusing the results to derive a result, according to an embodiment of the present disclosure.
18 is a diagram illustrating an example in which inference is performed by adjusting the depth of proceeding to a lower layer in multi-layer knowledge inference according to an embodiment of the present disclosure.
19 is a structural diagram illustrating a computing device for implementing a processing method according to an embodiment of the present disclosure.

Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

In the present specification, expressions described in the singular may be construed in the singular or plural unless an explicit expression such as “a” or “single” is used.

In addition, terms including an ordinal number such as first, second, etc. used in an embodiment of the present disclosure may be used to describe the components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present disclosure, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, a multi-layered knowledge base system and a processing method thereof according to an embodiment of the present disclosure will be described with reference to the drawings.

1 is a diagram illustrating a structure of a multi-layered knowledge base system according to an embodiment of the present disclosure.

1 , a multi-layered knowledge base system 1 according to an embodiment of the present disclosure includes a learning data storage unit 10 , a semantic space knowledge storage unit 20 , a graph knowledge storage unit 30 , and a meaning It includes a spatial learning unit 40 , a relational knowledge learning unit 50 , a knowledge element management unit 60 , an inference processing unit 70 , and a knowledge retrieval processing unit 80 .

The learning data storage unit 10 is configured to store knowledge source elements that grow knowledge. A plurality of training data exists, and each training data is configured based on a domain and a task. The learning data may be divided into structured data, semi-structured data, and unstructured data. Structured data represents data expressing knowledge in a specific structure, such as a database (DB), Structured Digital Abstracts (SDA), or graph. Unstructured data refers to data that is randomly constructed without a specific structure on the web, etc. Semi-structured data refers to data having a data structure so that knowledge suitable for a specific purpose is embedded based on a task or domain, rather than a structure expressing complete knowledge such as DB or graph, for example, it refers to learning data of machine learning. Such learning data is stored in the learning data storage unit 10 , and is managed as a knowledge source element based on the database schema by the knowledge element management unit 60 .

The training data is used for learning of semantic space and learning graph knowledge, and the data used for knowledge learning includes both single modal data composed only of text such as corpus and multimodal data such as image/video and corpus pair. do. The learning data stored in the learning data storage unit 10 is provided to the semantic space learning unit 40 , the relational knowledge learning unit 50 , and the inference processing unit 70 according to a call.

The semantic space learning unit 40 is configured to learn a function that takes the learning data from the learning data storage unit 10 and transforms it into a semantic space. Here, the transform function transforms the training data into a vector of a semantic space (also called a semantic vector) based on the knowledge inherent in the training data, and may have several forms, such as a neural network, fuzzy mapping, or matrix function. When the training data is single-modal data, graph embedding, word embedding, image embedding, etc. can be used. When the training data is multi-modal data, the individual embeddings are simply connected or embedded in the same way as multi-modal word embedding. can do.

The semantic space knowledge storage unit 20 is configured to store and manage a semantic vector of the learning data learned by the semantic space learning unit 40 . The semantic vector is indexed and managed by the learning data, and metadata corresponding thereto is managed by the knowledge element management unit 60 .

The knowledge element management unit 60 is configured to manage metadata related to a semantic vector indexed by the learning data, that is, semantic space knowledge metadata.

2 is a schematic diagram illustrating semantic spatial knowledge metadata management according to an embodiment of the present disclosure.

Semantic space learning is individually performed on a plurality of training data including the training data 1 to the training data N to obtain corresponding semantic vectors. For example, as shown in FIG. 2 , learning data 1 is transformed into a semantic space through a transform function 1 to obtain a corresponding semantic vector, and learning data N is transformed into a semantic space through a transform function N to obtain a corresponding semantic vector. is obtained Accordingly, a semantic vector set including semantic vectors for a plurality of learning data is obtained. The semantic vector set can also be viewed as a set of semantic space knowledge. As such, the knowledge element management unit 60 includes the learning data used for semantic space learning, a semantic vector that is a vectorized result extracted by learning, a transformation function that is a knowledge model that is a result of learning, and semantic space knowledge that is a corresponding knowledge result. The process of extracting knowledge linking them is represented and managed as metadata, that is, semantic space knowledge metadata.

Meanwhile, the relational knowledge learning unit 50 is configured to learn the relation between the semantic vectors stored in the semantic space knowledge storage unit 20 . The relational knowledge learning unit 50 calls learning data associated with a semantic vector and uses it to learn a relation. For example, a relationship using two semantic vectors as an input is expressed as a matrix or a neural network, learning data associated with a semantic vector is used to learn them, and a label corresponding to the relationship between the semantic vectors is output. A relation function expressed as such a matrix or a neural network outputs an intensity corresponding to the relation for each type of relation between two semantic vectors, and if the strength is greater than a preset threshold, it is regarded as a valid relation.

The relational knowledge about the relationship between the semantic vectors generated by the relational knowledge learning unit 50 is converted into graph knowledge and stored in the graph knowledge storage unit 30 . Graph knowledge consists of graphs with semantic vectors as nodes and relationships as edges.

The knowledge element management unit 60 is configured to manage metadata related to a connection relationship between data used for learning the relationship between these semantic vectors, that is, graph knowledge metadata.

3 is a schematic diagram illustrating graph knowledge metadata management according to an embodiment of the present disclosure.

For a plurality of semantic space knowledges including semantic vectors, that is, semantic space knowledge 1 to semantic space knowledge N, learning is performed on the relation between semantic space knowledges to obtain corresponding graph knowledge. For example, as shown in FIG. 3 , for a plurality of semantic space knowledges, a process of obtaining graph knowledge representing the relationship through a relation function by inputting two semantic space knowledges is performed, respectively, so that the plurality of graph knowledge this is obtained The knowledge element management unit 60 is the knowledge that connects the learning data and semantic vectors (semantic spatial knowledge) used for relational knowledge learning, the relational function that is the knowledge model that is the result of learning, and the graph knowledge that is the corresponding knowledge result. The extraction process is represented and managed as metadata, that is, graph knowledge metadata.

As described above, the knowledge acquired according to the learning performed by the semantic space learning unit 40 and the relational knowledge learning unit 50 based on the learning data may be managed in multiple layers.

4 is a diagram illustrating the concept of multi-layered knowledge management according to an embodiment of the present disclosure.

A training data layer (including multimodal data) that includes the training data that is the basis of learning, and a semantic space layer (multi-dimensional vector space) that includes a semantic space vector (semantic vector) obtained by learning a transform function with the training data , and by learning the semantic space transformation function with the semantic space vector, a multi-layer including a graph knowledge layer including graph knowledge representing the relationship between semantic spaces is formed, and knowledge management can be performed based on the multi-layer.

On the other hand, the inference processing unit 70 is configured to infer a relation through predictive reasoning with respect to a relation that does not exist in the graph knowledge storage unit 30 . Here, the Latent Feature Model method that deals with relational functions that take intrinsic features such as RESCAL and TransE as input, or the Graph Feature model that utilizes the characteristics of the graph itself such as PRA (Path Ranking Algorithm) Model) can be used to predict and infer knowledge other than the graph knowledge generated by the training data. In the case of the intrinsic feature model, similar to acquiring semantic space knowledge, a semantic vector used in the semantic space can be utilized as an intrinsic feature.

Also, the inference processing unit 70 may utilize the relational function used in the relational knowledge learning unit 50 for predictive reasoning as it is. In this case, the knowledge may be grown by using the newly generated triple knowledge, a semantic vector related thereto, or other graph knowledge or relational function connected to the learning data. The newly generated graph knowledge is stored in the graph knowledge storage unit 30 .

The inference processing unit 70 may generate new knowledge by performing multi-layered knowledge search by utilizing information related to semantic spatial knowledge, relational function, transformation function, and learning data connected to graph knowledge metadata.

5 is an exemplary diagram illustrating a multi-layered knowledge search according to an embodiment of the present disclosure.

First, inference is performed using a plurality of relational functions (also called first inference for convenience of description, (FIG. 5(a)). After multiple knowledge growth processes, multiple relational functions are obtained for the same knowledge. Based on these multiple relational functions, we infer knowledge using graph reasoning techniques (inference using graph characteristic model), for example, we infer knowledge based on relational functions for _{arbitrary knowledge (SPO) 1. Multiple relations} The inference results using the functions (eg, (SPO) _r1 and (SPO) _r2 in Fig. 5 ) may be different from each other, in this case, the inference results (a new {(SPO) _r } set) used in learning An SPO triple (Subject-Predicate-Object Triple) having a larger value may be selected by normalizing the magnitude of the energy function and comparing the normalized values, but the present disclosure is not limited thereto.

Next, inference using semantic space knowledge is performed (also called second inference for convenience of description, (b) of FIG. 5 ). Select a semantic vector in the semantic space (the semantic vector of a valid SPO triple) connected to the relational function used in the first reasoning above and a similar semantic vector in the semantic space, and based on the relational function corresponding to these semantic vectors, a new knowledge SPO Create a triple (eg (SPO) _m1 ). In this case, in order to measure the similarity between the semantic vectors, various methods such as cosine similarity or Hamming distance can be used.

Then, inference is performed using the learning data (also referred to as third inference for convenience of explanation, (c) of FIG. 5 ). Find other similar training data based on training data (learning data related to inference in (a) and (b) of FIG. 5 above), and perform predictive reasoning on semantic spatial knowledge, relational function, and graph knowledge connected with similar training data can do. _{That is, new knowledge (eg, (SPO) d} ) is acquired by finding a semantic vector connected to learning data similar to the learning data, and performing predictive inference based on a relational function corresponding to the found semantic vectors. In this case, the aforementioned predictive inference methods may be used. On the other hand, when searching for similar training data, similar training data can be found by vectorizing the properties of the training data and comparing them using a method such as cosine similarity or Hamming distance. Alternatively, similar training data may be found by using various methods such as using a similarity measurement table between training data in advance. When obtaining the similarity with the semantic vector of the effective knowledge of the original space in the semantic space related to similar learning data, the dimension of the higher-order semantic vector is reduced, and then the dot product between the two reduced dimensions and the similarity between the corresponding learning data The similarity is obtained by multiplying, and based on the obtained similarities, a semantic vector having the greatest similarity among semantic vectors of a plurality of learning data, that is, the most similar semantic vector is selected, and new knowledge is obtained using a relational function related to the selected semantic vector. infer

6 is an exemplary diagram illustrating a similarity measurement of learning data according to an embodiment of the present disclosure.

For example, since the degree of similarity between the training data D _1, D _2, D _S one time, D ₂ is closer than the D ₁ for the learning data of the D _S, the training data is selected for D _2.

It is possible to generate new knowledge as shown in FIG. 5C by performing inference using the similarity of the learning data.

As described above, the first reasoning, the second reasoning, and the third reasoning are performed on the reasoning request to find the reasoning request, that is, the answer to the question. In this case, different weights may be assigned to each of the first inference result, the second inference result, and the third inference result, and a final inference result may be derived based on the assigned weight. For example, since it is an inference from knowledge that is more certain as you go to a higher layer, high reliability is given to the inference result of the upper layer, and the final reliability of individual inference results through the product of the weight and reliability used in the inference from each layer, etc. (Overall weight) may be obtained, and an inference result having a reliability greater than or equal to a threshold may be finally used among final reliability values of individual inference results. Such hierarchical reliability can be arbitrarily set by the knowledge base designer or determined using the accuracy of inference based on statistics.

Specifically, inferring a plurality of relational functions according to the first inference may be performed, and new knowledge may be inferred by weighting the results.

In addition, according to the second inference, inference is performed with a semantic vector similar to the semantic vector of valid knowledge in the semantic space, and new knowledge can be inferred by weighting the result.

In addition, according to the third inference, by using the degree of similarity between the learning data, inference using a relational function connected to similar learning data may be performed, and new knowledge may be inferred by weighting the result.

7A to 7C are diagrams illustrating examples of performing inference according to an embodiment of the present disclosure.

As an inference query, we can consider inference given S-P and finding valid O, or given S, O and finding valid P, or given P, O and finding valid S.

The relational function learns the validity of P, and the semantic space learns the vectorization of entities S and O.

For example, as shown in FIG. 7A , when S _q , P _q is given as an inference query, a valid O is obtained through a relational function during the first inference to obtain an inference result (SPO) _r1 . Here, the first inference outputs the validity of P through the relational function, aligns the inference results with respect to the validity, and uses the validity value as a weight. For example, a relation function value 0.9, which is the validity value of P for _{S q} -P _q -O deduced based on the r1 relational function, is obtained, and this is used as the relation function weight.

Meanwhile, the second reasoning uses the similarity of entities in the semantic space as a weight for S or P used in the query. For example, as shown in Figure 7b, when the S _q, P _q given as an inference query, the second inference when, S _'q is selected having a mean vector with a 0.8 degree of similarity in the S _q mean vector and a mean space and, S 'based on the m1 related functions for _{q S' q -P q -O (} (SPO) m 1) is is obtained, is obtained between the functional value of 0.9 for this. Here, a similarity of 0.8 (semantic spatial similarity) may be used as a weight for the second inference result.

Meanwhile, in the second inference, when the semantic space similarity is equal to or greater than the threshold, the validity value of the relational function is used, and the similarity between entities is multiplied by the validity value of the relational function to be used as a weight for the inference result. For example, a value of 0.72 obtained by multiplying a relation function value of 0.9 by a similarity of 0.8 may be used as a weight for the second inference result.

On the other hand, the third inference sets the learning data connected by the inference result and metadata with the greatest relational function validity to the inference query as the basic training data, and uses the training data and metadata that have a similarity with the basic training data over a certain threshold. For the connected knowledge graph and semantic space knowledge, the first reasoning and the second reasoning are executed to induce the third reasoning. In this case, the similarity between the training data may be multiplied by the weight and used as a weight for the inference result. Accordingly, the inherent knowledge can be utilized for inference by using multi-layered information through semantic spatial similarity and learning data similarity.

For example, as in FIG. 7C , when S _q , P _q is given as an inference query, the inference result (eg, S _q -P) having the greatest relational function validity in the inference result (SPO) _{d according to the first inference q} -O) and connected learning data are set as basic training data, training data having a similarity of 0.7 to the basic training data is selected from among the training data, and a semantic vector connected to the selected training data and a semantic vector having a similarity of 0.8 is selected. with S _'q is selected, S' based on the function of the relation r2 _q is obtained an _{S 'q -P q -O ((} SPO) r2), is obtained between the functional value of 0.9 for this. Here, the value 0.504 obtained by multiplying the similarity 0.7 between the training data by the relation function value 0.9 and the semantic space similarity 0.8 can be used as a weight for the third inference result.

The final reliability, that is, the overall weight, obtained for each inference performed in this way may be as shown in Table 1 below.

inference layer relational function
weight meaning space
Similarity training data
Similarity hierarchy
reliability general
weight first reasoning 0.9 One One One 0.9 second reasoning 0.9 0.8 One 0.8 0.576 third reasoning 0.9 0.8 0.7 0.7 0.28224

Based on giving high reliability to the inference result of the upper layer, as in the example of Table 1 above, a layer reliability of 1 is given to the layer (or first layer) of the first inference, and the layer of the second inference (or the second layer) may be given a layer reliability of 0.8, and the layer of the third inference (or the third layer) may be given a layer reliability of 0.7. The overall weight, which is the final reliability, is obtained by multiplying the weight used in inference from each layer by the layer reliability. For example, the overall weight (0.9) obtained by multiplying the weight 0.9 for the first inference by the layer reliability 1 of the first layer in which the inference is performed, and the layer reliability 1 of the first layer in which the inference is performed to the weight 0.72 for the second inference The overall weight (0.576) multiplied by the layer reliability 0.8 of the second layer, and the weight 0.504 for the third inference, the layer reliability 1 of the first layer and the layer reliability 0.8 of the second layer and the layer reliability of the second layer, which are the reliability of all layers in which the inference was performed A total weight (0.28224) obtained by multiplying the layer reliability of the three layers by 0.7 is finally obtained. In addition, only an inference result having a final reliability equal to or greater than a threshold value among the finally obtained overall weights, that is, the final reliability values, may be used as a valid inference result.

This multi-layered knowledge reasoning can be used to grow knowledge. By performing multi-layered knowledge inference from new facts extracted from interaction information such as new learning data or conversations, it is possible to complete the missing knowledge about the current learning data and grow knowledge. This will be described in more detail later.

Meanwhile, the types of knowledge that can be extracted according to the type of learning data and the types of knowledge extraction are shown in Table 2 below.

training data type training data meaning space graph knowledge predictive reasoning structural ○ X ○ ○ semi-structural ○ ○ ○ ○ unstructured ○ △ △ △

Referring to Table 2, there is no semantic space knowledge because structural learning data can be directly converted into graph knowledge. Accordingly, in predictive reasoning, this part can be used for inference by identifying the same valid entity using the entity resolution technique between graphs.

Since knowledge of all layers exists in semi-structured learning data, knowledge inference is possible using all of the techniques according to an embodiment of the present disclosure.

Since unstructured learning data does not have labels or tagging with human supervised knowledge in the learning data, semantic space and graph knowledge must be constructed using unsupervised learning techniques or self-supervised learning techniques. do. If such upper layer knowledge is configured, knowledge inference is possible using all of the techniques according to an embodiment of the present disclosure.

Meanwhile, the knowledge element management unit 60 is configured to connect learning data, a transformation function, a semantic vector set (semantic space knowledge) of a semantic space, a relational function, a graph knowledge set, and the like, and store it in the database as metadata and manage it.

When each knowledge element is needed in the process of knowledge growth through knowledge prediction inference, the knowledge element management unit 60 provides corresponding metadata in response to a call of another element so that the corresponding element acquires the knowledge element.

8 to 11 are exemplary views illustrating metadata according to an embodiment of the present disclosure.

As exemplified in FIG. 8 , the training data metadata is associated with multimodal learning data such as image/corpus/audio in correspondence with the identifier (eg, dataset_id) of one data set. , a data identifier (eg imageset_id) and a file (eg imagefile) correspond to each training data. In addition, new knowledge according to predictive reasoning is linked.

As illustrated in FIG. 9 , the semantic space knowledge metadata includes a semantic vector and a transformation function (eg, neural_net) linked to the learning data, and a vector value, an identifier (vector_id), etc. This corresponds to In addition, new knowledge according to predictive reasoning is linked.

As illustrated in FIG. 10 , the relational knowledge learning metadata corresponds to an inference result (eg, prediction_id) related to prediction inference using a relational function.

As illustrated in FIG. 11, graph knowledge metadata is based on a graph with a semantic vector as a node and a relation as an edge, and in connection with this graph knowledge, a relation function, a semantic vector associated therewith, Learning data, new graph knowledge based on predictive inference, etc. are linked.

Meanwhile, the knowledge retrieval processing unit 80 is configured to provide corresponding knowledge in response to a knowledge request. The knowledge retrieval processing unit 80 performs a graph search in the graph knowledge storage unit 30 according to a knowledge request to determine whether the corresponding knowledge exists, that is, whether a knowledge triple (SPO) exists, and if the corresponding knowledge exists, it is converted into a graph. It is fetched from the knowledge storage unit 30 and output. If there is no corresponding knowledge, the knowledge retrieval processing unit 80 may search for knowledge elements related to the corresponding knowledge through the inference processing unit 70 to search for a plurality of other graphs as well.

As described above, the learning data storage unit 10, the semantic space knowledge storage unit 20, the graph knowledge storage unit 30, the semantic space learning unit 40, the relational knowledge of the multi-layered knowledge base system 1 as described above. The functional classification of the learning unit 50 , the knowledge element management unit 60 , the inference processing unit 70 , and the knowledge retrieval processing unit 80 is shown in FIG. 12 .

12 is a diagram illustrating an example of functionally classifying a multi-layered knowledge base system according to an embodiment of the present disclosure.

The knowledge generating block performs a function of generating knowledge by receiving learning data, which is a knowledge source element, as an input, and a semantic space for generating knowledge by receiving learning data, which is a knowledge source element, as an input from the learning data storage unit 10 of FIG. 1 . It may include a learning unit 40 , a relational knowledge learning unit 50 , and an inference processing unit 70 .

The knowledge element management block performs a function of managing metadata of knowledge of each layer, and may correspond to the knowledge element management unit 60 of FIG. 1 .

The knowledge storage block performs a function of storing the generated multi-layered knowledge, and may include the graph knowledge storage 30 and the semantic space knowledge storage 20 of FIG. 1 . Alternatively, the learning data storage unit 10 may be included.

The knowledge retrieval block performs a function of retrieving and providing knowledge from the knowledge storage block according to a knowledge request, and corresponds to the knowledge retrieval processing unit 80 of FIG. 1 .

Next, a processing method according to an embodiment of the present disclosure based on a multi-layered knowledge base system having such a structure will be described.

13 is a flowchart of a processing method according to an embodiment of the present disclosure.

In an embodiment of the present disclosure, a processing method of growing knowledge is performed based on a multi-layered knowledge base system, and the growth of knowledge can be achieved through a method of completing graph knowledge generated from learning data through multi-layer predictive inference. .

13, when learning data is input (S100), a semantic space transformation function and semantic space knowledge for the learning data are extracted from the semantic space knowledge storage unit 20 (S110). For example, training data similar to the input training data is found. In this case, similar training data may be found based on the similarity between the training data. And the semantic space knowledge (semantic vector) and transformation function connected with the found learning data are extracted.

Thereafter, the relational knowledge learning function for the semantic space is improved (S120). For example, the relation function corresponding to the semantic vector corresponding to the extracted semantic space knowledge is improved (modified). For example, if there are positive and negative cases in the training data, learning about the relational function to predict the validity of the SPO relationship based on them, or by changing the relational function parameter to improve a specific scale, can be improved Knowledge graph completion techniques such as RESCAL, TransE, and PRA (Path Ranking Algorithm) of the knowledge graph may be used.

Then, graph knowledge is generated by inferring a relationship through predictive inference (S130, S140). For example, the relationship is inferred through predictive inference with respect to a relationship that does not exist based on the relationship function improved in step 120 or the relationship function extracted in step S110 . New graph knowledge is generated according to this relational inference. That is, new graph knowledge is generated with the newly inferred relationship as the edge and the related semantic vectors as the nodes.

In addition, in another embodiment of the present disclosure, the growth of knowledge may be achieved by generating new knowledge from newly generated fact information.

14 is a flowchart of a processing method according to an embodiment of the present disclosure.

14, when new fact information is input (S300), the previous prediction inference result is verified (S310). New factual information can also be entered through human interactions, such as new data sets or conversations. The new fact is that it is transformed into a triple form of S-P-O, and each entity is a state in which multimodal data is transformed into a semantic vector in the semantic space. The new fact is responsible for the verification of relationships created by multi-layer predictive inference for relationships that do not exist in the previous training data. Based on the new facts, the existing relationship is verified. For example, with respect to S-P-O for new facts, if the overall weight of the relation function is less than or equal to a certain value, the relation created through the method of re-learning the relation function including the new fact may be verified. Or, for the SPO linked to the SPO for the new fact, the average effective value (the effective value is calculated using a relational function using each of S, P, and O, and the average of the valid values) is less than or equal to a specific value Further learning of the relation function can be performed to verify the created relation.

Thereafter, the learning function (relation function) of the relational knowledge is modified so that the erroneously predicted relation is discarded according to the verification result. That is, the relational knowledge learning function for the semantic space, that is, the relational function is improved ( S320 ). On the other hand, new facts can also be used as input for determining a threshold value when improving a learning function (relational function) of relational knowledge.

And graph knowledge is generated by inferring a relationship through predictive inference (S330, S340). New facts can act as learning data for new relational knowledge learning, triggering incomplete relationships in existing graph knowledge. Therefore, based on the new relational knowledge function, that is, the relational function improved in step S320, new knowledge is generated and grown through multi-layer prediction inference.

To explain this process with a specific example, a new fact is converted into a triple form of the SPO, a similar semantic vector is found from the semantic space knowledge storage 20 based on the semantic vector of the converted triple of the SPO, and the A new knowledge SPO triple is generated based on the relational function using the semantic vector and similar semantic vectors found. In addition, new knowledge can be generated by improving the relational function used to generate the new knowledge S-P-O triple, and performing new predictive reasoning based on the improved relational function.

Generating new knowledge from new facts (fact information) can be performed per domain and task.

15 and 16 are exemplary views illustrating knowledge growth based on input of new fact information according to an embodiment of the present disclosure.

For example, as shown in FIG. 15 , a plurality of tasks and heterogeneous learning data having different domains may be continuously input to a multi-layered knowledge base system and used to fuse individual knowledge.

Also, as shown in FIG. 16 , knowledge can be grown by extracting knowledge from an interaction including a human-to-human or human-machine conversation of an interactive system, and inputting it as fact information into a multi-layered knowledgebase system.

The multi-layered knowledge base knowledge inference and growth by the processing method according to an embodiment of the present disclosure is performed using metadata information for a connection relationship between knowledge layers.

In this case, considering that there are very many connections and it takes a lot of time because the amount of computation increases when predictive inference is performed using them, in an embodiment of the present disclosure, additionally, inference is executed on the knowledge generated for each domain in parallel and the A method of deriving a result by converging the result and a method of shortening the reasoning time by adjusting the depth from multi-layered knowledge reasoning to the lower layer are provided. All of these methods reduce the amount of inferred knowledge compared to the case of performing predictive inference on the entire knowledge base. In addition, the above two methods can be equally applied to the case of knowledge retrieval.

17 is a diagram illustrating an example of inferring knowledge generated for each domain in parallel and deriving a result by fusing the result according to an embodiment of the present disclosure.

For example, as shown in FIG. 17 , a prediction reasoner may be configured for each domain. For example, the inference processing unit 70 may be configured to include a prediction reasoner for each domain.

Predictive reasoners may run in parallel within the same machine, or in parallel through a distributed machine. In the case of merging the inference results of the prediction reasoners, if the representation of the S-P-O triple of each prediction reasoner is different, the inference results may be merged using an entity resolution technique. In addition, in the case of inference, query, knowledge search, and retrieval, a method of extracting task information and giving priority to the inference result of a predictive reasoner suitable for the extracted task among the inference results may also be used.

18 is a diagram illustrating an example in which inference is performed by adjusting the depth of progressing to a lower layer in multi-layer knowledge inference according to an embodiment of the present disclosure.

Inference execution time can be controlled by limiting the depth of reasoning from multi-layer knowledge reasoning to lower layers. The depth of reasoning can be limited according to a time that satisfies the response time or inference request time required for reasoning, query, knowledge search, and retrieval. If there is a time limit, the search layer is limited accordingly, so that the search, that is, inference can be performed only on the searchable layer within the time.

In addition, the number of inferences may be limited by the number of connections of metadata. In this case, various methods, such as a method of calculating a total score by weighting the connection of metadata for each layer, and limiting connection of metadata based on the score, may be used. In addition, it is also possible to determine the value for the limit depth by inferring the relationship between the required response time and the predicted inference time using a query or regression analysis between the task and the limit depth, or a neural network such as a recurrent neural network (RNN). Do.

According to this embodiment of the present disclosure, as the existing knowledge base has a structured or explicit knowledge expression and processing method such as a text-oriented database, corpus, graph, etc., the problem of loss of information and knowledge of multi-modal learning data is solved. It can be solved, and knowledge extracted from multimodal learning data can be fused with not only explicit knowledge but also intrinsic knowledge to form a knowledge base to grow the knowledge base.

19 is a structural diagram illustrating a computing device for implementing a processing method according to an embodiment of the present disclosure.

19 , the processing method according to an embodiment of the present disclosure may be implemented using the computing device 100 .

The computing device 100 may include at least one of a processor 110 , a memory 120 , an input interface device 130 , an output interface device 140 , a storage device 150 , and a network interface device 160 . . Each of the components may be connected by a bus 170 to communicate with each other. In addition, each of the components may be connected through an individual interface or a separate bus centering on the processor 110 instead of the common bus 170 .

The processor 110 may be implemented in various types such as an application processor (AP), a central processing unit (CPU), a graphic processing unit (GPU), and the like, and executes a command stored in the memory 120 or the storage device 150 . It may be any semiconductor device that does The processor 110 may execute a program command stored in at least one of the memory 120 and the storage device 150 . The processor 110 may be configured to implement the functions and methods described based on FIGS. 1 to 18 above. For example, the processor 110 may be configured to perform functions of a semantic space learning unit, a relational knowledge learning unit, a knowledge element management unit, an inference processing unit, and a knowledge retrieval processing unit.

The memory 120 and the storage device 150 may include various types of volatile or non-volatile storage media. For example, the memory may include a read-only memory (ROM) 121 and a random access memory (RAM) 122 . In an embodiment of the present disclosure, the memory 120 may be located inside or outside the processor 110 , and the memory 120 may be connected to the processor 110 through various known means. The storage device 160 may be configured to include, for example, a learning data storage unit, a semantic space knowledge storage unit, and a graph knowledge storage unit.

The input interface device 130 is configured to provide data to the processor 110 , and the output interface device 140 is configured to output data from the processor 110 .

The network interface device 160 may transmit or receive signals with other dogs through a wired network or a wireless network.

The computing device 100 having such a structure is named as a multi-layered knowledge base system, and may implement a processing method according to an embodiment of the present disclosure.

In addition, at least a part of the processing method according to an embodiment of the present disclosure may be implemented as a program or software executed in the computing device 100 , and the program or software may be stored in a computer-readable medium.

In addition, at least a part of the processing method according to an embodiment of the present disclosure may be implemented as hardware capable of being electrically connected to the computing device 100 .

Embodiments of the present disclosure are not implemented only through the apparatus and/or method described above, but may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present disclosure, a recording medium in which the program is recorded, etc. Also, such an implementation can be easily implemented by those skilled in the art to which the present disclosure pertains from the description of the above-described embodiments.

Although the embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improved forms of the present disclosure are also provided by those skilled in the art using the basic concept of the present disclosure as defined in the following claims. is within the scope of the right.

Claims (20)

A processing method in a knowledge base system, comprising:
performing semantic space learning for transforming the learning data into a semantic vector that is a vector of a semantic space by learning a transformation function based on a plurality of learning data;
performing relational knowledge learning for acquiring a relation between the semantic vectors by learning a relational function based on the semantic vectors obtained according to the semantic space learning; and
converting the obtained relation into graph knowledge, wherein the graph knowledge has the relation as an edge and a semantic vector corresponding to the relation as a node.
including,
The knowledge base system includes a learning data layer for storing and managing the learning data, a semantic space layer for storing and managing semantic space knowledge that is a semantic vector obtained according to the semantic space learning, and a graph obtained according to the relational knowledge learning. A processing method comprising a graph knowledge hierarchy for storing and managing knowledge. According to claim 1,
generating semantic space knowledge metadata for mapping the learning data used for the semantic space learning, the transformation function, and the acquired semantic vector; and
Generating graph knowledge metadata that maps the semantic vector used in the relational knowledge learning, the relational function, and the related graph knowledge
further comprising,
A processing method, wherein search and association between the learning data layer, the semantic space layer, and the graph knowledge layer are made based on the metadata related to the learning data, the semantic space knowledge metadata, and the graph knowledge metadata. According to claim 1,
The learning data is multi-modal data including at least one of an image and a sound, a processing method. According to claim 1,
Further comprising the step of performing inference for each layer of the knowledge base system according to the inference request,
The step of performing the inference is
performing first inference based on graph knowledge included in the graph knowledge layer;
performing second inference based on semantic space knowledge included in the semantic space layer; and
performing a third inference based on the training data included in the training data layer
A method comprising at least one of: 5. The method of claim 4,
Reliability for the first inference, the reliability for the second inference, and the reliability for the third inference are given differently, respectively, based on the different confidence levels, the result of the first inference, the result of the second inference, and the obtaining a final inference result based on the result of the third inference
Further comprising, a processing method. 5. The method of claim 4,
The performing of the inference is a processing method of performing inference in parallel for each of a plurality of domains. 5. The method of claim 4,
The performing of the speculation includes selectively performing at least one of performing the first speculation, performing the second speculation, and performing the third speculation according to a set speculation depth. . 5. The method of claim 4,
The performing of the speculation includes selectively performing at least one of performing the first speculation, performing the second speculation, and performing the third speculation according to a set time. 5. The method of claim 4,
The step of performing the inference is
Step of generating new knowledge through multi-layered knowledge inference based on input learning data or input fact information
comprising, a processing method. 10. The method of claim 9,
The step of generating the new knowledge is,
with respect to the input training data, finding similar training data in the training data layer, and extracting a semantic vector corresponding to the similar training data and a transformation function therefrom in the semantic space layer;
inferring a new relation by performing predictive inference based on the relation function with respect to the extracted semantic vector; and
generating graph knowledge of the inferred new relationship and storing it in the graph knowledge layer;
comprising, a processing method. 10. The method of claim 9,
The step of generating the new knowledge is
verifying a previous prediction inference result based on input fact information;
modifying a relational function corresponding to the inference result based on the verification result;
inferring a new relationship by performing predictive inference based on the modified relationship function; and
generating graph knowledge for the inferred new relationship and storing it in the graph knowledge layer;
comprising, a processing method. an interface device configured to receive data;
a storage device configured to store knowledge information; and
a processor configured to form a knowledge base based on the data
including,
the processor
a semantic space learning unit configured to learn a transformation function based on a plurality of learning data input through the interface device and perform semantic space learning for transforming the learning data into a semantic vector that is a vector of a semantic space; and
A relational knowledge learning unit configured to learn relational knowledge to obtain a relation between the semantic vectors by learning a relational function based on the semantic vectors obtained according to the semantic space learning.
including,
The relation knowledge learning unit is additionally configured to convert the obtained relation into graph knowledge, wherein the graph knowledge has the relation as an edge and a semantic vector corresponding to the relation as a node. 13. The method of claim 12,
the storage device
a learning data storage unit configured to store a learning data layer that stores and manages the learning data;
a semantic space knowledge storage unit configured to store a semantic space layer for storing and managing semantic space knowledge, which is a semantic vector obtained according to the semantic space learning; and
A graph knowledge storage unit configured to store a graph knowledge layer that stores and manages graph knowledge acquired according to the relational knowledge learning
Including, a multi-layered knowledge base system. 14. The method of claim 13,
The processor is
Generating learning data metadata related to learning data, generating semantic space knowledge metadata that maps the learning data used in the semantic space learning, the transformation function, and the obtained semantic vector, and the semantic vector used in the relational knowledge learning , a knowledge element manager configured to generate graph knowledge metadata mapping the relational functions and related graph knowledge.
further comprising,
Based on the learning data metadata, the semantic space knowledge metadata, and the graph knowledge metadata, a multi-layered search and linkage between the learning data layer, the semantic space layer, and the graph knowledge layer, stored in the storage device, is made. knowledge base system. 14. The method of claim 13,
the processor
Further comprising an inference processing unit configured to perform inference for each layer of the knowledge base system according to an inference request,
The inference processing unit,
performing first inference based on graph knowledge included in the graph knowledge layer;
performing second inference based on semantic space knowledge included in the semantic space layer; and
Performing a third inference based on the training data included in the training data layer
A multi-layered knowledge base system, configured to perform at least one of: 16. The method of claim 15,
The inference processing unit additionally,
Reliability for the first inference, the reliability for the second inference, and the reliability for the third inference are given differently, respectively, based on the different confidence levels, the result of the first inference, the result of the second inference, and the A multi-layered knowledge base system, configured to perform an operation of obtaining a final reasoning result based on a result of the third reasoning. 16. The method of claim 15,
The inference processing unit
A plurality of predictive reasoners configured to perform inference on a domain-by-domain basis
including,
The plurality of predictive reasoners are configured to perform inference in parallel, a multi-layered knowledge base system. 16. The method of claim 15,
The inference processing unit,
configured to selectively perform at least one of performing the first speculation, performing the second speculation, and performing the third speculation according to at least one of a set speculation depth and set time, A multi-tiered knowledge base system. 16. The method of claim 15,
The inference processing unit is configured to generate new knowledge through multi-layer knowledge inference based on input learning data or input fact information. 16. The method of claim 15,
The inference processing unit,
configured to infer new knowledge by performing reasoning on a plurality of relational functions according to the first reasoning and weighting the results when performing the operation of performing the first reasoning, or
When performing the operation of performing the second inference, the second reasoning is configured to infer new knowledge by inferring a semantic vector similar to that of valid knowledge in the semantic space according to the second inference, and weighting the result; or
When performing the operation of performing the third inference, the third reasoning is configured to infer new knowledge by using the similarity between the learning data to proceed with inference using a relational function connected to the similar learning data, and by weighting the result , a multi-layered knowledge base system.

Images