KR20220026111A - Knowledge completion method and apparatus through neural symbolic-based rule generation - Google Patents

Abstract

The present invention discloses a knowledge completion method and a device through neural symbolic-based rule generation. The present invention comprises: a processor; and a memory connected to the processor. The memory includes program commands, which can be executed by the processor to generate an embedded knowledge base by embedding a triple and rule template included in an incomplete knowledge base in a multi-dimensional space based on neural symbolic, to allow a neural theorem prover (NTP) to obtain a new inference rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition through embedding learning of updating the embedded knowledge base, and to complete knowledge by applying the new inference rule to the incomplete knowledge base. Therefore, provided are a knowledge completion method and a device through neural symbolic-based rule generation, wherein knowledge can be completed efficiently and accurately.

Description

Knowledge completion method and apparatus through neural symbolic-based rule generation

The present invention relates to a method and apparatus for completing knowledge through neural symbol-based rule generation, and more particularly, to a method and apparatus for building a high-quality knowledge base by completing missing knowledge in the knowledge base.

Recently, the number of cases of providing various services using the knowledge base is increasing. Accordingly, the need to secure a high-quality knowledge base is emerging.

Among the existing knowledge completion techniques, link prediction predicts a Head entity that is the ? part of (?, Predicate, Object) or predicts a Tail entity that corresponds to the ? of (Head, Relation, ?). Here, ? means missing elements.

Assuming that E is all entities included in the knowledge base, the method of determining the link prediction result is to create a negative triple by replacing Head or Tail with all entities e ∈ E for the correct triple (Head, Relation, Tail), Through the Score function f(Head, Relation, Tail) for each of the correct and negative triples, the reliability score is obtained and ranked in descending order.

In this case, the average rank (MR), which is an index that can determine the performance of the model, and Hit@10, which means the ratio of correct triples in the top 10 items, are obtained. Hit@1 is an indicator of when the correct triple is the ratio with the highest reliability score, which is equivalent to the accuracy of the model.

KG-BERT is a representative example of performing link prediction, which is currently known to have the best performance.

1 is a view showing a KG-BERT according to the prior art.

KG-BERT is a neural network model that is fine-tuned to use the BERT model, a natural language processing model announced by Google in 2019, for tasks such as link prediction using a knowledge base. Link prediction using KG-BERT proceeds as follows.

For example, if a triple such as (Steven Jobs, founded, Apple Inc.) is input, each entity and relation in the triple is converted into a token consisting of a corresponding description and used as input to the model, and all tokens are converted into multidimensional vectors. will become Description tokens for each entity and relation are distinguished by a special token called [SEP]. [CLS] is a special token that is always located at the beginning of the model input, and the model output corresponding to this token can be called the reliability score for the triple.

The conventional knowledge completion model embeds a knowledge base, defines a specific objective function, or performs a classification task on triples to learn embedding and neural networks.

In the case of KG-BERT, when performing link prediction, BERT model operation is performed on correct triples (Head, Relation, Tail) and negative triples in which Head or Tail entities are converted into all entities of the knowledge base. In the case of 100 million and BERT Large, an operation must be performed between the parameters corresponding to 300 million and the sentence vector in which the triple is tokenized.

Performing knowledge completion using such a neural network model is computationally inefficient than inference using inference rules because it is necessary to perform vector-to-vector operation and neural network operation. In addition, when performing knowledge completion using conventional reasoning rules, a person must directly check data and define inference rules, but it is impossible to directly define and verify all inference rules for a vast amount of knowledge base.

Republic of Korea Patent 10-2091240

In order to solve the problems of the prior art, the present invention intends to propose a knowledge completion method and apparatus through neural symbol-based rule generation capable of efficiently and accurately performing knowledge completion.

In order to achieve the above object, according to an embodiment of the present invention, there is provided an apparatus for completing knowledge through neural symbol-based rule generation, comprising: a processor; and a memory connected to the processor, wherein the memory generates an embedded knowledge base by embedding triples and rule templates included in the incomplete knowledge base in a multi-dimensional space based on a neural symbol, Neural Theorem Prover (NTP) A new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition is obtained through embedding learning to update the embedded knowledge base, and the new reasoning rule is applied to the incomplete knowledge base to complete knowledge To do so, a knowledge completion device including program instructions executable by the processor is provided.

The update of the embedded knowledge base is updating the Predicate embedding vector value of the rule template.

The NTP may include an OR module, an Unify module, and an AND module, and the Unify module may be called by the OR module to match a target triple of a logical element to be verified and a rule head of the rule template.

The Unify module may perform the matching process using Neural Symbolic Unification.

The AND module creates a new goal using the entity matched in the Unify module, performs Unify between the created new goal and the entire knowledge base to complete a new body, and repeats the same operation for the new body to create a tree can form.

The NTP may include an aggregation module and select the new inference rule as the most semantically matching inference rule having the target triple as a rule head.

According to another aspect of the present invention, there is provided a method of completing knowledge through neural symbol-based rule generation in a device including a processor and a memory, (a) using triples and rule templates included in an unfinished knowledge base in a multidimensional space based on a neural symbol generating an embedded knowledge base by embedding it in ; (b) acquiring, by a Neural Theorem Prover (NTP), a new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition through embedding learning to update the embedded knowledge base; and (c) applying the new reasoning rule to the incomplete knowledge base to complete the knowledge.

Step (b) may include updating a Predicate embedding vector value of the rule template.

According to another aspect of the present invention, there is provided a computer readable program for performing the above method.

According to another aspect of the present invention, there is provided a knowledge completion system through neural symbol-based rule generation, comprising: an incomplete knowledge base; an embedded knowledge base generated by embedding triples and predefined rule templates included in the incomplete knowledge base in a multidimensional space based on a neural symbol; Neural Theorem Prover (NTP) that acquires a new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition through embedding learning to update the embedded knowledge base; and an inference engine that completes knowledge by applying the new reasoning rule to the incomplete knowledge base.

According to the present invention, an inference rule capable of inferring a new triple corresponding to a conclusion part through a triple corresponding to a conditional part of an inference rule is learned and generated from a knowledge base using NTP, and computational based on the generated inference rule It has the advantage of being able to complete knowledge efficiently and accurately.

1 is a view showing a KG-BERT according to the prior art.
2 is a diagram illustrating the overall structure of a knowledge completion model using a neural symbol-based rule generation and an inference engine through a Neural Theorem Prover (NTP) according to an embodiment of the present invention.
3 is a diagram schematically illustrating the Unify module constituting the NTP according to the present embodiment.
4 is a diagram illustrating a process in which the NTP constructs a Proof Tree according to the present embodiment.
5 is a diagram comparing Neural Symbolic Unification according to the present embodiment and conventional Symbolic Unification.
6 is a diagram schematically illustrating the OR module constituting the NTP according to the present embodiment.
7 is a diagram schematically illustrating the AND module constituting the NTP according to the present embodiment.
8 is a diagram schematically illustrating completion of knowledge using an inference rule that can be obtained when learning an embedding vector according to the present embodiment.
9 is a diagram illustrating a configuration of a knowledge completion device according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

2 is a diagram illustrating the overall structure of a knowledge completion model using a neural symbol-based rule generation and an inference engine through a Neural Theorem Prover (NTP) according to an embodiment of the present invention.

Referring to FIG. 2 , the knowledge completion model according to the present embodiment is first expressed in a triple form in a knowledge base 200 (Incomplete Knowledge Base (KB)), a rule template (Rule Templete, 202), and an embedded knowledge base (Embedded KB). , 204 ), NTP 206 , and an Inference Engine 208 .

Here, the knowledge base 200 is an incomplete knowledge base that has not yet been completed.

According to this embodiment, for knowledge completion, a rule template 202 defining a form of an inference rule that can be generated from a triple of the incomplete knowledge base 200 and the knowledge base 200 is embedded in a multidimensional space based on a neural symbol. Neural Symbolic Embedding.

An embedded knowledgebase 204 is created by multidimensional spatial embedding.

The NTP 206 learns embedding through the mapping between the incomplete knowledge base 200 and the rule template 202 .

Here, learning of embedding is defined as a process of updating the embedded knowledge base 204 .

The NTP 206 is largely composed of an OR module 210 , an Unify module 212 , and an AND module 214 .

The OR module 210 and the AND module 214 are called recursively with each other to form a triple in the form of an inference rule and update the knowledge base 204 embedded in the multidimensional space.

Here, the update of the embedded knowledge base 204 is defined as updating the Predicate embedding vector value of the rule template.

New inference rules can be obtained from embeddings learned with NTP (Induced Rule).

After that, if the inference engine 208 is applied to the incomplete knowledge base 200 based on the inference rule obtained from the NTP 206, the knowledge can be completed by inferring a conclusion by finding a triple that meets the conditions of the inference rule from the knowledge base. There is (Completed Knowledge Graph).

Hereinafter, a process of performing embedding learning in NTP according to the present embodiment will be described in detail with reference to the drawings.

3 is a diagram schematically illustrating the Unify module constituting the NTP according to the present embodiment.

Referring to FIG. 3 , the Unify module 212 is an operator that matches two logical elements using Neural Symbolic Unification, and a rule head corresponding to a target triple of a logical element to be proved and a conclusion of an inference rule. (Rule Head) to match the operation is performed.

4 is a diagram illustrating a process in which the NTP constructs a Proof Tree according to the present embodiment.

와 같은 형태라고 할 때, 목표 트리플은 먼저 규칙 헤드인

와 매칭된다. 3 to 4 , for example, the target triple is locatedIn(france, europe), and the rule template is

In the form of , the target triple is first

is matched with

는 규칙 바디(Rule Body)이다. here,

is the rule body.

At this time, the Unify module 212 generates a Proof State S.

와 맵핑된 트리플들로 형성된 추론 규칙의 신뢰성 점수인 Proof Score

를 갖는 값을 Proof State S라고 한다. Subsitution having information corresponding to the entity of the triple mapped to variables such as X and Y of the rule template

Proof Score, which is the reliability score of the inference rule formed with triples mapped with

A value with is called Proof State S.

는 다차원 공간에 임베딩된 지식베이스(204)가 갖는 트리플의 Predicate, Subject, Object와 규칙 템플릿의 Predicate 그리고

에 맵핑된 엔티티 간의 L2 유사도(simliarity)의 최소값으로 정의되며, 이는 학습 시 업데이트 되어야할 파라미터인 임베딩 벡터간의 유사도임과 동시에 추론 규칙의 신뢰도 값으로 정의될 수 있다. Proof Score

is the triple of Predicate, Subject, and Object of the knowledge base 204 embedded in the multidimensional space, and the Predicate of the rule template, and

It is defined as the minimum value of L2 similarity between entities mapped to .

를 Positive Data에 대해서는 증가하고 Negative Data에 대해서는 감소하도록 이루어진다. Learning of embedding through NTP (206) is

is increased for positive data and decreased for negative data.

As described above, the Unify module 212 uses Neural Symbolic Unification.

5 is a diagram comparing Neural Symbolic Unification according to the present embodiment and conventional Symbolic Unification.

Referring to FIG. 5 , since Unify (integration) is performed based on words in data, in the conventional Symbolic Unification, Unify fails for two predicates, locatedIn and situatedIn.

와 같은 벡터로 파라미터화되기 때문에 두 벡터간의 유사도를 구해 Unify가 가능하게 된다. However, Neural Symbolic Unification according to the present embodiment does not depend on words of data, but all words are converted into vectors and inference rules are given in the form of templates, and the Predicate of these inference rules is also

Since it is parameterized with the same vector as , Unify is possible by finding the similarity between two vectors.

와 같은 벡터 형태이기 때문에 locatedIn(france, europe)에 대한 Unify 뿐만 아니라 situatedIn(france, europe)도 Unify가 가능하다. Also, the Predicate of the inference rule is

Because it is in the same vector form as , it is possible to unify not only locatedIn (france, europe) but also situatedIn (france, europe).

형태의 규칙 템플릿으로부터

와 Therefore,

from a rule template of the form

Wow

와 같은 추론 규칙도 도출할 수 있다.

Inference rules such as

6 is a diagram schematically illustrating the OR module constituting the NTP according to the present embodiment.

)와의 통합(Unify)를 수행하고 해당 규칙 바디(Rule Body,

)에 대한 통합을 수행하기 위해, 규칙 바디를 새로운 목표(New Goal)와 바디(New Body)로 나눠주는 작업을 하는 AND 모듈(214)을 재귀적으로 호출한다. OR module 210 is a target triple and a rule head (

) and the corresponding Rule Body (Rule Body,

), the AND module 214 that divides the rule body into a new goal and a new body is recursively called.

와 같을 때, 규칙 헤드는

가 되고 규칙 바디는

가 된다. For example, given the target triple is locatedIn(france, europe), the inference rule is

When equal to, the rule head is

becomes and the rule body is

becomes

와 목표 트리플인 locatedIn(france, europe)에 대한 매칭을 수행한다. OR module 210 calls Unify module 212 to

and the target triple locatedIn(france, europe) are matched.

를 새로운 목표(서브 목표)로 지정한다.In addition, the OR module 210 calls the AND module 214 to

is designated as a new target (sub-goal).

는 새로운 바디가 되어 새로운 목표에 대해 다시 OR 모듈(210)이 재귀적으로 호출된다.

becomes a new body, and the OR module 210 is called recursively for a new target again.

7 is a diagram schematically illustrating an AND module constituting an NTP according to the present embodiment.

Referring to FIG. 7 , the AND module 214 allows recursive proof of a new goal.

와 같을 때, 규칙 헤드와 목표 트리플 간의 Unify를 수행하게 되면 규칙 헤드와 목표 트리플 간의 Unify 과정에서 규칙의 X, Y와 같은 변수에 맵핑된 트리플의 엔티티(X/france, Y/europe)에 해당하는 정보를 갖는 Substitution

와 맵핑된 트리플들로 형성된 추록 규칙의 신뢰값이라고 볼 수 있는 Proof Score

를 갖는 Proof State S를 얻을 수 있다.For example, the target triple is locatedIn(france, europe), and the inference rule is

When unify between the rule head and the target triple is performed, the triple entity (X/france, Y/europe) mapped to the variables such as X and Y of the rule in the unify process between the rule head and the target triple is Substitution with information

Proof Score, which can be regarded as the confidence value of the supplementary rule formed with triples mapped with

We can obtain a Proof State S with .

에서 맵핑된 X의 엔티티(Unify 모듈에서 매칭된 엔티티)를 가져와 새로운 목표를 생성하고 새로운 목표와 지식베이스 간의 Unify를 수행하여 규칙 바디(새로운 바디)를 완성하고 새로운 바디에 대해서도 동일한 작업을 반복하여 트리 구조를 형성하게 된다. AND module 214 is

Import the mapped entity of X (matched entity in the Unify module), create a new goal, and perform Unify between the new goal and the knowledge base to complete the rule body (new body), and repeat the same operation for the new body to the tree will form a structure.

The branches of the tree generated by this process are generated as many as all possible cases from the target triple with respect to the rule template, and one inference rule that most semantically matches the target triple as a conclusion is added to the aggregation module (216). , max pooling).

는 Unify를 통해 proof tree 한 가지에서 구해진 L2 유사도 중 최소값 하나만 선택된다. 예를 들어, 트리의 각 가지에서 최소값을 위치시켜 max pooling 하였을 때 최종 Proof Score가

가 되었다면 locatedIn과

간의 유사도가 증가하도록 벡터를 업데이트하게 된다. Proof State of the selected inference rule

is selected from the minimum value of L2 similarity obtained from one proof tree through Unify. For example, when max pooling by locating the minimum value in each branch of the tree, the final Proof Score is

If it becomes locatedIn and

The vector is updated to increase the similarity between them.

8 is a diagram schematically illustrating completion of knowledge using an inference rule that can be obtained when learning an embedding vector according to the present embodiment.

는 locatedIn과 벡터가 유사하게 학습되고,

는 neighborOf와 유사하도록 학습된다. When learning is completed in the above manner,

is trained similarly to locatedIn and vectors,

is trained to be similar to neighborOf.

벡터와 유사하게 학습된 지식베이스의 predicate와

를 매칭시키면

와 같은 추론 규칙을 얻을 수 있다. each

Similar to vectors, the predicate of the learned knowledge base and

If you match

Inference rules such as

트리플이 조건에 대한 트리플로 규칙 바디를 형성하게 되고, 규칙 바디로부터 결론에 해당하는 규칙 헤드를 추론하게 되고, locatedIn(italy, europe)과 같은 새로운 트리플을 추론함으로써 지식을 완성한다. If we apply the inference rule to the knowledge base,

The triple forms the rule body as a triple for the condition, infers the rule head corresponding to the conclusion from the rule body, and completes the knowledge by inferring a new triple such as locatedIn(italy, europe).

9 is a diagram illustrating the configuration of a knowledge completion device according to an embodiment of the present invention.

Referring to FIG. 9 , the knowledge completion apparatus according to the present embodiment is an apparatus for performing the process of FIG. 1 , and may include a processor 900 and a memory 902 .

The processor 900 may include a central processing unit (CPU) or other virtual machine capable of executing a computer program.

Memory 902 may include a non-volatile storage device such as a fixed hard drive or a removable storage device. The removable storage device may include a compact flash unit, a USB memory stick, and the like. Memory 902 may also include volatile memory, such as various random access memories.

The memory 902 stores program instructions executable by the processor 900 .

The program instructions according to an embodiment of the present invention generate an embedded knowledge base by embedding triples and rule templates included in the incomplete knowledge base in a multi-dimensional space based on a neural symbol, and Neural Theorem Prover (NTP) is the embedded A new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition is obtained through embedding learning for updating the knowledge base, and the new reasoning rule is applied to the incomplete knowledge base to complete knowledge.

Here, the update of the embedded knowledge base is defined as updating the Prediacate embedding vector value of the rule template.

As described above, the NTP creates a new reasoning rule through the OR module, the Unify module, and the AND module, and completes the knowledge of the incomplete knowledge base through the newly created reasoning rule.

The above-described embodiments of the present invention have been disclosed for purposes of illustration, and various modifications, changes, and additions will be possible within the spirit and scope of the present invention by those skilled in the art having ordinary knowledge of the present invention, and such modifications, changes and additions should be regarded as belonging to the following claims.

Claims (10)

As a knowledge completion device through neural symbol-based rule creation,
processor; and
a memory coupled to the processor;
The memory is
The embedded knowledge base is created by embedding the triples and rule templates included in the unfinished knowledge base in the multi-dimensional space based on the neural symbol,
Neural Theorem Prover (NTP) acquires a new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition through embedding learning to update the embedded knowledge base,
To complete the knowledge by applying the new reasoning rule to the incomplete knowledge base,
A knowledge completion device including program instructions executable by the processor. According to claim 1,
The update of the embedded knowledge base is defined as updating a value of a Predicate embedding vector of the rule template. According to claim 1,
The NTP includes an OR module, an Unify module and an AND module,
The Unify module is called by the OR module to match a target triple of a logical element to be verified and a rule head of the rule template. 4. The method of claim 3,
The Unify module is a knowledge completion device that performs the matching process using Neural Symbolic Unification. 4. The method of claim 3,
The AND module creates a new goal using the entity matched in the Unify module, performs Unify between the created new goal and the entire knowledge base to complete a new body, and repeats the same operation for the new body to create a tree knowledge completion device that forms 4. The method of claim 3,
The NTP includes an aggregation module, and a knowledge completion device for selecting the new reasoning rule with the most semantically matching reasoning rule having the target triple as a rule head. A method of completing knowledge through neural symbol-based rule generation in a device comprising a processor and a memory, the method comprising:
(a) generating an embedded knowledge base by embedding the triples and rule templates included in the incomplete knowledge base in a multidimensional space based on a neural symbol;
(b) acquiring, by a Neural Theorem Prover (NTP), a new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition through embedding learning to update the embedded knowledge base; and
(c) completing the knowledge by applying the new reasoning rule to the incomplete knowledge base. 8. The method of claim 7,
The step (b) includes updating a Predicate embedding vector value of the rule template. A computer readable program for performing the method according to claim 7. As a knowledge completion system through neural symbol-based rule creation,
unfinished knowledge base;
an embedded knowledge base generated by embedding triples and predefined rule templates included in the incomplete knowledge base in a multidimensional space based on a neural symbol;
Neural Theorem Prover (NTP) that acquires a new reasoning rule including a rule head corresponding to a conclusion and a rule body corresponding to a condition through embedding learning to update the embedded knowledge base; and
and an inference engine for completing knowledge by applying the new reasoning rule to the incomplete knowledge base.

Images