CN110275960B - Method and system for expressing knowledge graph and text information based on named sentence - Google Patents
Method and system for expressing knowledge graph and text information based on named sentence Download PDFInfo
- Publication number
- CN110275960B CN110275960B CN201910501471.3A CN201910501471A CN110275960B CN 110275960 B CN110275960 B CN 110275960B CN 201910501471 A CN201910501471 A CN 201910501471A CN 110275960 B CN110275960 B CN 110275960B
- Authority
- CN
- China
- Prior art keywords
- vector
- entity
- textual
- relation
- knowledge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Animal Behavior & Ethology (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a method and a system for expressing knowledge graph and text information based on a named sentence, which relate to the technical field of machine learning and comprise the following steps: modeling the knowledge graph to obtain an entity vector and a relation vector; obtaining deep semantic information related to the relationship contained in the plain text, and performing knowledge modeling to obtain a textual relationship vector; acquiring deep semantic information related to an entity and contained in a plain text, and performing knowledge modeling to obtain a textual entity vector; and constructing optimization parameters based on the entity vector, the relation vector, the textual entity vector and the word vector to realize the joint representation of the knowledge graph and the text information. The invention carries out text modeling on the entity in the knowledge graph by using the 'named sentence', realizes the noise reduction of the joint representation of the knowledge graph and the text information, and thus improves the quality of the representation and deduction of the knowledge.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a method and a system for expressing knowledge graph and text information based on a named sentence.
Background
Knowledge maps (knowledgegraph) have received attention in recent years in a number of research areas because of their ability to efficiently model and describe abstract concepts and concrete examples in the real world. The knowledge graph representation learning method can effectively alleviate the problems by mapping the entities and the relations in the knowledge graph to a low-dimensional vector space, and enhances the knowledge learning capability in the aspects of knowledge representation, knowledge deduction, knowledge fusion, knowledge complementation and the like. More and more researches show that the additional text information can provide rich semantic resources for knowledge graph representation and has an important auxiliary effect on optimizing the learning of the knowledge graph representation based on the translation model. Therefore, knowledge graph and text information joint representation learning becomes a current research hotspot, and the task is to mine entities and relations from texts and map the entities and relations to the same space on the basis of representing the entities and relations in the knowledge graph as vectors in a low-dimensional space. Some models provide a joint representation learning model for mapping an entity in a knowledge graph and words in a text to the same vector space through an alignment mechanism, and other schemes model context information to a certain extent.
Disclosure of Invention
The embodiment of the invention provides a method and a system for expressing a knowledge graph and text information based on a reference sentence.
In a first aspect, an embodiment of the present invention provides a method for representing a knowledge graph and text information based on a reference sentence, where the method includes:
modeling the knowledge graph to obtain an entity vector and a relationship vector;
obtaining deep semantic information related to the relationship contained in the text information, and performing knowledge modeling based on the deep semantic information related to the relationship to obtain a textual relationship vector; obtaining deep semantic information related to the entity contained in the text information, and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector;
constructing optimization parameters based on the entity vector, the relationship vector, the textual entity vector, and the word vector, and jointly representing the knowledge-graph and the textual information based on the optimization parameters.
Optionally, the modeling the knowledge graph to obtain an entity vector and a relationship vector includes:
and adopting a translation model as a knowledge graph representation learning model, and performing knowledge graph learning through the translation model to obtain an entity vector and a relation vector.
Optionally, learning a knowledge graph through the translation model to obtain an entity vector and a relationship vector, including:
learning a knowledge graph through the translation model to obtain a head entity vector and a tail entity vector;
representing a relation vector according to the head entity vector and the tail entity vector;
constructing a first scoring function according to a first triple set consisting of the head entity vector, the tail entity vector and the relation vector;
and constructing a first loss function according to the first scoring function, and performing model training on the first triple set to obtain a trained entity vector and a trained relation vector in the knowledge graph.
Optionally, deep semantic information related to the relationship, which is included in the text information, is obtained, and knowledge modeling is performed based on the deep semantic information related to the relationship to obtain a textual relationship vector, including;
performing vector modeling on the textual relation in the text information through a convolutional neural network to obtain a first model;
taking a word vector which has a defined relation with the knowledge graph in the entity pair vector as the input of the first model, and outputting a textual relation vector through a convolutional layer and a pooling layer in a convolutional neural network;
and constructing a second loss function based on the distance between the relation vector and the textual relation vector, and performing model training on the relation vector and the textual relation vector through the second loss function to obtain a trained textual relation vector.
Optionally, the obtaining deep semantic information related to the entity included in the text information, and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector includes:
adopting a recurrent neural network to form an encoder, and generating a vector representation of a designated entity corresponding to a designated sentence through the encoder;
and selecting a specified number of the named sentences from the vector representation of the named sentences corresponding to the specified entities by using the attention model to form a textual entity vector of the specified entities.
Optionally, the obtaining deep semantic information related to the entity included in the text information, and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector, further includes:
after a textual entity vector of a specified entity is formed, a second scoring function is constructed based on a second triple set composed of the textual entity vector and a relationship vector;
and constructing a third loss function according to the second scoring function, and performing model training on the second triple set through the third loss function to obtain a trained textual entity vector in the knowledge graph.
Optionally, a recursive neural network is used to form an encoder, and the encoder generates a vector representation of the designated entity corresponding to the designated sentence, including:
and adopting a recurrent neural network with a long and short term memory unit to form an encoder, updating the hidden state vector of the recurrent neural network at each time step, and generating a vector representation of the designated entity corresponding to the designated sentence through the encoder.
Optionally, selecting a specified number of the reference sentences from the vector representation of the reference sentences corresponding to the specified entity by using the attention model includes:
calculating an attention factor by combining the structural representation of the entity and the designation sentence of the entity;
and selecting the designation sentence with the attention factor higher than the designated threshold in the designation sentence of the entity.
Optionally, constructing an optimization parameter based on the entity vector, the relationship vector, the textual entity vector, and the word vector, includes:
and constructing model parameters based on the trained entity vector, the trained relation vector, the trained textual entity vector and the word vector, and obtaining optimization parameters based on the model parameters.
In a second aspect, an embodiment of the present invention provides a system for representing a knowledge graph and text information based on a named sentence, including:
a knowledge graph representation learning module to model the knowledge graph to obtain an entity vector and a relationship vector;
the textual relation representation learning module is used for acquiring deep semantic information related to the relation contained in the text information and carrying out knowledge modeling based on the deep semantic information related to the relation to acquire a textual relation vector;
the text entity representation learning module is used for acquiring deep semantic information related to the entity contained in the text information and carrying out knowledge modeling based on the deep semantic information related to the entity to obtain a text entity vector;
and the function construction module is used for constructing optimization parameters based on the entity vectors, the relation vectors, the textual entity vectors and the word vectors, and performing joint representation on the knowledge graph and the text information based on the optimization parameters.
The embodiment of the invention carries out text modeling on the entity in the knowledge graph by using the 'named sentence', realizes the noise reduction of the joint representation of the knowledge graph and the text information, thereby improving the quality of the representation and deduction of the knowledge and obtaining the positive technical effect.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart of a method of an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a system according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The embodiment of the invention provides a method for representing a knowledge graph and text information based on a named sentence, which comprises the following steps of:
step S1, modeling the knowledge graph to obtain an entity vector and a relation vector;
in this embodiment, an entity vector h is obtained through knowledge graph learningGAnd tGAnd the relation vector represents rGWherein h isGRepresenting head entity vector tGRepresenting a tail entity vector;
step S2, obtaining deep semantic information related to the relationship contained in the text information, and performing knowledge modeling based on the deep semantic information related to the relationship to obtain a textual relationship vector rD。
Step S3, obtaining entity-related deep semantic information contained in the text information, and performing knowledge modeling based on the entity-related deep semantic information to obtain a textual entity vector hDAnd tD。
Step S4, constructing optimization parameters based on the entity vectors, the relation vectors, the textual entity vectors and the word vectors, and performing joint representation of knowledge maps and text information based on the optimization parameters.
In this embodiment, step S1, modeling the knowledge graph to obtain an entity vector and a relationship vector, includes:
and adopting a translation model as a knowledge graph representation learning model, and performing knowledge graph learning through the translation model to obtain an entity vector and a relation vector.
Wherein, learning knowledge graph through the translation model to obtain entity vector and relation vector comprises:
step S101: learning a knowledge graph through the translation model to obtain a head entity vector and a tail entity vector;
step S102: representing a relation vector according to the head entity vector and the tail entity vector;
step S103: constructing a first scoring function according to a first triple set consisting of the head entity vector, the tail entity vector and the relation vector;
step S104: and constructing a first loss function according to the first scoring function, and performing model training on the first triple set to obtain a trained entity vector and a trained relation vector in the knowledge graph.
In this embodiment, a translation model, such as a TransE model, is used as a knowledge graph representation learning model to obtain an entity vector (h) through knowledge graph learningGAnd tG) And a relation vector (r)G). For each entity pair (h, t) in the knowledge-graph G, assume its potential relationship vector as rhtDenotes from hGTo tGThe "translation" of (c):
rht＝tG-hG (1)
since each triplet (h, r, T) e T indicates that there exists an explicit relationship vector r between entity h and entity TGTherefore, the triple definition can be scoredFunction:
equation (2) shows that for any triplet (h, r, T) e T, T is expectedG≈hG+rG. Based on the scoring function above, a loss function is defined across all triplets in T:
wherein, T is a triple set in the knowledge base, T' is a triple set of negative sampling, and the construction mode is as follows:
T′＝{(h′,r,t)}∪{(h,r′,t)}∪{(h,r,t′)} (4)
h ' epsilon E represents a head entity obtained by random sampling, R ' epsilon R represents a relation obtained by random sampling, and t ' epsilon E represents a tail entity obtained by random sampling. Mu is a spacing distance parameter with a value greater than 0, [ x]+Representing a positive value function. After the model training is completed, vector representation of the entities and the relations in the knowledge graph G can be obtained.
In this embodiment, obtaining deep semantic information related to a relationship included in the text information, and performing knowledge modeling based on the deep semantic information related to the relationship to obtain a textual relationship vector, including;
step S201: performing vector modeling on the textual relation in the text information through a convolutional neural network to obtain a first model;
step S202: taking a word vector which has a defined relation with the knowledge graph in the entity pair vector as the input of the first model, and outputting a textual relation vector through a convolutional layer and a pooling layer in a convolutional neural network;
step S203: and constructing a second loss function based on the distance between the relation vector and the textual relation vector, and performing model training on the relation vector and the textual relation vector through the second loss function to obtain a trained textual relation vector.
In this embodiment, through a remote supervision policy, a sentence in which a head entity h and a tail entity t appear is subjected to vector modeling through a convolutional neural network. The invention uses a convolution neural network to model the textual relation in the plain text, and the flow is as follows: for a sentence containing an entity pair (h, t), s ═ x1,x2,…,xn) If the relation r defined by the knowledge graph G exists in the entity pair, the word vector { x of the words in the sentence s is used1,x2,…,xnTaking the vector r of the textual relation as input and outputting the vector r through the action of a convolutional layer and a pooling layer in a convolutional neural networkD. By minimising rGAnd rDAs a function of the loss of training CNN, i.e.
ψr(s)＝||rD-rG||2 (5)
Thus, the loss function defined over all sentences in the entire corpus D is
Wherein γ is a spacing distance parameter with a value greater than 0.
In this embodiment, obtaining deep semantic information related to an entity included in the text information, and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector, includes;
step S301: adopting a recurrent neural network to form an encoder, and generating a vector representation of a designated entity corresponding to a designated sentence through the encoder;
step S302: and selecting a specified number of the named sentences from the vector representation of the named sentences corresponding to the specified entities by using the attention model to form a textual entity vector of the specified entities.
Optionally, after the step of selecting a specified number of the reference sentences using the attention model and generating the textual entity vector of the given entity, the method further includes:
step S303: constructing a second scoring function according to the triples formed by the textual entity vectors and the relationship vectors;
step S304: and constructing a third loss function according to the second scoring function, and performing model training to obtain a trained textual entity vector in the knowledge graph.
A recurrent neural network with long and short term memory elements is used to construct the encoder and to generate a vector representation of the corresponding named sentence for an entity, and an attention model is used to select the first number of most informative sentences to generate a textual representation for the given entity. For an entity, the textual entity representation learning model aims to infer and model the semantic connotation of the entity from its named sentences to reduce the noise impact of text on entity representation learning. Defining a scoring function for each triplet (h, r, T) e T
σr(h,t)＝||(tD-hD)-rG||2 (7)
Wherein the head entity vector hDAnd tail entity vector tDAre text-based representations learned from named sentences. Based on the scoring function above, a loss function is defined across all triplets in T, as follows:
wherein η is an interval distance parameter whose value is greater than 0, T is a triple set in the knowledge base, and T ″ is a triple set of negative sampling, and the construction method is as follows:
T″＝{(h′,r,t)}∪{(h,r,t′)} (9)
optionally, a recursive neural network is used to form an encoder, and the encoder generates a vector representation of the designated entity corresponding to the designated sentence, including:
and adopting a recurrent neural network with a long and short term memory unit to form an encoder, updating the hidden state vector of the recurrent neural network at each time step, and generating a vector representation of the designated entity corresponding to the designated sentence through the encoder.
In this embodiment, the encoder is a named sentence encoder:
suppose that the meaning of an entity e can be referred to from it as seIs extracted out. The present embodiment uses a recurrent neural network + long-short term memory network as a sentence encoder to extract the entity meaning from the named sentence. In the textual entity representation learning model proposed in this embodiment, the recurrent neural network uses a word vector { x ] of a reference sentence1,x2,…,xnAs input (containing word and position characteristics). The recurrent neural network maintains a hidden state h over time. At each time step i, the hidden state vector hiThe update is done using the following formula:
hi＝tanh(Wxi+Uhi-1+b) (10)
the recurrent neural network reads each word representation in the input sentence one by one. The hidden state of the RNN is dynamically adjusted according to equation (10) as each word representation is read. When the sentence end mark is read, the whole sequence of the mark is read completely, so as to obtain the hidden state vector hnAs output (n represents the number of words in the sentence). Finally, the last hidden state vector hnConsidered to be a sentence-level representation. Because of hnIs the designation sentence s corresponding to the entity eeThe embodiment of the present invention also expresses the above-mentioned sentence level h for uniform expressionnIs denoted by se. On the basis, the embodiment of the invention introduces a long-short term memory network to overcome the gradient disappearance problem when learning the long-distance dependence relationship, and the long-short term memory network introduces memory cells capable of storing the state for a long time.
Optionally, the selecting a specified number of the reference sentences using the attention model comprises,
calculating an attention factor by combining the structural representation of the entity and the designation sentence of the entity;
and selecting the designation sentence with the attention factor higher than the designated threshold in the designation sentence of the entity.
The attention mechanism in this embodiment is based on a structured table of entitiesIndicating semantic similarity to its nominal sentence-level representation. Each entity e has a structural representation eG. For a sentence-level representation s belonging to an entity ee，seAnd eGThe attention factor in between is calculated as follows:
has higher attention att(s)e,eG) Is named sentence seThe entity e corresponding to the entity e can be better and more explicitly expressed, based on which, in the embodiment, the first m named sentences are selected, and then the text representation e of the entity e is obtained as followsD：
In this embodiment, constructing the optimization parameters based on the entity vector, the relationship vector, the textual entity vector, and the word vector includes:
and constructing model parameters based on the trained entity vector, the trained relation vector, the trained textual entity vector and the word vector, and obtaining optimization parameters based on the model parameters.
In this embodiment, the joint representation of entities, relationships, and words is denoted as model parameters
Wherein the content of the first and second substances,
representing learning of an entity vector representation, i.e. the relevant parameters of a structural representation of an entity,
indicating that learning an entity vector representation, i.e. the relevant parameters of the textual representation of the entity,
a correlation parameter representing the learning of a relationship vector representation, i.e. a structural representation of a relationship,
representing the learning of a relational vector representation, i.e. a relevant parameter, theta, of a textual representation of a relation from plain textVRepresenting the relevant parameters of the word vector representation in the lexicon V.
The objective of the proposed method is to obtain the optimized parameters:
wherein the content of the first and second substances,
is a loss function defined on the knowledge graph G and the corpus D, which can be decomposed into:
a second aspect of the embodiments of the present invention provides a system for representing a knowledge graph and text information based on a reference sentence, as shown in fig. 2, including:
the knowledge graph representation learning module is used for modeling the knowledge graph to obtain an entity vector and a relation vector;
in particular, a knowledge graph representation learning module is used to model the knowledge graph to obtain an entity vector (h)GAnd tG) And a relation vector (r)G)。
A textual relation representation learning module for obtaining the text informationIncluding relationship-related deep semantic information, and performing knowledge modeling based on the relationship-related deep semantic information to obtain a textual relationship vector rD；
Specifically, the textual relationship representation learning module fully mines deep semantic information about relationships embedded in plain text to refine knowledge modeling for learning textual relationships (r)D). And automatically selecting the most beneficial named sentence for the representation of the related entity from the plurality of named sentences by adopting an attention mechanism, thereby achieving the effect of relieving the adverse effect of text noise.
A textual entity representation learning module for obtaining deep semantic information related to the entity contained in the text information and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector hDAnd tD；
In particular, the textual entity representation learning module fully mines deep semantic information about the entity embodied in the plain text to refine knowledge modeling for learning the textual representation (h) of the model learning entityDAnd tD)。
And the function construction module is used for constructing optimization parameters based on the entity vector, the relation vector, the textual entity vector and the word vector, and realizing the joint representation of the knowledge graph and the text information.
The knowledge graph representation learning module, the textual relation representation learning module and the textual entity representation learning module enhance the potential connection between the knowledge graph representation learning process and the text information representation learning process by constructing a 'tight coupling' loss function.
Wherein, the text entity representation learning module comprises three parts of word representation, a sentence-to-be-referred coder and an attention model on the sentence-to-be-referred,
(1) word representation
To designate the sentence s ═ x1,x2,…,xn) Word vector of each word in { x }1,x2,…,xnAs input. In the present invention, the wordsThe representation comprises two parts of word characteristics and position characteristics: the word characteristics are learned by using a Skip-Gram model based on negative sampling to obtain word vectors, and the word vectors obtained by learning are directly regarded as the word characteristics; for the position information, given a sentence, the position feature of the entity mention it contains is labeled as 0, and the position features of other words are labeled as relative distances from the entity mention.
(2) The nominal sentence encoder:
suppose that the meaning of an entity e can be referred to from it as seIs extracted out. The invention uses recurrent neural network + long-short term memory network as sentence coder to extract entity meaning from the named sentence. In the textual entity representation learning model proposed by the present invention, the recurrent neural network uses a word vector { x ] of a reference sentence1,x2,…,xnAs input (containing word and position characteristics). The recurrent neural network maintains a hidden state h over time. At each time step i, the hidden state vector hiThe update is done using the following formula:
hi＝tanh(Wxi+Uhi-1+b) (10)
the recurrent neural network reads each word representation in the input sentence one by one. The hidden state of the RNN is dynamically adjusted according to equation (12) as each word representation is read. When the sentence end mark is read, the whole sequence of the mark is read completely, so that the hidden state vector h can be obtainednAs output (n represents the number of words in the sentence). Finally, the last hidden state vector hnConsidered to be a sentence-level representation. Because of hnIs the designation sentence s corresponding to the entity eeThe present invention also expresses the above-mentioned h in a sentence level for the purpose of unifying the expression modesnIs denoted by se. On the basis, the invention introduces a long-short term memory network to overcome the gradient disappearance problem when learning the long-distance dependence relationship, and the long-short term memory network introduces memory cells capable of storing the state for a long period of time.
(3) Attention model on the reference sentence:
the invention providesA multi-instance learning algorithm based on attention mechanism is developed to automatically select the m top informative benefit ranking index from a plurality of index for a particular entity to explicitly interpret the entity. The attention mechanism in the present invention is based on semantic similarity of the structured representation of an entity to its sentence-level representation of its nominal sentence. Each entity e has a structural representation eG. For a sentence-level representation s belonging to an entity ee，seAnd eGThe attention factor in between is calculated as follows:
has higher attention att(s)e,eG) Is named sentence seAre believed to better and more explicitly express the entity e to which it corresponds. Selecting the first m index sentences, and then obtaining the text representation e of the entity e according to the following modeD：
The invention introduces a designation sentence selection strategy based on an attention mechanism to realize text noise filtration in combined representation and reasoning research of the knowledge graph and the text information.
The method has wide application range and can be widely applied to various applications such as knowledge representation and deduction, knowledge map completion, textual relation mining and the like.
The method realizes the comprehensive utilization of multi-source heterogeneous data: and the knowledge representation learning (structured representation) and the text information representation learning (textual representation) are combined, so that the efficiency of knowledge representation and deduction is improved.
The method of the invention realizes 'double' noise reduction: enabling the named sentences from a high-quality knowledge base (such as Wikipedia) to guarantee text quality for a knowledge and text joint modeling framework; subsequently, the present invention introduces a mechanism of attention to filter low quality phrases, since low quality phrases also introduce noise, since there are high and low quality scores for phrases that are also provided by the high quality knowledge base.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (9)
1. A method for representing knowledge graph and text information based on a named sentence is characterized by comprising the following steps:
modeling the knowledge graph to obtain an entity vector and a relationship vector;
obtaining deep semantic information related to the relationship contained in the text information, and performing knowledge modeling based on the deep semantic information related to the relationship to obtain a textual relationship vector; obtaining deep semantic information related to the entity contained in the text information, and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector;
constructing optimization parameters based on the entity vectors, the relationship vectors, the textual entity vectors, and word vectors, and jointly representing knowledge-graphs and textual information based on the optimization parameters;
obtaining deep semantic information related to the relationship contained in the text information, and performing knowledge modeling based on the deep semantic information related to the relationship to obtain a textual relationship vector, including;
performing vector modeling on the textual relation in the text information through a convolutional neural network to obtain a first model;
taking a word vector which has a defined relation with the knowledge graph in the entity pair vector as the input of the first model, and outputting a textual relation vector through a convolutional layer and a pooling layer in a convolutional neural network;
and constructing a second loss function based on the distance between the relation vector and the textual relation vector, and performing model training on the relation vector and the textual relation vector through the second loss function to obtain a trained textual relation vector.
2. The method of claim 1, wherein modeling the knowledge-graph to obtain an entity vector and a relationship vector comprises:
and adopting a translation model as a knowledge graph representation learning model, and performing knowledge graph learning through the translation model to obtain an entity vector and a relation vector.
3. The method of claim 2, wherein obtaining entity vectors and relationship vectors through knowledge-graph learning by the translation model comprises:
learning a knowledge graph through the translation model to obtain a head entity vector and a tail entity vector;
representing a relation vector according to the head entity vector and the tail entity vector;
constructing a first scoring function according to a first triple set consisting of the head entity vector, the tail entity vector and the relation vector;
and constructing a first loss function according to the first scoring function, and performing model training on the first triple set to obtain a trained entity vector and a trained relation vector in the knowledge graph.
4. The method of claim 1, wherein obtaining deep semantic information related to an entity contained in the textual information and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector comprises:
adopting a recurrent neural network to form an encoder, and generating a vector representation of a designated entity corresponding to a designated sentence through the encoder;
and selecting a specified number of the named sentences from the vector representation of the named sentences corresponding to the specified entities by using the attention model to form a textual entity vector of the specified entities.
5. The method of claim 4, wherein obtaining deep semantic information related to an entity contained in the textual information and performing knowledge modeling based on the deep semantic information related to the entity to obtain a textual entity vector, further comprises:
after a textual entity vector of a specified entity is formed, a second scoring function is constructed based on a second triple set composed of the textual entity vector and a relationship vector;
and constructing a third loss function according to the second scoring function, and performing model training on the second triple set through the third loss function to obtain a trained textual entity vector in the knowledge graph.
6. The method of claim 5, wherein constructing an encoder using a recurrent neural network, generating a vector representation of the designated entity for the designated sentence by the encoder comprises:
and adopting a recurrent neural network with a long and short term memory unit to form an encoder, updating the hidden state vector of the recurrent neural network at each time step, and generating a vector representation of the designated entity corresponding to the designated sentence through the encoder.
7. The method of claim 6, wherein selecting a specified number of the reference sentences from the vector representation of reference sentences corresponding to the specified entity using the attention model comprises:
calculating an attention factor by combining the structural representation of the entity and the designation sentence of the entity;
and selecting the designation sentence with the attention factor higher than the designated threshold in the designation sentence of the entity.
8. The method of claim 7, wherein constructing optimization parameters based on the entity vector, the relationship vector, the textual entity vector, and a word vector comprises:
and constructing model parameters based on the trained entity vector, the trained relation vector, the trained textual entity vector and the word vector, and obtaining optimization parameters based on the model parameters.
9. A system for representing knowledge-graph and textual information based on a named sentence, comprising:
a knowledge graph representation learning module to model the knowledge graph to obtain an entity vector and a relationship vector;
the textual relation representation learning module is used for acquiring deep semantic information related to the relation contained in the text information and carrying out knowledge modeling based on the deep semantic information related to the relation to acquire a textual relation vector;
the text entity representation learning module is used for acquiring deep semantic information related to the entity contained in the text information and carrying out knowledge modeling based on the deep semantic information related to the entity to obtain a text entity vector;
the function construction module is used for constructing optimization parameters based on the entity vectors, the relation vectors, the textual entity vectors and the word vectors, and performing joint representation on the knowledge graph and the text information based on the optimization parameters;
obtaining deep semantic information related to the relationship contained in the text information, and performing knowledge modeling based on the deep semantic information related to the relationship to obtain a textual relationship vector, including;
performing vector modeling on the textual relation in the text information through a convolutional neural network to obtain a first model;
taking a word vector which has a defined relation with the knowledge graph in the entity pair vector as the input of the first model, and outputting a textual relation vector through a convolutional layer and a pooling layer in a convolutional neural network;
and constructing a second loss function based on the distance between the relation vector and the textual relation vector, and performing model training on the relation vector and the textual relation vector through the second loss function to obtain a trained textual relation vector.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910501471.3A CN110275960B (en) | 2019-06-11 | 2019-06-11 | Method and system for expressing knowledge graph and text information based on named sentence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910501471.3A CN110275960B (en) | 2019-06-11 | 2019-06-11 | Method and system for expressing knowledge graph and text information based on named sentence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110275960A CN110275960A (en) | 2019-09-24 |
| CN110275960B true CN110275960B (en) | 2021-09-14 |
Family
ID=67960649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910501471.3A Active CN110275960B (en) | 2019-06-11 | 2019-06-11 | Method and system for expressing knowledge graph and text information based on named sentence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110275960B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111476038B (en) * | 2020-05-06 | 2023-04-25 | 中国人民解放军国防科技大学 | Long text generation method, long text generation device, computer equipment and storage medium |
| CN112242187B (en) * | 2020-10-26 | 2023-06-27 | 平安科技（深圳）有限公司 | Medical scheme recommendation system and method based on knowledge graph characterization learning |
| CN112560477B (en) * | 2020-12-09 | 2024-04-16 | 科大讯飞(北京)有限公司 | Text completion method, electronic equipment and storage device |
| CN112818690B (en) * | 2021-01-22 | 2022-04-26 | 润联软件系统(深圳)有限公司 | Semantic recognition method and device combined with knowledge graph entity information and related equipment |
| CN113032580B (en) * | 2021-03-29 | 2023-07-25 | 浙江星汉信息技术股份有限公司 | Associated file recommendation method and system and electronic equipment |
| CN113393084B (en) * | 2021-05-13 | 2024-06-11 | 上海湃道智能科技有限公司 | Job ticket flow management system |
| CN113609311A (en) * | 2021-09-30 | 2021-11-05 | 航天宏康智能科技(北京)有限公司 | Method and device for recommending items |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106354710A (en) * | 2016-08-18 | 2017-01-25 | 清华大学 | Neural network relation extracting method |
| CN106886543A (en) * | 2015-12-16 | 2017-06-23 | 清华大学 | The knowledge mapping of binding entity description represents learning method and system |
| CN108197290A (en) * | 2018-01-19 | 2018-06-22 | 桂林电子科技大学 | A kind of knowledge mapping expression learning method for merging entity and relationship description |
| CN108563653A (en) * | 2017-12-21 | 2018-09-21 | 清华大学 | A kind of construction method and system for knowledge acquirement model in knowledge mapping |
| CN109359297A (en) * | 2018-09-20 | 2019-02-19 | 清华大学 | A kind of Relation extraction method and system |
-
2019
- 2019-06-11 CN CN201910501471.3A patent/CN110275960B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106886543A (en) * | 2015-12-16 | 2017-06-23 | 清华大学 | The knowledge mapping of binding entity description represents learning method and system |
| CN106354710A (en) * | 2016-08-18 | 2017-01-25 | 清华大学 | Neural network relation extracting method |
| CN108563653A (en) * | 2017-12-21 | 2018-09-21 | 清华大学 | A kind of construction method and system for knowledge acquirement model in knowledge mapping |
| CN108197290A (en) * | 2018-01-19 | 2018-06-22 | 桂林电子科技大学 | A kind of knowledge mapping expression learning method for merging entity and relationship description |
| CN109359297A (en) * | 2018-09-20 | 2019-02-19 | 清华大学 | A kind of Relation extraction method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110275960A (en) | 2019-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110275960B (en) | Method and system for expressing knowledge graph and text information based on named sentence | |
| CN110134968B (en) | Poem generation method, device, equipment and storage medium based on deep learning | |
| CN108510983B (en) | System and method for automatic unit selection and target decomposition for sequence annotation | |
| KR101950985B1 (en) | Systems and methods for human inspired simple question answering (hisqa) | |
| CN112528637B (en) | Text processing model training method, device, computer equipment and storage medium | |
| CN113811946A (en) | End-to-end automatic speech recognition of digital sequences | |
| CN112287670A (en) | Text error correction method, system, computer device and readable storage medium | |
| CN117113969A (en) | Machine understanding of unstructured text | |
| CN111126040B (en) | Biomedical named entity recognition method based on depth boundary combination | |
| CN109986569B (en) | Chat robot with role and personality | |
| CN115618045B (en) | Visual question answering method, device and storage medium | |
| CN108304376B (en) | Text vector determination method and device, storage medium and electronic device | |
| CN113987179A (en) | Knowledge enhancement and backtracking loss-based conversational emotion recognition network model, construction method, electronic device and storage medium | |
| CN116628171B (en) | Medical record retrieval method and system based on pre-training language model | |
| CN110019305B (en) | Knowledge base expansion method, storage medium and terminal | |
| CN114218926A (en) | Chinese spelling error correction method and system based on word segmentation and knowledge graph | |
| CN115293138A (en) | Text error correction method and computer equipment | |
| CN110297894B (en) | Intelligent dialogue generating method based on auxiliary network | |
| WO2021117180A1 (en) | Dialog processing device, learning device, dialog processing method, learning method, and program | |
| CN111428518B (en) | Low-frequency word translation method and device | |
| CN115019142A (en) | Image title generation method and system based on fusion features and electronic equipment | |
| CN117648429A (en) | Question-answering method and system based on multi-mode self-adaptive search type enhanced large model | |
| CN117708692A (en) | Entity emotion analysis method and system based on double-channel graph convolution neural network | |
| CN116821324A (en) | Model training method and device, electronic equipment and storage medium | |
| CN115905852A (en) | Story generation method, system, storage medium and terminal based on pre-training prompt |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |