CN110245238B - Graph embedding method and system based on rule reasoning and syntax mode - Google Patents

Graph embedding method and system based on rule reasoning and syntax mode Download PDF

Info

Publication number
CN110245238B
CN110245238B CN201910314357.XA CN201910314357A CN110245238B CN 110245238 B CN110245238 B CN 110245238B CN 201910314357 A CN201910314357 A CN 201910314357A CN 110245238 B CN110245238 B CN 110245238B
Authority
CN
China
Prior art keywords
node
link
short term
term memory
bilstm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910314357.XA
Other languages
Chinese (zh)
Other versions
CN110245238A (en
Inventor
贾雨葶
霍晓阳
傅洛伊
王新兵
严宇辰
田畅达
王睿杰
何小晟
朱元坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN201910314357.XA priority Critical patent/CN110245238B/en
Publication of CN110245238A publication Critical patent/CN110245238A/en
Application granted granted Critical
Publication of CN110245238B publication Critical patent/CN110245238B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

The invention provides a graph embedding method and a system based on rule reasoning and syntax schema, wherein the method comprises the following steps that 1, a conditional random field algorithm-a bidirectional long-short term memory network is used for extracting entities/values from original problems; step 2, detecting a possible structure of a query subgraph by adopting a traditional rule reasoning method based on the entity/value information and the schema of the specific field knowledge graph in the first step; step 3, replacing the entity/value in the original question by using the class label result of the first step; step 4, according to the results of the second step of structure detection and the third step of link/attribute extraction, using a traditional rule inference method to form query subgraphs for all original problems; and 5, constructing a sentence pattern schematic diagram for the original problem which can not form a connection query subgraph, using Node2Vec to learn and express the sentence pattern diagram, and simulating hidden links in the original problem by using the output of the bidirectional long-short term memory network. So that the user can quickly convert the natural language into a database query statement.

Description

Graph embedding method and system based on rule reasoning and syntax mode
Technical Field
The invention relates to the field of text network exploration type search, in particular to a graph embedding method and system based on rule reasoning and syntax mode.
Background
With the increasing popularization of the knowledge graph technology, a knowledge base question answering system (KBQA) is more and more concerned by people. The current related KBQA technology is imperfect, the answer accuracy rate is low, open type knowledge graphs are mostly constructed, and a QA system aiming at the knowledge graphs in specific academic fields is lacked. The method has low sensitivity to diversified question sentences and unsatisfactory query results, and the method comprehensively utilizes the advantages of rule reasoning and provides a method for embedding a graph to identify and construct subgraphs of the original question sentences, thereby greatly increasing the sensitivity and accuracy of the system to the question sentences.
The first stage of KBQA is problem understanding, which aims to convert natural language into a machine-interpretable form, such as λ -DCS. Firstly, a knowledge base query subgraph is defined, which can be directly mapped to lambda-DCS, and the problem understanding is converted into a problem-to-subgraph task.
Node2Vec is a framework for learning continuous feature expression for nodes in a network, and obtains feature expression by reserving network neighborhoods of the nodes to the maximum extent in a d-dimensional feature space, and a Node community is generated by second-order random walk. BilSTM refers to a bidirectional long and short term memory network, which is formed by combining a forward long and short term memory network and a backward long and short term memory network, can better capture bidirectional semantic dependence, and is often used for modeling context information in natural language processing tasks.
The prior art related to the present application is patent document CN108268580A, which discloses a question-answering method and device based on knowledge graph. The method comprises the following steps: acquiring a natural query statement input by a user, and identifying a global unique identifier GUID of an entity in the natural query statement aiming at a knowledge graph, wherein the knowledge graph comprises attributes and attribute values of the entity and relations among the entities; analyzing the natural query statement into a syntax tree according to the context-free grammar rule, and obtaining a logic expression corresponding to the natural query statement according to the syntax tree; generating a machine query statement corresponding to the knowledge graph according to the logic expression and the GUID of the entity; and inquiring the question and answer result corresponding to the machine query statement in the knowledge graph according to the machine query statement, and feeding back the question and answer result to the user.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a graph embedding method and system based on rule reasoning and syntax mode.
The invention provides a graph embedding method based on rule reasoning and syntax mode, which comprises the following steps:
step 1: problem Q from primitive using conditional random field algorithm and two-way long-short term memory networkfExtracting entity information or value information to form a class label result;
step 2: based on the entity information or the value information and the schema of the specific field knowledge graph, adopting the structure detection of the traditional rule reasoning to obtain the possible structure of the query subgraph G and forming a structure detection result;
and step 3: replacement of original problem QfExtracting link information from the original problem by using the conditional random field algorithm and the bidirectional long-short term memory network again to form a link extraction result;
and 4, step 4: using conventional rule reasoning and graph embedding of sentence patterns to original question Q based on structure detection results and link extraction resultsfForming a query subgraph G;
and 5: for a portion of the original problem Q that cannot form a query subgraphfConstructing sentence pattern graph SSG, using Node2Vec learning to represent sentence pattern graph SSG, and simulating original problem Q by using output of bidirectional long-short term memory network BilSTMfHidden link e inh。
Preferably, the conditional random field algorithm is a typical discriminant model obtained by adding an observation value on the basis of a Markov random field;
the bidirectional long and short term memory network is formed by combining a forward long and short term memory network and a backward long and short term memory network, and can capture bidirectional semantic dependence.
Preferably, the structure detection is to check the structure of each link or attribute when searching the spatial structure to see if the extracted link or attribute is connected to only one tag; if so, the link or attribute is set to the connection between the known node and the final unknown node.
Preferably, the entity information or the value information extracted from the original problem is replaced by using a class label result, so as to enhance the extraction result of the entity information or the value information.
Preferably, the step 5 comprises:
step 5.1: replacing the entity with a class type or a value type based on the entity extraction to obtain an enhancement problem;
step 5.2: converting each mark in the enhancement problem into a node, adding an edge to each adjacent node, and constructing to obtain a sentence pattern graph;
step 5.3: embedding the sentence pattern graph SSG by using the Node2Vec, and recording the distance between the embedded vectors of two connected class type or value type nodes as the embedding result of the inter-Node link, wherein the distance between the embedded vector of the unknown Node and the embedded vector connected to the unknown Node is as follows:
eh＝hunknown-hconnected
ehrefers to the embedded vector of the hidden link in the original problem, hunknownIs an unknown node, hconnectedIs a node connected to an unknown node;
step 5.4: after embedding, problem QeSequence enhancementW of sentence1、W2、···WnConversion to node N in SSG1、 N2、···NnSequence as input to BilSTM, finding hidden e by BilSTMhThe output of BilSTM is ehThe prediction embedding of (a) is expressed as:
epredict＝BiLSTM(Qe)
wherein W denotes a word, subscript n denotes a serial number, BilSTM denotes a bidirectional long-short term memory network, ehEmbedded vectors referring to hidden links in the original problem, epredictIs ehPredictive embedding of (3);
step 5.5: optimizing the parameter θ of BilSTM to make epridictAnd ehMinimum absolute value of distance (d):
θ＝argθmin||epridict-eh||
centralizing the output e of a problem Q by testingpridictExtracted problem Q entity class set vsetIs { V1， V2，··，vnN is the serial number of the node, vsetEach class type or value type node around the middle node defines a Similarity score Similarity as follows:
Similarity(epridict,eV(i)，Vj)＝-α*θ(epridict,eV(i),Vj)-β*||epridict-eV(i),Vj||
selecting V according to the calculation result of the similarity scorejAnd the corresponding link with the largest similarity score;
wherein e isV(i)Vector values representing edges connecting the i nodes around the current node and the current node; alpha and beta respectively represent optimization parameters;
step 5.6: all query subgraphs G formed at different stages are combined.
The invention provides a graph embedding system based on rule reasoning and syntax mode, which comprises:
module 1: problem Q from primitive using conditional random field algorithm and two-way long-short term memory networkfMiddle liftAcquiring entity information or value information to form a class label result;
and (3) module 2: based on the entity information or the value information and the schema of the specific field knowledge graph, adopting the structure detection of the traditional rule reasoning to obtain the possible structure of the query subgraph G and forming a structure detection result;
and a module 3: replacement of original problem QfExtracting link information from the original problem by using the conditional random field algorithm and the bidirectional long-short term memory network again to form a link extraction result;
and (4) module: using conventional rule reasoning and graph embedding of sentence patterns to original question Q based on structure detection results and link extraction resultsfForming a query subgraph G;
and a module 5: for a portion of the original problem Q that cannot form a query subgraphfConstructing a sentence pattern graph SSG, expressing the sentence pattern graph SSG by using Node2Vec learning, and simulating an original problem Q by using the output of a bidirectional long-short term memory network BilSTMfHidden link e inh。
Preferably, the conditional random field algorithm is a typical discriminant model obtained by adding an observation value on the basis of a Markov random field;
the bidirectional long and short term memory network is formed by combining a forward long and short term memory network and a backward long and short term memory network, and can capture bidirectional semantic dependence.
Preferably, the structure detection is to check the structure of each link or attribute when searching the spatial structure to see if the extracted link or attribute is connected to only one tag; if so, the link or attribute is set to the connection between the known node and the final unknown node.
Preferably, the entity information or the value information extracted from the original problem is replaced by using a class label result, so as to enhance the extraction result of the entity information or the value information.
Preferably, said module 5 comprises:
module 5.1: replacing the entity with a class type or a value type based on the entity extraction to obtain an enhancement problem;
module 5.2: converting each mark in the enhancement problem into a node, adding an edge to each adjacent node, and constructing to obtain a sentence pattern graph;
module 5.3: embedding the sentence pattern graph SSG by using the Node2Vec, and recording the distance between the embedded vectors of two connected class type or value type nodes as the embedding result of the inter-Node link, wherein the distance between the embedded vector of the unknown Node and the embedded vector connected to the unknown Node is as follows:
eh＝hunknown-hconnected
ehrefers to the embedded vector of the hidden link in the original problem, hunknownIs an unknown node, hconnectedIs a node connected to an unknown node;
module 5.4: after embedding, problem QeW of sequence by enhancement sentence1、W2、···WnConversion to node N in SSG1、 N2、···NnSequence as input to BilSTM, finding hidden e by BilSTMhThe output of BilSTM is ehThe prediction embedding of (a) is expressed as:
epredict＝BiLSTM(Qe)
wherein W denotes a word, subscript n denotes a serial number, BilSTM denotes a bidirectional long-short term memory network, ehEmbedded vectors referring to hidden links in the original problem, epredictIs ehPredictive embedding of (3);
module 5.5: optimizing the parameter θ of BilSTM to make epridictAnd ehMinimum absolute value of distance (d):
θ＝argθmin||epridict-eh||
centralizing the output e of a problem Q by testingpridictExtracted problem Q entity class set vsetIs { V1， V2，··，vnN is the serial number of the node, vsetEach class type or value type node around the middle node defines a Similarity score Similarity as follows:
Similarity(epridict,eV(i)，Vj)＝-α*θ(epridict,eV(i),Vj)-β*||epridict-eV(i),Vj||
selecting V according to the calculation result of the similarity scorejAnd the corresponding link with the largest similarity score;
wherein e isV(i)Vector values representing edges connecting the i nodes around the current node and the current node; alpha and beta respectively represent optimization parameters;
module 5.6: all query subgraphs G formed at different stages are combined.
Compared with the prior art, the invention has the following beneficial effects:
1. RI-SSGE is focused on the structural detection of problems to avoid the problem of generalization failure in other models based on various domain-specific knowledge graph templates.
2. Through web representation learning, a new sentence-schema-graph (SSG) is proposed, aiming to mimic human thinking.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 shows two slave units
Figure GDA0002115976730000051
To G*(iii) expansion of (iii);
FIG. 2 shows three types of
Figure GDA0002115976730000052
And the corresponding isomers. (ii) a
FIG. 3 is a diagram of a portion of schemas and their structural information according to the present invention;
FIG. 4 is the RI-SSGE framework.
Fig. 5 is a structure classification.
FIG. 6 is a SSG build process.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
The invention relates to the positioning and extraction of entities in natural language, the judgment of query subgraph structure based on inference rules, the conversion of statement entities to category labels and the construction of sentence pattern graphs through the graph embedding framework implementation based on rule inference and syntactic patterns; specifically, the method comprises the following steps:
step S1: problem Q from primitive using conditional random field algorithm-two-way long short term memory networkfExtracting entities/values;
step S2: detecting possible structures of the query subgraph G by adopting a traditional rule reasoning method based on entity/value information in the first step and a schema of a specific field knowledge graph, wherein QeRefer to questions, graph-SSG refer to sentence pattern graphs;
step S3: replacing the original question Q with the class-tagged result of the first stepfThe entity/value of (1);
step S4: based on the results of the second and third step structure detection and link/attribute extraction steps, RI-SSGE attempts to use traditional rule inference methods for all Q' sfG is formed. Wherein RI-SSGE refers to rule inference and sentence pattern graph embedding, QfRefers to the original problem, and G refers to the query subgraph.
Step S5: for Q that cannot form a query subgraphfA sentence pattern graph (SSG) is constructed, and the representation of the SSG is learned by using the method of Node2 Vec. Simulation of Q with the output of BilSTM by representation of SSGfHidden link e inh. Wherein QfReferring to the original problem, Node2Vec is a framework for learning continuous feature expression for nodes in a network, and feature expression is obtained by reserving the network neighborhood of the nodes in a d-dimensional feature space to the maximum extent. A community of nodes is generated using second-order random walks. BilsTM finger pairThe forward long-short term memory network and the backward long-short term memory network are combined, bidirectional semantic dependence can be captured well, and natural language processing tasks are often used for modeling context information.
The step S1 includes:
step S1.1: use of CRF-BilSTM to solve the original problem QfExtract the entity/value. The CRF-BilSTM is a typical discrimination model obtained by adding some observed values (characteristics) on the basis of a Markov random field, and has good application in the fields of word segmentation, part of speech tagging, named entity recognition and the like.
The step S2 includes:
step S2.1: for the problem that the links/attributes can be successfully extracted from in the first step, the structure of each link/attribute is checked to see if there is an extracted link/attribute connected to only one tag when searching the spatial structure. If so, this link/attribute may be the connection between the known node and the final unknown node.
Take the problem of three detection nodes as an example. The query subgraph of the problem is classified into three classes according to the class/value type labels of the three nodes and the precision of the first step
Figure GDA0002115976730000071
And determining the connection mode of the three nodes and the unknown node. Having three known label classes/values can be classified as one of three structures.
Structure 1. none of these three class labels are connected to the other two, the remaining unknown nodes must connect three known nodes to form a connected subgraph G if the rule inference method and graph (schema) can uniquely determine the class of the unknown node, G is determined at this stage.
Structure 2 there is a link between two tags, the rest of the tags are not linked to the two tags in the schema. The unknown node must connect the isolated node and one of the two connected nodes. Such QeThe next phase will be entered.
Structure 3 with two links between the three tags, the remaining nodes that are unknown can be connected to any of the three nodes. It also needs to go to the next stage.
The step S3 includes:
step S3.1: replacing the original question Q with the class-tagged result of the first stepfThe entity/value in the method is used for enhancing Q and improving CRF-BilSTM link/attribute extraction results, wherein CRF-BilSTM refers to a conditional random field algorithm-a two-way long and short term memory network, CRF is a conditional random field, is a typical discrimination model obtained by adding some observed values (characteristics) on the basis of a Markov random field, and has good application in the fields of word segmentation, part of speech tagging, named entity recognition and the like, and BilSTM refers to a two-way long and short term memory network which is formed by combining a forward long and short term memory network and a backward long and short term memory network, can well capture two-way semantic dependence, and is commonly used for modeling context information in natural language processing tasks.
The step S4 includes:
step S4.1: for the problem in the second and third steps from which the links/attributes can be successfully extracted, the structure of each link/attribute is checked to see if there is an extracted link/attribute connected to only one tag when searching the spatial structure. If so, this link/attribute may be the connection between the known node and the final unknown node.
The step S5 includes:
step S5.1: the method comprises the following steps of extracting an entity based on CRF-BilSTM, replacing the entity with a class/value type of the entity, and enhancing Q in the same way as link extraction, wherein the CRF-BilSTM refers to a conditional random field algorithm-a two-way long-short term memory network, the CRF is a conditional random field, is a typical discrimination model obtained by adding some observed values (characteristics) on the basis of a Markov random field, and has good application in the fields of word segmentation, part of speech tagging, named entity recognition and the like, the BilSTM refers to a two-way long-short term memory network, is formed by combining a forward long-short term memory network and a backward long-short term memory network, can well capture two-way semantic dependence, and is often used for modeling context information in natural language processing tasks.
Step S5.2: by mixing QeConverts each mark (word/class/value type) in (a) into a node, and adds an edge e to each adjacent node (word1, word2), (word1, class1), (value1, word1), etc. to construct a new graph-SSG; wherein Q iseRefers to questions, and graph-SSG refers to sentence pattern graphs.
Step S5.3: node2Vec is used to embed SSGs. The distance between the embedded vectors of two connected class/value type nodes is taken as the embedding result of the link between them. QeThe embedded vector of the medium hidden link is ehAnd is a label of BilSTM. From this, an unknown node (h) can be calculatedunknow1) And connecting to the unknown node (h)connecte) The distance between the embedded vectors of the nodes of (1), i.e.:
eh＝hunknow1-hconnecte
step S5.4: after embedding, QeW of sequence by enhancement sentence1、W2、···WnConversion to node N in SSG1、 N2、···NnThe sequence serves as the input to BilSTM. Finding hidden e by BilSTMh. The output of this BilSTM can be considered as ehThe prediction embedding:
epredict＝BiLSTM(Qe).
wherein QeThe problem is pointed out, W is a word, BilSTM is a bidirectional long and short term memory network, the bidirectional long and short term memory network is combined with a backward long and short term memory network, bidirectional semantic dependence can be well captured, and the bidirectional long and short term memory network is often used for modeling context information in natural language processing tasks. e.g. of the typehEmbedded vectors referring to hidden links in the original problem, epredictIs ehPredictive embedding of (3);
step S5.5: optimizing parameters θ of BilSTMepridictAnd ehMinimum absolute value of distance (d):
θ＝argθmin||epridict-eh||
concentrating the output e of one Q by testingpridictExtracted Q entity class set vsetIs { V1，V2，··，vn}. Let us be vsetEach class/value type node around the middle node defines a Similarity score Similarity:
Similarity(epridict,eV(i)，Vj)＝-α*θ(epridict,eV(i),Vj)
-β*||epridict-eV(i),Vj||.
based on the results, V is selectedjAnd the corresponding link with the largest similarity function value. The BilSM refers to a bidirectional long and short term memory network, is formed by combining a forward long and short term memory network and a backward long and short term memory network, can better capture bidirectional semantic dependence, and is often used for modeling context information in natural language processing tasks. Q refers to the problem, and alpha and beta are optimization parameters. Theta represents the optimized BilSTM final parameter; argθWhen the formula is optimal, theta is taken as an optimal solution; alpha and beta respectively represent hyper-parameters and weight values; | | epredict-eh| | represents the distance between the link vector predicted by the model and the actual link vector; e.g. of the typepredictA link vector value representing a model prediction; e.g. of the typehRepresenting the actual exact link vector value; e.g. of the typev(i)Vector values representing edges connecting the i nodes around the current node and the current node; vjRepresents the current node j and the set of nodes around the current node j; | | epredict-eV(i),VjAnd | | represents the distance between the predicted link vector value at the j node and the vector value of the edge of the i node around the j node.
Step S5.6: all G s formed at different stages are combined, where G refers to the query subgraph.
In a specific implementation, as shown in fig. 1, the question is which articles are published by the Pascal professor of michigan university at the top-level conference AAAI2017, and the question is represented in a subgraph manner, where (a) Operation nodes represent Operation operations on unknown nodes, in this question, the total number of the articles is counted, and (b) Multi-unknown nodes represent additional addition of the unknown nodes, in this question, collaborators of the Pascal professor are found in the unknown articles.
The relationship between the query diagram and the chemical structure is shown in FIG. 2, which shows three isomers of pentane, N-pentane: n-pentane, Iso-pentane: isopentane, Neo-pentane: neopentane.
Fig. 3 shows an abstracted skeleton diagram of the knowledge-graph, Partofschema, which shows that the skeleton diagram only shows a partial structure of the entire knowledge-graph: Paper-Author-Field, the bottom half of knowledge base is shown for an exact data example.
The algorithm framework flow chart of the whole patent application as shown in fig. 4 shows the whole process from the original natural language question to the conversion into a so-called question sub-graph.
As shown in FIG. 5, which shows a possible Case of connecting edges under three nodes, Case1, Case 2, Case 3 respectively represent the Case of three nodes being separated, two nodes being connected and three nodes being connected.
As shown in FIG. 6, the process of constructing the SSG graph with the maximum innovation point in the present invention is to predict the key link of author _ is _ in _ Field by using the neural network model to construct an accurate question sub-graph from the non-canonical original natural language input by the user, such as Whereisauuthorin Field (which domain the author studies).
The invention focuses mainly on the actual problem to the sub-graph task (Qf, G) and proves that more complex problems can be easily solved on its basis. A framework based on rule-based reasoning and syntactic pattern embedding (RI-SSGE) is proposed to solve the (Qf, G) task. Inspired by the isomeric structure in chemistry, we focused RI-SSGE on the structural detection of problems to avoid the problem of poor generalization in other models based on various domain-specific knowledge map templates. In order to solve the problem of error propagation, the RI-SSGE creatively combines the traditional rule reasoning method and the graphical representation method, thereby ensuring the performance of the whole framework. After observing that a person can mine hidden relations by combining questions and knowledge graph structures, we propose a new sentence-schema-graph (SSG) in the last network representation learning phase of RI-SSGE, aiming to mimic the way a person thinks. In addition to the RISSGE, a new dataset named AceQG was introduced, which has 133,143 (factual problem, subgraph) pairs on an open academic knowledge graph named AceKG. Experiments on the AceQG and Geoquery-880 data sets show that RI-SSGE is superior to other algorithms.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (8)

1. A graph embedding method based on rule reasoning and syntax schema is characterized by comprising the following steps:
step 1: problem Q from primitive using conditional random field algorithm and two-way long-short term memory networkfExtracting entity information or value information to form a class label result;
step 2: based on the entity information or the value information and the schema of the specific field knowledge graph, adopting the structure detection of the traditional rule reasoning to obtain the possible structure of the query subgraph G and forming a structure detection result;
and step 3: replacement of original problem QfExtracting link information from the original problem by using the conditional random field algorithm and the bidirectional long-short term memory network again to form a link extraction result;
and 4, step 4: using conventional rule reasoning and graph embedding of sentence patterns to original question Q based on structure detection results and link extraction resultsfForming a query subgraph G;
and 5: for a portion of the original problem Q that cannot form a query subgraphfConstructing sentence pattern graph SSG, and learning by using Node2VecThe sentence pattern graph SSG is expressed, and the output of the bidirectional long-short term memory network BilSTM is used for simulating the original problem QfA hidden link in;
the step 5 comprises the following steps:
step 5.1: replacing the entity with a class type or a value type based on the entity extraction to obtain an enhancement problem;
step 5.2: converting each mark in the enhancement problem into a node, adding an edge to each adjacent node, and constructing to obtain a sentence pattern graph;
step 5.3: embedding the sentence pattern graph SSG by using the Node2Vec, recording the distance between the embedded vectors of two connected class type or value type nodes as the embedded result of the link between the nodes,
eh＝hunknown-hconnected
ehrefers to the embedded vector of the hidden link in the original problem, hunknownIs an unknown node, hconnectedIs a node connected to an unknown node;
step 5.4: after embedding, problem QeW of sequence by enhancement sentence1、W2、···WnConversion to node N in SSG1、N2、···NnSequence as input to BilSTM, finding hidden e by BilSTMhThe output of BilSTM is ehThe prediction embedding of (a) is expressed as:
epredict＝BiLSTM(Qe)
wherein W denotes a word, subscript n denotes a serial number, BilSTM denotes a bidirectional long-short term memory network, ehEmbedded vectors referring to hidden links in the original problem, epredictIs ehPredictive embedding of (3);
step 5.5: optimizing the parameter θ of BilSTM to make epridictAnd ehMinimum absolute value of distance (d):
Figure FDA0003050164450000021
centralizing a problem through testingOutput e of QpridictExtracted problem Q entity class set vsetIs { V1，V2，··，vnN is the serial number of the node, vsetEach class type or value type node around the middle node defines a Similarity score Similarity as follows:
Similarity(epridict,eV(i)，Vj)＝-α*θ(epridict,eV(i),Vj)-β*||epridict-eV(i),Vj||
selecting V according to the calculation result of the similarity scorejAnd the corresponding link with the largest similarity score;
wherein e isV(i)Vector values representing edges connecting the i nodes around the current node and the current node; alpha and beta respectively represent optimization parameters;
step 5.6: all query subgraphs G formed at different stages are combined.
2. The method of claim 1 in which the conditional random field algorithm is a canonical discriminant model based on markov random fields plus observations;
the bidirectional long and short term memory network is formed by combining a forward long and short term memory network and a backward long and short term memory network, and can capture bidirectional semantic dependence.
3. The graph embedding method based on rule-based reasoning and syntax schema as claimed in claim 1, wherein the structure detection is checking the structure of each link or attribute in searching the spatial structure to see if the extracted link or attribute is connected to only one tag; if so, the link or attribute is set to the connection between the known node and the final unknown node.
4. The rule-based reasoning and syntactic pattern embedding method of claim 1, wherein said replacing the extracted entity information or value information in the original question is performed using class label result to enhance the extraction result of the entity information or value information.
5. A graph embedding system based on rule-based reasoning and syntactic patterns, comprising:
module 1: problem Q from primitive using conditional random field algorithm and two-way long-short term memory networkfExtracting entity information or value information to form a class label result;
and (3) module 2: based on the entity information or the value information and the schema of the specific field knowledge graph, adopting the structure detection of the traditional rule reasoning to obtain the possible structure of the query subgraph G and forming a structure detection result;
and a module 3: replacement of original problem QfExtracting link information from the original problem by using the conditional random field algorithm and the bidirectional long-short term memory network again to form a link extraction result;
and (4) module: using conventional rule reasoning and graph embedding of sentence patterns to original question Q based on structure detection results and link extraction resultsfForming a query subgraph G;
and a module 5: for a portion of the original problem Q that cannot form a query subgraphfConstructing a sentence pattern graph SSG, expressing the sentence pattern graph SSG by using Node2Vec learning, and simulating an original problem Q by using the output of a bidirectional long-short term memory network BilSTMfA hidden link in;
the module 5 comprises:
module 5.1: replacing the entity with a class type or a value type based on the entity extraction to obtain an enhancement problem;
module 5.2: converting each mark in the enhancement problem into a node, adding an edge to each adjacent node, and constructing to obtain a sentence pattern graph;
module 5.3: embedding the sentence pattern graph SSG by using the Node2Vec, recording the distance between the embedded vectors of two connected class type or value type nodes as the embedded result of the link between the nodes,
eh＝hunknown-hconnected
ehrefers to the embedded vector of the hidden link in the original problem, hunknownIs an unknown node, hconnectedIs a node connected to an unknown node;
module 5.4: after embedding, problem QeW of sequence by enhancement sentence1、W2、···WnConversion to node N in SSG1、N2、···NnSequence as input to BilSTM, finding hidden e by BilSTMhThe output of BilSTM is ehThe prediction embedding of (a) is expressed as:
epredict＝BiLSTM(Qe)
wherein W denotes a word, subscript n denotes a serial number, BilSTM denotes a bidirectional long-short term memory network, ehEmbedded vectors referring to hidden links in the original problem, epredictIs ehPredictive embedding of (3);
module 5.5: optimizing the parameter θ of BilSTM to make epridictAnd ehMinimum absolute value of distance (d):
Figure FDA0003050164450000031
centralizing the output e of a problem Q by testingpridictExtracted problem Q entity class set vsetIs { V1，V2，··，vnN is the serial number of the node, vsetEach class type or value type node around the middle node defines a Similarity score Similarity as follows:
Similarity(epridict,eV(i)，Vj)＝-α*θ(epridict,eV(i),Vj)-β*||epridict-eV(i),Vj||
selecting V according to the calculation result of the similarity scorejAnd the corresponding link with the largest similarity score;
wherein e isV(i)Vector values representing edges connecting the i nodes around the current node and the current node; alpha, beta, fractionRespectively representing optimization parameters;
module 5.6: all query subgraphs G formed at different stages are combined.
6. The system for graph embedding based on regular inference and syntactic patterns according to claim 5, wherein said conditional random field algorithm is a classical discriminant model based on markov random fields with observations;
the bidirectional long and short term memory network is formed by combining a forward long and short term memory network and a backward long and short term memory network, and can capture bidirectional semantic dependence.
7. The graph embedding system based on rule-based reasoning and syntax schema of claim 5, wherein the structure detection is checking the structure of each link or attribute in a search space structure to see if the extracted link or attribute is connected to only one tag; if so, the link or attribute is set to the connection between the known node and the final unknown node.
8. The rule-based reasoning and syntactic pattern embedding system of claim 5, wherein said replacing the extracted entity information or value information in the original question is performed using class label results to enhance the extraction of the entity information or value information.
CN201910314357.XA 2019-04-18 2019-04-18 Graph embedding method and system based on rule reasoning and syntax mode Active CN110245238B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910314357.XA CN110245238B (en) 2019-04-18 2019-04-18 Graph embedding method and system based on rule reasoning and syntax mode

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910314357.XA CN110245238B (en) 2019-04-18 2019-04-18 Graph embedding method and system based on rule reasoning and syntax mode

Publications (2)

Publication Number Publication Date
CN110245238A CN110245238A (en) 2019-09-17
CN110245238B true CN110245238B (en) 2021-08-17

Family

ID=67883192

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910314357.XA Active CN110245238B (en) 2019-04-18 2019-04-18 Graph embedding method and system based on rule reasoning and syntax mode

Country Status (1)

Country Link
CN (1) CN110245238B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111046667B (en) * 2019-11-14 2024-02-06 深圳市优必选科技股份有限公司 Statement identification method, statement identification device and intelligent equipment
CN110866190B (en) * 2019-11-18 2021-05-14 支付宝(杭州)信息技术有限公司 Method and device for training neural network model for representing knowledge graph
CN111444348A (en) * 2020-02-27 2020-07-24 上海交通大学 Method, system and medium for constructing and applying knowledge graph architecture
CN111651613B (en) * 2020-07-08 2021-07-27 海南大学 Knowledge graph embedding-based dynamic recommendation method and system
CN112015868B (en) * 2020-09-07 2022-08-26 重庆邮电大学 Question-answering method based on knowledge graph completion
CN113255346B (en) * 2021-07-01 2021-09-14 湖南工商大学 Address element identification method based on graph embedding and CRF knowledge integration
WO2024073338A1 (en) * 2022-09-29 2024-04-04 Visa International Service Association Entity linking using subgraph matching

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105243064A (en) * 2014-06-23 2016-01-13 华为技术有限公司 Subgraph matching method and device
CN107330100A (en) * 2017-07-06 2017-11-07 北京大学深圳研究生院 Combine the two-way search method of image text of embedded space based on multi views
CN107491500A (en) * 2017-07-28 2017-12-19 中国人民大学 A kind of knowledge base complementing method of strong adaptability
CN107704637A (en) * 2017-11-20 2018-02-16 中国人民解放军国防科技大学 Knowledge graph construction method for emergency
CN108052547A (en) * 2017-11-27 2018-05-18 华中科技大学 Natural language question-answering method and system based on question sentence and knowledge graph structural analysis
CN108229582A (en) * 2018-02-01 2018-06-29 浙江大学 Entity recognition dual training method is named in a kind of multitask towards medical domain
CN108694469A (en) * 2018-06-08 2018-10-23 哈尔滨工程大学 A kind of Relationship Prediction method of knowledge based collection of illustrative plates
US10157226B1 (en) * 2018-01-16 2018-12-18 Accenture Global Solutions Limited Predicting links in knowledge graphs using ontological knowledge
CN109086356A (en) * 2018-07-18 2018-12-25 哈尔滨工业大学 The incorrect link relationship diagnosis of extensive knowledge mapping and modification method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101306667B1 (en) * 2009-12-09 2013-09-10 한국전자통신연구원 Apparatus and method for knowledge graph stabilization
KR20140145018A (en) * 2013-06-12 2014-12-22 한국전자통신연구원 Knowledge index system and method thereof
US10984045B2 (en) * 2017-05-24 2021-04-20 International Business Machines Corporation Neural bit embeddings for graphs

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105243064A (en) * 2014-06-23 2016-01-13 华为技术有限公司 Subgraph matching method and device
CN107330100A (en) * 2017-07-06 2017-11-07 北京大学深圳研究生院 Combine the two-way search method of image text of embedded space based on multi views
CN107491500A (en) * 2017-07-28 2017-12-19 中国人民大学 A kind of knowledge base complementing method of strong adaptability
CN107704637A (en) * 2017-11-20 2018-02-16 中国人民解放军国防科技大学 Knowledge graph construction method for emergency
CN108052547A (en) * 2017-11-27 2018-05-18 华中科技大学 Natural language question-answering method and system based on question sentence and knowledge graph structural analysis
US10157226B1 (en) * 2018-01-16 2018-12-18 Accenture Global Solutions Limited Predicting links in knowledge graphs using ontological knowledge
CN108229582A (en) * 2018-02-01 2018-06-29 浙江大学 Entity recognition dual training method is named in a kind of multitask towards medical domain
CN108694469A (en) * 2018-06-08 2018-10-23 哈尔滨工程大学 A kind of Relationship Prediction method of knowledge based collection of illustrative plates
CN109086356A (en) * 2018-07-18 2018-12-25 哈尔滨工业大学 The incorrect link relationship diagnosis of extensive knowledge mapping and modification method

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Probabilistic Latent Document Network Embedding;Tuan M.V.Le 等;《2014IEEE International Conference on Data Mining》;20141217;第270-279页 *
基于BIM的智能化辅助设计平台技术研究;李智杰;《中国博士学位论文全文数据库 信息科技辑》;20180715(第07(2018)期);第I138-81页 *
基于编辑距离图嵌入的图匹配算法研究;刘永强;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160115(第01(2016)期);第I138-919页 *
基于递归投影的结构性网络嵌入;孟亚文 等;《计算机应用研究》;20190129;第37卷(第3期);第659-661页 *
网络空间大搜索研究范畴与发展趋势;方滨兴 等;《通信学报》;20151225;第36卷(第12期);第1-8页 *

Also Published As

Publication number Publication date
CN110245238A (en) 2019-09-17

Similar Documents

Publication Publication Date Title
CN110245238B (en) Graph embedding method and system based on rule reasoning and syntax mode
CN110825881B (en) Method for establishing electric power knowledge graph
CN111291185B (en) Information extraction method, device, electronic equipment and storage medium
CN111581395B (en) Model fusion triplet representation learning system and method based on deep learning
US11093835B2 (en) Natural language question expansion and extraction
CN112200317A (en) Multi-modal knowledge graph construction method
CN111488734A (en) Emotional feature representation learning system and method based on global interaction and syntactic dependency
CN111931506B (en) Entity relationship extraction method based on graph information enhancement
CN110990590A (en) Dynamic financial knowledge map construction method based on reinforcement learning and transfer learning
CN111339313A (en) Knowledge base construction method based on multi-mode fusion
Nagy et al. Multiagent ontology mapping framework for the semantic web
CN111339269B (en) Knowledge graph question-answering training and application service system capable of automatically generating templates
Athreya et al. Template-based question answering using recursive neural networks
CN114064918A (en) Multi-modal event knowledge graph construction method
CN107679221A (en) Towards the time-space data acquisition and Services Composition scheme generation method of mitigation task
CN114996423A (en) Water conservancy knowledge graph complex question-answering method based on subproblem pruning
CN116484024A (en) Multi-level knowledge base construction method based on knowledge graph
Chandiok et al. CIT: Integrated cognitive computing and cognitive agent technologies based cognitive architecture for human-like functionality in artificial systems
CN112632223B (en) Case and event knowledge graph construction method and related equipment
CN112417170B (en) Relationship linking method for incomplete knowledge graph
CN113901224A (en) Knowledge distillation-based secret-related text recognition model training method, system and device
Bendaoud et al. Pactole: A methodology and a system for semi-automatically enriching an ontology from a collection of texts
Günther et al. Retro: Relation retrofitting for in-database machine learning on textual data
CN117149974A (en) Knowledge graph question-answering method for sub-graph retrieval optimization
CN115658845A (en) Intelligent question-answering method and device suitable for open-source software supply chain

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant