CN117648984A

CN117648984A - Intelligent question-answering method and system based on domain knowledge graph

Info

Publication number: CN117648984A
Application number: CN202311435968.2A
Authority: CN
Inventors: 刘琼昕; 方胜; 高雅芳; 牛文涛; 徐雅馨; 陈国梁
Original assignee: Beijing Institute of Technology BIT
Current assignee: Beijing Institute of Technology BIT
Priority date: 2023-10-31
Filing date: 2023-10-31
Publication date: 2024-03-05

Abstract

The invention relates to an intelligent question-answering method and system based on a domain knowledge graph, and belongs to the technical field of computer natural language processing. The method comprises the steps of firstly constructing a basic database, inputting a user question, processing the question by using a rule-based template matching method, and then processing the question. The question-answering system constructs a corresponding Cypher query sentence based on question answer information, performs answer query in a database, further processes answers, and returns the answers according to the question type. The system comprises a template module and a depth module. The invention provides good support for the question-answer model training based on the knowledge graph, improves the accuracy of the subtasks while saving the subtask time, eliminates the influence of the knowledge in the professional field on the relation recognition, strengthens the bidirectional communication and modeling between the knowledge base and the question sentence, improves the accuracy and the response speed of the system, reduces the maintenance and updating cost, ensures the accuracy of the question-answer model and ensures the interpretability of the model.

Description

Intelligent question-answering method and system based on domain knowledge graph

Technical Field

The invention relates to an intelligent question-answering method and system based on a domain knowledge graph, and belongs to the technical field of computer natural language processing.

Background

Since google search engine in 2012 introduced large-scale knowledge graph, knowledge graph has been widely used in various fields (such as medical treatment, finance, etc.). Knowledge maps can be divided into general fields and vertical fields according to the problem fields of application. The universal domain knowledge graph covers a wide range of fields, has a large scale and has a high degree of automation. In contrast, the vertical domain knowledge graph is constructed based on data of a specific industry, and has relatively small scale, but high knowledge quality and high accuracy. Compared with other fields, the construction of the knowledge graph and the data set in the non-public field in the vertical field is a very challenging task, and the information acquisition in the field has the characteristics of information fragmentation, high acquisition difficulty, complex relationship among entities and the like. In addition, the knowledge graph and the data set construction efficiency of the non-public domain are also lower. Therefore, the construction of the knowledge graph and the data set in the vertical field is more challenging, and intensive research is required to be conducted aiming at the characteristics of the knowledge graph and the data set.

In the aspect of knowledge graph construction in specific non-public fields, the existing research results comprise: li Hua et al train an entity-relationship joint extraction model based on the LEBERT model and relationship extraction rules, and finally realize a multi-modal computer science field knowledge graph capable of automatically extracting relationship triples. Deng Kai et al propose a domain knowledge graph construction method based on semi-structured data capable of being quickly migrated, which mainly comprises 1) dictionary construction; 2) Data acquisition and cleaning; 3) Entity maintenance and linking; 4) Map updating and visualization. The domain knowledge graph is constructed by means of acquiring the entity and the relation by a plurality of trusted data sources, so that the labor consumption can be effectively reduced, and the construction of the knowledge graph is quickened. Meng Xiaofeng based on knowledge graph, constructing essential features of semantic relation, taking triples as modeling granularity, fully fitting interaction among head entities, relations and tail entities, and providing an InterTris model. Because of the data specificity, the research results in the field are as follows: chen et al provides a field knowledge graph construction technology based on Internet open source and multiple data aiming at the problems of data isolation, associated organization deletion, difficult effective utilization of data and the like in an informatization process. Liu Chenguang et al propose a method for extracting entity relationship based on conditional random field (Conditional Random Field, CRF) and syntactic analysis tree, and optimize the construction of knowledge graph through massive data training, model comparison and improvement. In the face of massive data information, the effective knowledge extraction mode can greatly improve the construction efficiency of the knowledge graph, for example Hou Zhenyu et al use a BERT-CRF-PRF model for knowledge extraction, wherein the BERT mainly comprises an embedded layer, a transducer encoder and a loss optimization 3 part. The CRF model can represent the joint probability of the whole feature sequence through the adjacent position label relation, and the optimal prediction result of the whole sequence can be obtained.

Compared with general knowledge graph questions and answers, based on questions and answers of non-public domain knowledge graphs, the questions and answers are generally understood to require professional domain knowledge, and more professional information needs to be acquired to supplement. Therefore, the main difficulty of knowledge-graph-based questions and answers is in the construction of the knowledge graph and how to convert the user's intention into a structured sentence that can be queried in the knowledge graph. In the knowledge graph question-answering task based on the non-public field, the research results are as follows: yili Tai et al apply intelligent recognition technologies such as deep learning to the medical question-answering system, solve the information redundancy of the traditional medical question-answering and the limitation of low question-answering efficiency. And (3) performing knowledge extraction by using a joint learning model based on a bidirectional transducer, performing word segmentation recognition on a medical input question based on an intention recognition and slot filling algorithm of a Stack-generation framework, and constructing a medical knowledge graph by using a Neo4j graph database to realize question-answer retrieval. Zhang Keliang et al's question and answer system uses the domain ontology to classify questions and uses a structured semantic information extraction method to convert natural language questions into SPARQL query statements, and then searches the ontology knowledge base for answers to the questions. Based on the questions and answers of the knowledge graph in the non-public domain, the research results at present are as follows: dou Xiaojiang et al, which is composed of three modules of question understanding, question solving and answer generation, uses a naive bayes classifier to classify the questions and then combines the knowledge graph to answer the questions. Cheng Jie et al construct a Support Vector Machine (SVM) multi-classifier to classify questions and complete named entity recognition tasks using a bi-directional long-short-term memory network (BiLSTM) and conditional random fields to understand the entities, attributes and relationships in the questions. The user questions are then matched to the most similar question templates. And finally, inquiring answers from the constructed knowledge graph by using the generated graph database inquiry statement cytoer.

The question and answer based on the knowledge graph is an important subtask in dialogue generation, and the combination of the knowledge graph and the intelligent question and answer can improve the accuracy and efficiency of a question and answer system and provide better service and experience for users. Early knowledge-graph-based questions and answers mostly focus on simple questions comprising only one relationship, but real questions and sentences are mostly complex questions and sentences comprising multi-hop relationships and constraint limits, so that more attention is given to recent multi-hop questions and answers. One of the most prominent challenges of knowledge-based questions and answers is lexical gap, i.e., the same question can be expressed in different ways in natural language, while knowledge-based questions and answers are a canonical dictionary, how to learn a proper model to map a natural language question to a structured knowledge base, so that finding the correct answer is the key point of knowledge-based multi-hop questions and answers research. The first question answering system designed by Green and the like is introduced before and after the 60 s of the 20 th century, and the designed Baseball program can answer the questions related to the Baseball game by using common English. With the advent of machine learning and deep learning, the research direction of the question-answering system is also separated from the templatization and regularization, and the research direction is developed towards the large-scale data set and the intellectualization, and the research field is also extended from a fixed corpus to the whole internet.

At present, a method for solving a question-answer model based on a knowledge graph mainly comprises the following steps:

1. query graph generation model: in the field of question-answering of knowledge maps, a common model strategy is to generate a corresponding query map according to a specific problem, then convert the query map into a structured sentence, and query in the knowledge map. However, as the complexity of the problem grows, the search space of the query graph expands dramatically. To effectively address this challenge, researchers such as Chen and teams such as Lan have adopted a streamlined strategy in generating query graphs. Specifically, it refers to a bundle search method that considers the best relationships only when the query graph path is extended, not all possible relationships.

2. Model based on graph neural network: in recent years, graphic neural network models such as GNN-QA and GraftNet have been widely used in knowledge graph questions and answers. The models efficiently represent the knowledge graph through the graph neural network, so that complex problems can be processed, and the multi-jump question-answering performance of the knowledge graph is obviously optimized.

3. In the field of natural language processing, pre-training models such as BERT, roBERTa, GPT have achieved remarkable results. Recently, researchers have begun exploring the use of these models in knowledge-based question-answering systems, such as K-BERT, and the like. By learning a large amount of knowledge graph data, the models can further improve the accuracy and efficiency of questions and answers.

4. A cross-modality based model: in order to more efficiently process and fuse multimodal information such as text, images, etc., researchers have proposed cross-modal models such as KVMN, MNLM, etc. The models not only can be used for fusing input information in different forms, but also can realize deep semantic understanding and more accurate information extraction under a multi-mode background. This approach opens new possibilities for complex data processing and analysis tasks.

However, the current knowledge-graph-based question-answering model mainly depends on a fixed sentence pattern, and deep learning modeling is not performed on a question sentence, so that generalization capability of the model is affected. The question-answering forms in specific non-public fields are complex and various, so that the existing method is difficult to meet the variable and flexible question-asking requirements.

The classification-based knowledge graph question-answering method over emphasizes the accuracy of the model, but the interpretation of the model is insufficient. This means that although the answer may be correct, the user may still lack confidence in it because the model does not provide transparency of its decision making process. It is critical to provide a clear, interpretable basis for this decision, so future research should emphasize the transparency and interpretability of the model decision while pursuing accuracy.

Although the rule-based template matching method can be well applied to the professional or limited field, large-scale data training is not needed, and quick result return can be realized, but the rule-based template matching method is difficult to be qualified for non-standardized input and variable natural language expression, and the maintenance and updating of templates become time-consuming and resource-intensive along with the expansion and updating of a knowledge base.

Although depth models can better address complex, multi-level open questions through training, training and running depth models often requires expensive computational resources and may sometimes provide inaccurate or irrelevant answers.

Disclosure of Invention

Aiming at the problems and the defects existing in the prior art, the invention creatively provides an intelligent question-answering method and system based on a domain knowledge graph for effectively solving the technical problems of flexibility, interpretability and the like of question-answering of the knowledge graph.

The invention constructs a data set and a knowledge graph according to data in a non-public data table, comprising the following steps:

1. generating a corresponding triplet, and generating a non-public domain data set containing various random semantics by adopting a manual labeling form;

2. the triples are combined to form a knowledge graph and displayed in a database (such as Neo4j database).

On one hand, the invention provides an intelligent question-answering method based on a domain knowledge graph, which comprises the following steps:

step 1: constructing a basic database, inputting a user question, and processing the question by using a rule-based template matching method.

Specifically, step 1 includes the steps of:

step 1.1: and extracting entity library, event library, task library, relation library and attribute library information from the database, and constructing a basic database (such as a Jieba custom word library) required by question and answer.

Based on the question library, constructing a corresponding question template knowledge base for possible question types.

Based on the field database, constructing a corresponding knowledge graph, wherein the knowledge graph comprises entity nodes, entity attributes, neighbor nodes and relations between the entity nodes and the neighbor nodes.

Step 1.2: and for question input of a user, processing the question by using a rule-based question template matching method.

For example, using the precise mode of Jieba segmentation, the input question is precisely segmented, and simple entity recognition and part-of-speech tagging are performed.

Step 1.3: and removing the cut stop words and irrelevant words, calculating the minimum edit distance between the cut sentences and the entity list by using a minimum edit distance algorithm (such as a Levenshtein algorithm), and sequencing the entity list with the obtained similarity to obtain the entity information of the question. And then, constructing a synonym forest for the relation and the attribute information to obtain the relation attribute information in the question.

Step 1.4: and dividing the question into a plurality of question query type templates by using the extracted question entity, attribute and relation information, and matching the question into the predefined question templates.

The question query type template may specifically include 15 types of single-entity single-attribute query questions, single-entity multi-attribute query questions, whether single-entity single-attribute query questions are similar, one-jump-tail entity query questions, relation query questions, two-jump-question attribute query questions, two-jump-question statistical query questions, two-entity single-attribute comparison query questions, single-interval query questions, multi-area query questions, multi-entity single-attribute query questions, attribute maximum query questions, tail-entity confirmation query questions, maximum query questions, statistical query questions, and the like.

Step 2: the questions were processed using the BAMNet network, textCNN, BERT.

In step 1, because of the strong constraint of the rule-based problem template matching method on the question, the generalization capability is poor, and it is difficult to capture all semantic information with fixed rules. For this type of problem that the templates cannot be matched, the questions need to be processed using the BAMNet network, textCNN, BERT.

Specifically, step 2 comprises the following sub-steps:

step 2.1: inputting a question into a textCNN model, capturing semantic information of the sentence by using a convolutional neural network, and obtaining type information of the question, wherein L2 regularization and dropout are used in the training process to prevent overfitting.

Step 2.2: and removing the rest sentences of the entity from the question, and calculating the similarity of the relation between the question and the relation library by using the BERT model to obtain the relation information of the question.

Step 2.3: the question is entered into the BAMNet network.

For a given question, it is encoded into an intermediate hidden state vector representation using a bi-directional LSTM network;

for three kinds of information, namely the answer entity type, the answer entity context representation and the answer entity path, of the candidate answers in the knowledge graph, respectively encoding the three kinds of information into vector representation by using a bidirectional LSTM network;

storing candidate answer code information using a key value memory network; using an inference module formed by a bidirectional attention mechanism network and a generalization module to realize bidirectional interaction between question vector identification and candidate answer coding information;

and calculating similarity scores for the question vector representations and the candidate answer representations, and selecting the candidate answer entity with the highest score as the question answer. Specifically, the generated question answers include key information composed of question types, question entities, question attributes, question relationships and the like.

Step 2.4: acquiring the type information of the question based on the question type information acquired by the textCNN model and the BAMnet network, and acquiring the final type of the question by using a softmax function; based on the question relation information obtained by the BERT model in the step 2.2 and the question information obtained by the BAMNet, obtaining a final question relation by using a softmax function, and obtaining the question attribute by the BAMNet.

Step 3: the question-answering system constructs a corresponding query sentence (such as a Cypher sentence) based on the question answer information in the step 1 and the step 2, performs answer query in a database, further processes the answer, and returns the answer according to the question type.

Specifically, step 3 includes the steps of:

step 3.1: in the rule-based question template matching method, if a question can be perfectly matched with a template library, cypher sentence generation is carried out on the matched question query type, and answer retrieval is carried out in a graph database.

Step 3.2: and (3) for the user question which cannot be matched in the question template library, obtaining question key information of question representations (such as question types, question entities, question attributes, question relations and the like) in the step (2), constructing corresponding Cypher query sentences, and carrying out answer retrieval in the graph database.

Step 3.3: and processing and integrating answers retrieved by the graph data according to the types of the user questions (such as single-entity questions and answers and single-entity single-attribute questions and answers), and converting the answers into user-friendly answers to return.

Through the steps, the correct analysis and answer of the natural language question based on the domain knowledge graph are realized.

On the other hand, the invention provides an intelligent question-answering system based on the domain knowledge graph, which comprises a template module and a depth module.

The template module is used for preprocessing the question, matching the question template and generating the query sentence.

The user query interface of the template module performs question segmentation and entity recognition after receiving user question input. The template module can generate query sentences through the results returned by the template module or the depth module, search the knowledge base, perform primary search in the database and return corresponding answers according to the question types.

The depth module is used for learning complex features of question questions which cannot be solved by the template module. The depth module comprises a data module and a core task module.

The data module is used for generating a knowledge graph, and data preprocessing is carried out on data in a database (such as a MySQL database and a Neo4j database); obtaining accurate, complete and time-efficient data after data cleaning, constructing a corresponding knowledge graph, and further generating corresponding various problem representations;

The core task module is responsible for reasoning answers to questions and comprises a question coding module, a candidate answer generating module and an answer reasoning module.

The question coding module adopts a bidirectional LSTM coder, codes the questions input by the template module into hidden layer vectors, and outputs the hidden layer vectors to the answer reasoning module.

The candidate answer generation module is composed of three sub-modules, namely answer type coding, answer path coding and answer context coding. The embedded vectors input by the depth module (the embedded vectors are composed of information such as body node names, entity node ids, entity attributes, entity types, neighbor node names of entity nodes, neighbor node ids and the like) generate candidate answers about the questions through the candidate answer generation module, and input the candidate answers to the questions to the answer reasoning module.

The answer reasoning module is composed of a knowledge base perception bidirectional attention module and a generation module, and is used for carrying out answer reasoning on the question hidden layer vector input by the question coding module and the candidate answer representation input by the candidate answer generation module, generating key information answers such as question type, relation type, entity and attribute information of a question, and outputting the key information answers to the template module, and used for constructing key information of a query sentence.

Advantageous effects

Compared with the prior knowledge graph question-answering technology, the invention has the following advantages:

1. the invention builds a complete available data set in a specific field, enriches question types and professionals, and provides good support for question-answering model training based on knowledge graphs. And constructing a complete available knowledge graph, providing supplementary information for the question-answer model by utilizing the knowledge graph in the field, and visually displaying the supplementary information by utilizing a database.

2. Aiming at the problem that the accuracy of a downstream question-answering model is low due to the accumulation of the error rate of an upstream subtask of the question-answering model based on a knowledge graph in a specific field at present, the invention enters the entity identification method based on the minimum editing distance, improves the accuracy of the subtask while saving the subtask time, and eliminates the influence of the professional field knowledge on the relation identification.

3. The problem with the existing question-answer model of the knowledge graph is that the interactivity between the question and the knowledge graph is weak, and a large amount of information in the knowledge graph cannot be fully utilized. In order to solve the problem, the invention introduces a BAMNet network, establishes the association of the information such as entity type, relation path, context and the like in the knowledge base and directly strengthens the bidirectional communication and modeling between the knowledge base and the question.

4. The invention gets rid of the problem that the traditional knowledge graph question-answering model is limited by manual template marking, fuses the rule-based template matching question-answering with the deep learning question-answering, combines the advantages of the rule-based template matching question-answering and the deep learning question-answering, improves the accuracy and the response speed of the system, reduces the maintenance and updating cost, ensures the accuracy of the question-answering model and ensures the interpretability of the model.

Drawings

FIG. 1 is a flow chart of the present invention.

Fig. 2 is a system architecture diagram of the present invention.

Fig. 3 is an exemplary diagram of a knowledge graph of the present invention.

Figure 4 is a view of how much is the length of the inventive step 1 and example 1"sh-60B aircraft? "rule-based problem template matching method block diagram.

Figure 5 is how much is the length of the inventive step 1 and example 1"sh-60B aircraft? And carrying out dynamic cost matrix diagram of minimum editing distance between the rest sentences of the question sentences subjected to the Jieba word segmentation and the entities in the entity list.

Fig. 6 is a diagram of a BAMNet network model involved in the answer reasoning process in step 2 of the present invention.

Fig. 7 is a schematic diagram of a TextCNN model with two channels involved in question type classification in step 2 of the present invention.

Detailed Description

For the purpose of making the technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below by way of examples with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Examples

FIG. 2 is a system frame diagram of the present invention, consisting of two task modules, a depth module and a template module. The user interacts with the system through the user query interface and the answer return interface of the template module. The template module performs preliminary analysis on the question, and then performs template matching. The depth module is used for processing the complex problem which can not be processed by the template module. The depth module consists of a data module and a core task module, wherein the data module cleans data and generates a corresponding knowledge graph, and the core task module encodes the questions, generates candidate answers and infers the answers as shown in fig. 3. The specific flow is shown in figure 1.

FIG. 3 is an exemplary diagram of a knowledge graph in the present invention, and the following illustrates the construction process of the knowledge graph in the military field data set and the military field of the present invention.

The invention acquires information from a non-public data source, and the original data is stored in a MySQL database and comprises military information such as an airplane table, a ground facility table, a ship table, a weapon table and the like, wherein 4581 pieces of airplane table related data, 3064 pieces of ground facility table related data, 3026 pieces of ship table related data and 3337 pieces of weapon table related data are included.

And acquiring entities such as an airplane entity, a ground facility entity, a ship entity, a weapon entity and the like from the database corresponding data table, and corresponding the entity detailed information to the entities one by one. The detailed information of the aircraft entity "SH-60B hawk helicopter" contains 34 attributes in total. The embodiment obtains the relationship information from the non-public data source, and mainly comprises data in an attribute table, a relationship table and a special keyword table, wherein the attribute table data comprise 33 pieces, the relationship table data comprise 19 pieces and the special keyword table data comprise 6 pieces. The detailed information is shown in table 1.

TABLE 1 military field relationship sources

And (3) simply processing the raw information in the MySQL database to obtain detailed information, relationship information and answer information of entities such as airplanes, ground facilities, ships, weapons and the like. However, these data still contain noise, and these noise data have adverse effects on the learning process and the result of the model, so that further normalization processing is required for these data information, and the processing operations include the following three operations:

1. null and zero values in the data are removed. For example, isSelfdefined attribute value of SH-60B type "hawk" helicopter is 0, and the attribute is removed when data is selected.

2. Meaningless or unnecessary attributes in the data are removed. The hypological attribute in the attribute value cannot find the corresponding actual meaning, so this attribute is removed from the relationship table.

3. And meaningless punctuation marks in the entity are removed, and the influence of noise symbols is reduced. For example, the '-' and double quotes in the SH-60B type hawk helicopter are nonsensical symbols, and the removed entity is the SH60B type hawk helicopter.

The operation can remove noise in the data, and reduce the influence of the data noise problem on tasks such as a military knowledge graph-based question-answer model learning process, entity identification, relation extraction and the like.

And taking the data information obtained through the data cleaning operation as an information source of the military knowledge graph triplet, wherein an airplane entity, a ground facility entity, a ship entity and corresponding attribute values or relationship values and special vocabulary values of the airplane entity, the ground facility entity and the ship entity are taken as head and tail entity candidates of the triplet, and the information in the attribute table, the relationship table and the special keyword table is selected as the relationship in the triplet. The triads are obtained through strategies such as attribute screening, random matching and the like, for example, 33 triads related to the attribute of the entity of the SH-60B hawk helicopter can be created.

Aiming at some professional information in the relation table and the special keyword table, the triple tail entity corresponding to each entity is queried by combining the information searched by the network, and meaningful triples are inserted into the triplet set.

Expanding the generated triplet data pair, such as a triplet (SH-60B hawk helicopter, length, 15.2 meters) can generate a corresponding question "what is the length of SH-60B aircraft? "or" how long SH-60B is? ".

An intelligent question-answering method based on domain knowledge graph, as shown in figure 1, comprises the following steps:

what is the length of the question "SH-60B aircraft? If the answer can be made by the rule-based template matching method, the following procedure is performed, as shown in fig. 4:

step 1: preprocessing the user question, including text cleaning and stop word removal. Such as removing the actual meaning words in the question, such as spaces at the end of the beginning of the question, quotation marks, etc.

Step 2: if the question can be matched with the predefined template in the rule-based template matching, the information such as question type, question entity, question attribute, question relation and the like is directly extracted according to the question template.

Further, step 2 includes the steps of:

Step 2.1: what is the question "what is the length of the SH-60B aircraft? "precise segmentation in precise mode with Hidden Markov Model (HMM), the result after segmentation is [ ('SH', 'eng'), ('-' x '), (' 60B ',' eng '), (' aircraft ',' n '), (' ',' uj '), (' length ',' n_attr '), (' yes ',' v '), (' how much ','m '), (' x ') ]'). And removing words contained in the special keyword list to obtain a new sentence 'the length of SH-60B plane'.

Step 2.2: the minimum edit distance algorithm is used, an algorithm iteration matrix of the minimum edit distance algorithm is shown in fig. 5, the minimum edit distance between the length of the SH-60B plane and each entity in the entity list is calculated, and the similarity is calculated according to the minimum edit distance. And adding the entities with the similarity greater than 0.5 into a similarity entity list.

And sequencing the entities in the similarity entity list to obtain a subject entity list 1[ SH-60B type hawk helicopter, SH-60J type hawk helicopter and SH-60F type hawk helicopter ] which takes the first entity [ SH-60B type hawk helicopter ].

Step 2.3: based on the basis of step 2.2, the remaining sentence "length" of the entities in the removed question is determined. And obtaining the attribute length of the question by matching the attributes in the attribute library and the synonym forest of the attribute library.

Step 2.4: and (3) identifying the question as the attribute query type of the two entities by using the entity information [ SH-60B type hawk helicopter ] and the relation information [ length ] obtained in the step 2.2 and the step 2.3. The corresponding cypher statement is "MATCH (node: military equipment) WHERE node name = 'SH-60B type hawk helicopter' RETURN distict node. What is the question "what is the length of the SH-60B aircraft" corresponding to the Cypher statement? "the answer queried in Neo4j graph database is 15.2 meters.

Further by way of example, what is the question "SH-60B aircraft mounted? ", comprising the steps of:

step 1: what is mounted on an "SH-60B aircraft? "matching is performed using a rule-based template matching algorithm. Obtaining an entity [ SH-60B hawk helicopter ] and the remaining sentences [ what is mounted? ]

Step 2: and (3) according to the matching result in the step (1), if the question cannot be matched with the predefined template, inputting the question into a BAMNet network, a textCNN network and a BERT for processing.

Step 2.1: and (3) obtaining the relation types of the questions in the BERT model by the questions in the step (2). Specifically, input "what is mounted" into the BERT model, get the score of each relation category of its full connection layer, as shown in the following formula:

logits＝W·BERT _output +b

Where logits is the output of the fully connected layer, is a vector of dimension C, where C is the total number of relationship categories. W is a weight matrix whose dimension is c×d, where D is the dimension of the BERT output. Matrix multiplication and b is the bias vector. BERT _output Is the output of the BERT model [ CLS ]]The output of the token, the dimension of which is D.

Step 2.2: in the step 2, the question is input into a TextCNN model, as shown in fig. 7, and the type information of the input question is obtained.

Further, step 2 includes the steps of:

step 2.2.1: what is question "SH-60B airplane mounted? In the "input TextCNN model, sentences are expressed as:

wherein,is a series sign, x ₁ And obtaining the word vector representation of the sentence for the k-dimensional word vector of the woed2vec model.

Step 2.2.2: and (5) carrying out text convolution. At the convolution layer, a plurality of convolution kernels are used to convolve the input word embedding matrix. A convolution kernel is represented as a weight matrix w and a bias term b. The convolution operation is expressed as:

c _i ＝f(w·X _i：i+h-1 +b)

wherein, is a matrix multiplication representation, c _i Is the ith element of the convolution result;is a convolution kernel weight matrix; x is x _i：i+h-1 Is a window of the input matrix containing h consecutive word embedding vectors; f is a nonlinear activation function, such as a ReLU activation function; b is the bias term.

Here, h words are taken as a convolution window to generate a new feature vector c _i . In this sentence, for each possible word window { x } _1：h ，x _2：h+1 ，...，x _n-h+1：n -a feature map is generated:

c＝[c ₁ ，c ₂ ，...，c _n-h+1 ]

wherein,further apply the max pooling operation on the feature map +.>To capture the highest value features of each feature map

Step 2.2.3: regularization. Dropout regularization and l2 regularization were used before the last softmax layer to prevent model overfitting. Specifically, dropout randomly discards some hidden units with a probability of 1-p in forward propagation, in the following manner:

wherein,for element-by-element multiplication; />Random mask vector representing masking with 1-p probability,>representing an m-dimensional vector; />For the next to last layer feature map, +.>Representing the feature map after the maximum pooling; y is the output of forward propagation of the full connection layer, and the dimension is the number of sentence categories; w is the full connection layer weight matrix.

Step 2.3: further, the question in step 2 is in the BAMNet model, as shown in fig. 6, and includes the following sub-steps:

step 2.3.1: what is question "SH-60B airplane mounted? "denoted Q, encoded as an intermediate hidden state vector using a bi-directional LSTM network, denoted HQ.

Step 2.3.2: for three kinds of information, namely answer entity type, answer entity context representation and answer entity path, in the knowledge graph of the candidate answer, the two-way LSTM network is used for respectively encoding the three kinds of information into H _t 、H _p 、H _c . Storing answer candidate entities and knowledge-graph information by using a key value memory network, and performing linear projection on the answer information by using the following steps:

wherein,and->Is answer information->D-dimensional key and value representation, +.>Is a hidden layer representation of answer information. Defining M as key memory, its row is denoted +.>The dimensions are +.> Representing a linear mapping function.

Step 2.3.3: and a reasoning module consisting of a bidirectional attention mechanism and a generalization module is used for realizing bidirectional interaction between the question vector identification and the candidate answer coding information.

The knowledge graph attention sensing module is as follows: the vector representation q of d dimension is generated for HQ using a self-attention mechanism as follows:

q＝BiLSTM([H ^Q A ^QQ T，H ^Q ])

A ^QQ ＝softmax((H ^Q ) ^T H ^Q )

wherein BiLSTM represents a bidirectional LSTM network, A ^Q Represents the attention weight, and T represents the matrix transpose.

Acquiring entity type m using self-attention mechanism _t Physical path m _p And the context means m _c It is calculated as follows:

where |a| represents the number of candidate answers, a ^x For attention weight, the problem abstract q and a knowledge base are represented Is a relationship strength of (a). Thereby obtaining the knowledge base information vector representation m= [ m ] _t ；m _p ；m _c ]. By-> Calculating word q in question _i Attention to knowledge-graph information. Maximum attention by means of a maximum pooling layer>

Importance module: the importance module focuses on the correlation between the problem and information of different aspects of the knowledge graph. By the method of A ^QM Normalization is carried out to obtain the attention moment arrayObtaining a problem-aware memory representation using>

A ^QM ＝(M ^k H ^Q ) ^T

Wherein,question awareness representing knowledge base values, A being the number of candidate answers, +.>A value representing a certain answer to the knowledge base. />Representing a question memory perception representation->A key representation representing an answer aspect of the knowledge base. />Is a normalized attention matrix representing the importance of each answer aspect to each candidate answer. softmax is a normalization function, and T represents the matrix transpose. A is that ^M Representing a two-dimensional attention matrix representing a strong maximum relationship between each knowledge base and the most relevant word in the questionDegree. A is that ^QM Representing a three-dimensional attention tensor, representing the strength of the relationship between each word in the question and the knowledge base key, M ^k Is a key representation of the knowledge base.

Reinforcing module: the module re-emphasizes the mutual representation of the question and the knowledge graph using a second level of attention mechanism. A is that ^QM Representing a three-dimensional attention tensor, for A ^QM Performing maximum pooling and normalization to obtainIt is then incorporated into the question representation +.>Finally pass->And obtaining d-dimensional knowledge graph-question reinforcement representation.

Similarly, the reinforcement representation between question-knowledge graph is obtained by:

wherein,knowledge base representation representing question enhancement, +.>Is an attention vector that represents the relevance of each aspect in the knowledge base to the problem. />Is a knowledge base perceived attention vector, which represents the importance of the problem qi in the knowledge base, and is represented by matrix multiplication.

The generation module is used for: a one-hop attention mechanism module is added before the answer module. Calculation of knowledge graph-question reinforcement representation by attention mechanismAnd key memory->The degree of attention between the above is finally passed through BN layer to obtain result +.>The following is shown:

wherein BN is a batch normalization layer,representing a problem representation after a one-jump attention process, q' being a problem representation updated using GRU (recurrent neural network), is +.>For the problem perception digest, a represents the attention weight between the problem and the knowledge base.

Step 2.3.4: for question vector representationAnd candidate answer representation +.>Calculating similarity scores by using the following formulas to obtain score vector representations of question key information such as question types, question relations, question attributes and the like of questions: / >

Step 3: and aggregating the BAMNet, BERT and textCNN information to obtain the final key information answer of the question sentence, and constructing a Cypher sentence to search the answer in the graph data.

Further, step 3 includes the steps of:

step 3.1: based on the output vector of the BERT model in the step 2.1 for the relation full-connection layer, the output vector of the BAMNet for the relation in the step 2.3 is respectively input into a softmax layer, the corresponding probability distribution is obtained as shown in the following formula, and the relation [ mount ] taking the relation type with the maximum probability as a question is summed up:

wherein,respectively represent the index representations of the i, j-th elements of the full connection layer output vector x.

Based on the output vector of the textCNN in the step 2.2 for the question type full-connection layer, and the output vector of the BAMNet in the step 2.3 for the question type full-connection layer, respectively inputting a softmax layer to obtain the corresponding probability distribution, and summing the question type with the highest probability as the question type [ one-jump-tail entity query problem ]. Based on the question attribute of the BAMNet, the final attribute of the question is taken as the final attribute of the question.

Step 3.2: utilizing question key information such as question types [ one-jump tail entity query problem ], question relations [ mount ], question entities [ SH-60B type hawk helicopter ], question attributes [ none ] and the like obtained in 3.1 to construct a Cypher sentence ' MATCH (node { name: ' SH-60B type hawk helicopter ' }) - [ relation: the mount ] - (relay description name) performs the map database search and answer processing, and RETURNs the answer.

TABLE 2 Intelligent question-answering method effects based on military knowledge graph

/>

The invention is applied in the military field, and uses Precision and response time index as the standard for measuring the quality of the model, the effect is shown in table 2, 15 question types common in military questions and answers, 1000 test cases for each question, the accuracy rate obtained by testing 15000 test cases is 81.2%, and the average response time of the model is 0.41s.

As shown in FIG. 2, the intelligent question-answering system based on the domain knowledge graph comprises a template module and a depth module.

The data module is used for generating a knowledge graph and preprocessing data of the database; obtaining accurate, complete and time-efficient data after data cleaning, constructing a corresponding knowledge graph, and further generating corresponding various problem representations;

According to the embodiment of the invention, the problem that the existing knowledge graph question-answering model depends on a manually predefined template and can only answer fixed sentence patterns is solved. The method is applied to the military field, ensures that the model can answer various military non-fixed semantic sentence patterns, simultaneously ensures the interpretability of the model, and greatly improves the decision accuracy of military commanders.

Claims

1. An intelligent question-answering method based on a domain knowledge graph is characterized by comprising the following steps:

step 1: constructing a basic database, inputting a user question, and processing the question by using a rule-based template matching method;

step 2: processing the question sentence by using a BAMNet network, a textCNN and a BERT;

step 3: the question-answering system constructs a corresponding query sentence based on question answer information, performs answer query in a database, further processes the answer, and returns the answer according to the question type;

the method comprises the steps of aggregating BAMNet, BERT and textCNN information to obtain a final key information answer of a question, constructing a Cypher sentence, and carrying out answer retrieval in graph data;

step 3.1: in the problem template matching method based on rules, if the question can be perfectly matched with a template library, performing Cypher sentence generation on the matched question query type, and performing answer retrieval in a graph database;

Step 3.2: for user question which cannot be matched in a question template library, question key information represented by questions is obtained in the step 2, corresponding Cypher query sentences are constructed, and answer retrieval is carried out in a graph database;

step 3.3: and processing and integrating answers retrieved by the graph data according to the question types of the users, and converting the answers into user-friendly answers to return.

2. The intelligent question-answering method based on the domain knowledge graph as claimed in claim 1, wherein the step 1 comprises the steps of:

step 1.1: extracting information of an entity library, an event library, a task library, a relation library and an attribute library from the database, and constructing a basic database required by question and answer;

based on the question library, constructing a corresponding question template knowledge base for possible question types;

based on the field database, constructing a corresponding knowledge graph, wherein the knowledge graph comprises entity nodes, entity attributes, neighbor nodes and relations between the entity nodes and the neighbor nodes;

step 1.2: for question input of a user, a question is processed by using a rule-based question template matching method;

step 1.3: removing the cut stop words and irrelevant words, calculating the minimum edit distance between the cut sentences and the entity list by using a minimum edit distance algorithm, and sequencing the entity list with the obtained similarity to obtain the entity information of the question; then, constructing a synonym forest for the relation and the attribute information to obtain relation attribute information in the question;

3. The intelligent question-answering method based on the domain knowledge graph according to claim 2, wherein the question query type template comprises a single entity single attribute query question, a single entity multi-attribute query question, a single entity single attribute query question whether or not, a one-hop tail entity query question, a relation query question, a two-hop question attribute query question, a two-hop question statistics query question, a two-entity single attribute comparison query question, a single interval query question, a multi-region query question, a multi-entity single attribute query question, an attribute maximum query question, a tail entity confirmation query question, a maximum query question, and a statistics query question.

4. The intelligent question-answering method based on the domain knowledge graph as claimed in claim 1, wherein the step 2 comprises the steps of:

step 2.1: inputting a question into a textCNN model, capturing semantic information of the sentence by using a convolutional neural network, and obtaining type information of the question, wherein L2 regularization and dropout are used for preventing overfitting in a training process;

Step 2.2: the method comprises the steps of removing the residual sentences of entities from the question, calculating the similarity of the relation between the question and a relation library by using a BERT model, and obtaining relation information of the question;

step 2.3: inputting the question into a BAMNet network;

calculating similarity scores for the question vector representations and the candidate answer representations, and selecting the candidate answer entity with the highest score as a question answer;

5. The intelligent question-answering method based on domain knowledge graph according to claim 4, wherein the generated question answers include question type, question entity, question attribute, question relation.

6. The intelligent question-answering method based on domain knowledge graph as set forth in claim 1 or 4, wherein in step 2, a question is input into the BERT model to obtain a score of each relation category of the full connection layer, as shown in the following formula:

logits＝W·BERT _output +b

wherein logits is the output of the fully-connected layer, is a vector with dimension C, and C is the total number of relation categories; w is a weight matrix whose dimensions are C D, where D is BThe dimension of the ERT output; is a matrix multiplication, b is a bias vector; BERT _output Is the output of the BERT model [ CLS ]]An output of the token, the dimension of which is D;

step 2.2: step 2, inputting a question into a textCNN model to obtain type information of the input question;

step 2.2.1: inputting a question into a TextCNN model, and expressing the sentence as:

x _1:n ＝x ₁ ⊕x ₂ ⊕…⊕x _n

where ∈ is a series sign, x ₁ Obtaining word vector representation of sentences for k-dimensional word vectors of the woed2vec model;

step 2.2.2: a text convolution; at the convolution layer, performing convolution operation on the input word embedding matrix by using a plurality of convolution kernels; a convolution kernel is represented as a weight matrix w and a bias term b; the convolution operation is expressed as:

c _i ＝f(w·x _i:i+h-1 +b)

Wherein, is a matrix multiplication representation, c _i Is the ith element of the convolution result;is a convolution kernel weight matrix; x is x _i:i+h-1 Is a window of the input matrix containing h consecutive word embedding vectors; f is a nonlinear activation function, such as a ReLU activation function; b is a bias term;

taking the h words as a convolution window to generate a new feature vector c _i The method comprises the steps of carrying out a first treatment on the surface of the In this sentence, for each possible word window { x } _1:h ,x _2:h+1 ,…,x _n-h+1:n -a feature map is generated:

c＝[c ₁ ,c ₂ ,…,c _n-h+1 ]

Step 2.2.3: regularization; dropout regularization and l2 regularization are used before the last softmax layer;

dropout randomly discards several hidden units with probability 1-p in forward propagation in the following way:

wherein,for element-by-element multiplication; />Random mask vector representing masking with 1-p probability,>representing an m-dimensional vector; />For the next to last layer feature map, +.>Representing the feature map after the maximum pooling; y is the output of forward propagation of the full connection layer, and the dimension is the number of sentence categories; w is a weight matrix of the full connection layer;

step 2.3: the question is in the BAMNet model, comprising the following sub-steps:

Step 2.3.1: representing the question as Q, encoding it as an intermediate hidden state vector representation H using a bi-directional LSTM network ^Q ；

Step 2.3.2: for the answer entity type of candidate answers in the knowledge graph, the answer entity context representation and the answer entityThree kinds of information of the route, use the two-way LSTM network to encode it as H separately _t 、H _p 、H _c The method comprises the steps of carrying out a first treatment on the surface of the Storing answer candidate entities and knowledge-graph information by using a key value memory network, and performing linear projection on the answer information by using the following steps:

wherein,and->Is answer information->D-dimensional key and value representation, +.>A hidden layer representation of answer information; defining M as key memory, its row is denoted +.>The dimensions are +.> Representing a linear mapping function;

step 2.3.3: using an inference module formed by a bidirectional attention mechanism and a generalization module to realize bidirectional interaction between question vector identification and candidate answer coding information;

the knowledge graph attention sensing module is as follows: pair H using self-attention mechanism ^Q A vector representation q of d dimensions is generated as follows:

A ^QQ ＝softmax((H ^Q ) ^T H ^Q )

wherein BiLSTM represents a bidirectional LSTM network, A ^Q Representing attention weights, T representing matrix transpose;

where |a| represents the number of candidate answers, a ^x For attention weight, the problem abstract q and a knowledge base are representedIs a relationship strength of (2); thereby obtaining the knowledge base information vector representation m= [ m ] _t ；m _p ；m _c ]The method comprises the steps of carrying out a first treatment on the surface of the By-> Calculating word q in question _i Attention to knowledge-graph information; maximum is obtained through a maximum pooling layerAttention->

Importance module: the importance module side focuses on the correlation of information of different aspects of the problem and the knowledge graph; by the method of A ^QM Normalization is carried out to obtain the attention moment arrayObtaining a problem-aware memory representation using>

A ^QM ＝(M ^k H ^Q ) ^T

Wherein,question awareness representing knowledge base values, A being the number of candidate answers, +.>A value representation representing a certain answer to the knowledge base; />Representing a question memory perception representation->A key representation representing an answer aspect of the knowledge base; />Is a normalized attention matrix representing the importance of each answer aspect to each candidate answer; softmax is a normalization function, T represents the matrix transpose; a is that ^M Representing a two-dimensional attention matrix representing the maximum strength of relationship between each knowledge base and the most relevant word in the question; a is that ^QM Representing a three-dimensional attention tensor, representing the strength of the relationship between each word in the question and the knowledge base key, M ^k A key representation for the knowledge base;

reinforcing module: the module uses a second-level attention mechanism to strengthen the mutual representation of the question sentence and the knowledge graph again; a is that ^QM Representing a three-dimensional attention tensor, for A ^QM Performing maximum pooling and normalization to obtainWhich is then incorporated into the problem representationFinally pass->Obtaining d-dimensional knowledge graph-question reinforcement representation;

wherein,knowledge base representation representing question enhancement, +.>Is an attention vector representing the relevance of each aspect in the knowledge base to the problem; />Is a knowledge base perceived attention vector, which represents the importance of the problem qi in the knowledge base, and is represented by matrix multiplication;

the generation module is used for: adding a one-hop attention mechanism module before the answer module; calculation of knowledge graph-question reinforcement representation by attention mechanismAnd key memory->Attention betweenThe degree, finally, the result is obtained through BN layer>The following is shown:

wherein BN is a batch normalization layer,representing a question after a one-jump attention process, q' being the question after updating with GRU,/for example>For a problem perception abstract, a represents the attention weight between the problem and the knowledge base;

Step 2.3.4: for question vector representationAnd candidate answer representation +.>Calculating similarity score by using the following formula to obtain question type, question relation, question attribute and the like of the questionsScore vector representation of sentence key information:

7. an intelligent question-answering system based on a domain knowledge graph is characterized by comprising a template module and a depth module;

the template module is used for preprocessing a question, matching the question template and generating a query sentence;

the user query interface of the template module performs question segmentation and entity recognition after receiving user question input; the template module can generate query sentences through the results returned by the template module or the depth module, search the knowledge base, perform primary search in the database, and return corresponding answers according to the question types;

the depth module is used for learning complex features of questions and sentences which cannot be solved by the template module; the depth module comprises a data module and a core task module;

The core task module is responsible for reasoning answers to questions and comprises a question coding module, a candidate answer generating module and an answer reasoning module;

the question coding module adopts a bidirectional LSTM coder, codes the questions input by the template module into hidden layer vectors, and outputs the hidden layer vectors to the answer reasoning module;

the candidate answer generation module consists of three sub-modules, namely answer type coding, answer path coding and answer context coding; the embedded vector input by the depth module generates a candidate answer about the question through the candidate answer generation module and inputs the candidate answer to the answer reasoning module;

the answer reasoning module is composed of a knowledge base perception bidirectional attention module and a generation module and is used for carrying out answer reasoning on the question hidden layer vector input by the question coding module and the candidate answer representation input by the candidate answer generation module, generating a question key information answer and outputting the question key information answer to the template module for constructing key information of a query sentence.