CN109271506A - A kind of construction method of the field of power communication knowledge mapping question answering system based on deep learning - Google Patents

Abstract

The construction method for the field of power communication knowledge mapping question answering system based on deep learning that the present invention relates to a kind of, realize step are as follows: step 1: semanteme parsing, the problem of being proposed to user with natural language q is pre-processed, therefrom extract the keyword of user query, the entity w as question sentence such as inquiry sentence focus of attention.Step 2: semantic expressiveness will pass through pretreated natural language problem vectorization, while the matching degree by the Candidate Set vectorization of answer a, for subsequent computational problem q and answer a.Step 3: calculated by semantic matching degree, inquiry and the methods of reasoning, find out most matched with problem q, most accurate answer a so that the question and answer are to (q, score S (q, a) highest a).Study through the invention by knowledge mapping construct question answering system the national grid communications field feasibility.

Description

Construction method of electric power communication field knowledge map question-answering system based on deep learning

Technical Field

The invention belongs to the application range of Knowledge maps in the power communication industry, and particularly relates to a method for constructing a Knowledge map question-answering system (Knowledge base responding) based on Deep learning (Deep learning).

Background

Knowledge Graph (Knowledge Graph): is essentially a structured semantic knowledge base with pairs of entities with attributes linked by relationships. Which contains a large number of entity-pair relationships that are used to symbolically describe concepts in the physical world and interrelationships between concepts. From a graph perspective, a knowledge graph is actually a conceptual network, with nodes of the network being entities in the real world, and edges of the network representing connections (relationships) between pairs of entities^[1]。

The concept of a knowledge graph was first introduced in 2012 by Google corporation to enhance the knowledge base of its search engine functionality^[2]The method aims to systematize the search keywords so that each keyword has a complete knowledge system, thereby improving the search quality^[3]. Although the traditional search engine technology can quickly screen and sort a large number of webpages according to the query structure of a user, the efficiency of information retrieval is improved. However, as the user cannot quickly and accurately feed back specific answers to questions, the information retrieval method gradually fails to meet the needs of the user as the total amount of internet information increases explosively. The appearance of the knowledge graph provides a feasible scheme for solving the problem of user information retrieval under big data^[4].

Common knowledge graphs can be divided into two broad categories, general knowledge graphs and industry knowledge graphs. The universal knowledge graph emphasizes that the knowledge graph is constructed on the basis of big data, and a typical large-scale knowledge base is provided with Freebase^[5]、Wikidata^[6]、DBpedia^[7]、 YAGO^[8]And the like, which not only contain large amounts of semi-structured and unstructured data, but also have high domain coverage. And industry knowledge graph usageThe knowledge base is constructed by data of a specific industry, so the data size is smaller than that of a general knowledge graph generally, but the attribute and the data mode of an entity are richer. Industry knowledge maps are more accurate than general knowledge maps, but are only applicable to specific areas. At present, industry knowledge maps are researched and developed in the industries of electronic commerce, finance and the like^[9]。

Knowledge-graphs are typically represented by "triplets", i.e., G ═ E, R, S. Wherein E ═ { E ═ E₁,e₂,...,e_|E|Represents a set of entities in the knowledge base, representing a total of | E | different entities; r ═ R₁,r₂,...,r_|R|Represents the set of entity relationships between pairs of entities in the knowledge base, | R | is the number of entity relationships;what is represented is a set of triples of the knowledge-graph, i.e., a representation of the entire knowledge-base. The triple set of the knowledge-graph represents the relationship between entities or the relationship between concepts and attributes.

Question Answering System (Question Answering System): QA for short is an advanced form of information retrieval, and its simple and accurate interaction mode makes the question-answering system become another research hotspot in the application field of Natural Language Processing (NLP). Different from the traditional search engine, the question-answering system accepts that the user proposes questions (such as how large yaoming is. While the traditional network information query tool mainly searches based on matching of keywords, although the scheme can generally meet the requirements of users, due to the fact that the way of expressing the problems by the keywords lacks background information on the context of the problems, the retrieved results are often tens of thousands of related web pages which are sorted by PageRank, and the requirements of the users are often a small part of the related web pages. So that the question-answering system matches the keywordsThere are substantial differences between conventional search engines that are configured to return a collection of related documents^[10]。

In recent years, many question-answering systems have appeared at home and abroad. A relatively sophisticated question-answering system developed abroad is the Start of the Artificial Intelligence laboratory of the university of Mandarin province^[11]AnswerBus of michigan university^[12]、IBM^[13]A statistical-based question-answering system, etc. Compared with an English question-answering system, the Chinese question-answering system is more complex in construction process because a large-scale Chinese knowledge base is very difficult to construct. At present, the inquiry and answer system developed at home is NKI developed by Chinese academy of sciences^[14]Remote intelligent question answering system designed by Shanghai university of transportation^[15]And an automatic question-answering system based on WeChat developed by Kaiki, etc. of electronic science and technology university^[16]And the like. Compared with the English question-answering system, the Chinese question-answering system has late research start and is not mature enough, and the research start is related to various factors such as the complexity of grammar, syntax and semantics of Chinese and the like^[17]。

The question-answering system can be divided into the following categories according to its representation: chat robots (such as Siri), knowledge base-based question-answering systems, intelligent search engines, free text-based question-answering systems. The knowledge can be divided into a question-answering system in a limited domain and a question-answering system in an open domain according to the domain range to which the knowledge belongs. The question-answering system in the open field is more complex than that in the limited field, and the question-answering system in the limited field is more targeted and professional^[18]。

Deep learning: as a branch of machine learning, it is a research focus in the field of machine learning in recent years. The advent of deep learning has led to a great deal of development in the subject areas of Natural Language Processing (NLP) and computer vision. The concept of deep learning stems from the study of artificial neural networks. In 2006, Hinton et al proposed the concept of "deep learning". As a branch of machine learning, deep learning is opposite to 'shallow learning', and the motivation is to establish and simulate a neural network for human brain to analyze and learn, and to simulate the neural networkInterpreting data in the human brain, e.g. processing of images, sounds and text^[19]。

Before deep learning occurs, algorithms mainly adopted in the field of machine learning include logistic regression (logistic regression), Support Vector Machine (SVM), Hidden Markov Model (HMM), and the like, and all the models do not include a hidden node layer or only include a hidden layer, and therefore all the models belong to shallow learning models. In the last 80 th century, Zipser et al proposed a back propagation algorithm^[20]The algorithm can automatically correct the parameters of the artificial neural network model in the training process, and greatly promotes the development of the artificial neural network model and machine learning. In 2011, researchers of microsoft corporation adopt a deep learning technology to greatly reduce the error rate of voice recognition, and a technical breakthrough is brought to voice recognition. In 2012, Google Brain introduced the Google Brain project and used a Deep Neural Network (DNN) with 10 hundred million nodes for training to intelligently recognize pictures. In addition, large internet companies such as Facebook and Baidu also invest deep learning into research, so as to see the weight of deep learning in the future machine learning field^[20]。

Disclosure of Invention

With the increasing maturity of knowledge graph construction technology, knowledge base construction for structured and unstructured data has become possible. After the Chinese general knowledge graph (such as 'Baidu consciousness') is gradually formed, the Chinese industry knowledge graph also appears in succession. At present, common application of knowledge graph is basically staying in the aspects of 'search engine' and 'question-answering system'. In the application aspect of the question-answering system, compared with the traditional question-answering system, the question-answering system based on the knowledge map mostly has one or more knowledge bases, and can answer the questions posed by the users by utilizing a retrieval algorithm or a logical reasoning algorithm. Thus, the number of entities in the knowledge-graph directly determines how well such a question-answering system performs. The question-answering system based on the knowledge graph has the advantage of being capable of accurately reasoning and answering questions posed by users.

The invention provides a method for constructing a question-answering system based on an industry knowledge graph by utilizing various data sources of national power grid power communication. Because national power grid communication has a large amount of structured data and unstructured text data, in order to better perform entity recognition on the unstructured data in the process of constructing a knowledge graph, a Natural Language Processing (NLP) technology based on deep learning is adopted to perform word segmentation, named entity recognition, entity relationship recognition and the like on the text data. In general, the construction process of the knowledge-graph question-answering system based on deep learning can be divided into two parts: the first part is the construction of a knowledge graph based on deep learning; and the second part is the construction of a question-answering system based on a knowledge graph. In view of the fact that the construction process of the question-answering system is mainly described herein, the construction of the knowledge graph is omitted, namely, the knowledge graph based on national power grid communication is assumed to be constructed. A vector modeling method based on deep learning is adopted as a question-answering algorithm, and the method has the core idea that: and mapping the entity of the knowledge base and the question in the natural language into the same vector space, and searching out the optimal candidate answer by calculating the semantic similarity.

The invention adopts the following scheme to solve the technical problems:

a construction method of a knowledge graph question-answering system in the electric power communication field based on deep learning is characterized in that natural language questions put forward by users are expressed as q ═ omega₁...ω_nAll candidate answers to the question are represented as answer candidate set C_qThe method specifically comprises the following steps:

step 1: semantic analysis, specifically, preprocessing a question q which is provided by a user through natural language, extracting keywords inquired by the user from the question q, and taking the focus concerned by an inquiry sentence and the like as an entity w of the inquiry sentence;

step 2: performing semantic representation, specifically vectorizing the preprocessed natural language questions, and vectorizing a candidate set of the answer a at the same time, wherein the candidate set is used for subsequently calculating the matching degree of the question q and the answer a;

and step 3: and finding out the most accurate answer a which is most matched with the question q through semantic matching degree calculation, query and reasoning so that the score S (q, a) of the question-answer pair (q, a) is the highest.

In the above method for constructing a knowledge graph question-answering system in the field of power communication based on deep learning, the step 1 specifically includes:

step 1.1, problem word segmentation and part of speech tagging: performing problem word segmentation and part-of-speech tagging by adopting a Chinese lexical analysis system; after word segmentation, the natural language question is expressed as q ═ ω₁...ω_nWherein w is_nIs a Chinese word;

step 1.2, problem named entity recognition: screening out the named entities described in the question q according to the part of speech marked in the step 1.1, and using the named entities as keywords and focus of attention of the question; the common method for extracting the keywords is a statistical method, namely, the weights of the candidate words are determined, and the candidate words with large weights are screened out to be used as final keywords; the specific method for extracting the keywords comprises the following steps:

step 1.21, performing word segmentation processing on the text;

step 1.22, filtering stop words, namely, fictional words, tone words, punctuation marks and the like;

step 1.23, respectively calculating word frequency factors and position factors of the words according to the counted word frequency and position information;

step 1.24, calculating the weights of the words by using a specific word weight function, sequencing the words, and selecting the word with the larger weight as a keyword;

step 1.23, after the extraction process, identifying named entities so as to calculate the subject and the object of the sentence;

step 1.3, generating answer candidate set: according to the question qThe key words and the named entities are searched in a knowledge base, the entity nodes and the adjacent nodes within the range of 2-hops are searched, and an answer candidate set C of the question q is formed by the key words and the named entities_q。

In the above method for constructing a knowledge graph question-answering system in the field of power communication based on deep learning, the step 2 specifically includes:

step 2.1, problem q vectorization: for preprocessed problem q ═ ω₁...ω_nEach word ω thereof is expressed_jMapping into a vector wj of a low-dimensional space, namely vectorization of the word; embedding matrix W by words_vConvert it into a d-dimensional distributed vector, i.e.

w_j＝W_vu(ω_j)

Wherein (u (ω)_j)∈{0,1}^|V|) As the word omega_jThe corresponding one-hot form of the corresponding one,is a word embedding matrix (word embedding matrix), and | V | represents the size of the vocabulary; in the training process of the MCCNNs network, W_vIs continuously updated as a hyper-parameter;

then, a representation of the problem q is computed using a sliding window at the convolutional layer of the neural network; for example, for column i of MCCNNs, an n-dimensional vector of the problem q is calculated as follows:

wherein 2s +1 is the size of the sliding window,is a weight matrix of the convolutional layer,denotes a deviation vector, h (-) is notLinear functions (e.g., sigmoid function, tanh function, etc.);

finally, training at the maximum pooling level outputs a vector representation of the problem q of fixed size, where the output of the i-th column of active units of MCCNNs is:

wherein max is a maximum function; this results in a low-dimensional space vector f for the problem q_i(q)；

Step 2.2, vectorization of the candidate answer set: for three features of the answer, the answer path, the answer context information and the answer type, the vectorization process is respectively shown as follows:

for Answer Path (Answer Path): the represented is the incidence relation between the answer node and the named entity in the question; then the distributed representation g of the answer path₁(a) The vector representation of (c) can be calculated using the following formula:

wherein, | |₁Denotes the L1 norm, u_p(a)∈R^|R|×1A binary vector is used to indicate whether each association of the answer path exists,is a parameter matrix, | R | is the number of incidence relations;

for Answer Context information (Answer Context): its role is a constraint to deal with the problem; taking the entity relationship and the entity corresponding to the answer entity in the range of 1 hop (hop) as the context information of the answer entity; the distributed expression of answer context information obtained in the same way is:

wherein u is_c(a)∈R^|C|×1A binary vector representing whether an inode exists, | C | represents the number of entity-pair relationships in which an answer context exists,is a parameter matrix;

for Answer Type (Answer Type): type is a special entity relationship, such as the type of time 2018-01-01 is datetime; in the same way, the corresponding distributed expression is obtained as:

wherein,for embedded type of matrix, u_t(a)∈R^|T|×¹A binary variable representing whether an answer type exists, | T | is the number of answer types;

in combination with the above description, the final output function of the MCCNNs is obtained as:

in the above method for constructing a knowledge graph question-answering system in the field of power communication based on deep learning, the step 3 specifically includes:

step 3.1, training MCCNNs neural network models: for each correct answer a ∈ A to the question q_qRandomly from the answer candidate set C_qExtracting k wrong answers a' as samples, and taking the samples as negative samples of the training model parameters; then Hinge loss letterThe number may be defined as:

l(q,a,a')＝(m-S(q,a)+S(q,a'))₊

where S (·,) is the scoring function defined in step 2, m is an interval parameter used to normalize the interval between the two scores S (q, a) and S (q, a'), and (z)₊Max {0, z }; the subjective function is then:

wherein | A_qL represents the number of exact answers,is a set of k wrong answers;

training the model by adopting a back propagation algorithm of a neural network, and continuously updating the model by utilizing a gradient descent algorithm by calculating a gradient value of a hyper-parameter;

step 3.2, semantic matching and reasoning: all answer candidate sets C related to the question q are retrieved in the testing process_q(ii) a For each candidate answerCalculate its scoreThe answer a with the highest score in the candidate answer set is the most accurate answer of the question q;

for a natural language question q, there may be more than one answer due to its accuracy, such as "model type of device"; therefore, a scale criterion is needed to determine the final answer; the formula of the rating criterion is as follows:

where m is the interval defined in step 3.1.

Therefore, the invention has the following advantages:

1. the application range of the knowledge graph mainly focuses on two aspects of a search engine and a question-answering system at present, and the main content of the invention is to construct the question-answering system in the telecommunication field by means of the data structure of the knowledge graph. The invention aims to research the feasibility of a question-answering system constructed by a knowledge graph in the field of national power grid communication.

2. Because national power grid communication has a large amount of structured data and unstructured text data, in the process of constructing the knowledge graph, in order to better perform entity recognition on the unstructured data, a deep learning MCCNNs-based neural network model is adopted to perform word segmentation, named entity recognition, entity relationship recognition and the like on the text data.

Drawings

FIG. 1 is a schematic workflow diagram of a question-answering system.

Fig. 2 is a schematic diagram of the operation of the deep learning based question-answering system.

Detailed Description

Suppose we represent the natural language question posed by the user as q ═ ω₁...ω_nAll candidate answers to the question are represented as answer candidate set C_qThe operation principle of the conventional question-answering system based on deep learning is shown in fig. 2. The question-answering algorithm of the question-answering system based on deep learning is specifically implemented as follows:

step 1: semantic parsing is the initialization phase of the question-answering algorithm, i.e. the preprocessing process of the natural language question q. As is known, the grammar structure of chinese is more complex than that of english, so that processes such as word segmentation and part-of-speech tagging are required to be performed on a text, so that keywords of a problem and the focus of attention on the problem can be retrieved from the text.

Step 1.1 problem participles and part-of-speech tagging

The Chinese text word segmentation takes 'word' as a processing object, so that the Chinese text needs to be cut into parts according to the word and marked with a cut label. After the text is participled, the grammar category of each word in the sentence is determined, and the part of speech is determined and labeled. The above processes are respectively called Chinese text segmentation and part-of-speech tagging processes. Here we use the chinese lexical analysis system (ICTCLAS for short) to solve the above problem. After word segmentation, we express the natural language problem as q ═ ω₁...ω_nWherein w is_nIs a Chinese word.

Step 1.2 problem named entity recognition

And (4) screening out the named entities described in the question q according to the part of speech marked in the step 1.1, and using the named entities as keywords and focus of attention of the question. The common method for extracting keywords is a statistical method, that is, by determining the weights of candidate words, a final keyword with a higher weight is selected from the candidate words. The basic flow of keyword extraction is as follows:

1) performing word segmentation processing on the text;

2) filtering stop words, namely, fictional words, tone words, punctuation marks and the like;

3) respectively calculating word frequency factors and position factors of the words according to the counted word frequency and position information;

4) and calculating the weights of the words by using a specific word weight function, sequencing the words, and selecting the words with larger weights as keywords.

Meanwhile, in the extraction process of the keywords, named entities are identified so as to calculate the subject and the object of the sentence.

Step 1.3 generating answer candidate set

According to the keywords and named entities of the question q, searching the entity node and the adjacent nodes within the range of 2-hops in a knowledge base to form an answer candidate set C of the question q_q。

Step 2: semantic representation, namely the question q preprocessed in step 1 and a candidate answer set C_qVectorization.

Here we use the multi-convolutional neural network MCCNNs (multi-convolutional neural networks) model shown in FIG. 2 to solve the problem of the Answer Path (Answer Path), Answer Context information (Answer Context), Answer Type (Answer Type) in three aspects of q and C_qLearning is performed. The three aspects are respectively trained by three columns of activation units of the neural network, and sentence vectors of the final problem q are respectively marked as f₁(q),f₂(q),f₃(q) is carried out. Meanwhile, for each candidate answer a of the question q, a vector representation form is calculated and is marked as g₁(a),g₂(a),g₃(a) In that respect F is then₁ ^T(q)g₁(a)、Andand respectively representing the training output values of three aspects of answer paths, context information and answer types. Using the above vectors we can calculate the matching degree of the specific question-answer pair (q, a), and the calculation formula is:

the vectorization process for the question q and the candidate answer a is as follows:

step 2.1 problem q vectorization

For preprocessed problem q ═ ω₁...ω_nEach word ω thereof is expressed_jVector w mapped into a low dimensional space_jI.e. vectorization of words. We embed matrix W by words_vConvert it into a d-dimensional distributed vector, i.e.

w_j＝W_vu(ω_j)

Wherein (u (ω)_j)∈{0,1}^|V|) As the word omega_jThe corresponding one-hot form of the corresponding one,is a word embedding matrix (word embedding matrix), and | V | represents the size of the vocabulary. In the training process of the MCCNNs network, W_vIs continuously updated as a hyper-parameter.

A representation of the problem q is then computed using a sliding window at the convolutional layer of the neural network. For example, for column i of MCCNNs, an n-dimensional vector of the problem q is calculated as follows:

wherein 2s +1 is the size of the sliding window,is a weight matrix of the convolutional layer,representing the deviation vector, h (-) is a non-linear function (e.g., sigmoid function, tanh function, etc.).

Finally, training at the maximum pooling level outputs a vector representation of the problem q of fixed size, where the output of the i-th column of active units of MCCNNs is:

where max is a maximum function. Thus we finally get the low dimensional space vector f of the problem q_i(q)。

Step 2.2 vectorization of candidate answer sets

For three features of the answer: answer path, answer context information and answer type, we show the vectorization process:

1) for Answer Path (Answer Path): what is shown is the association of the answer node with the named entity in the question. Then the distributed representation g of the answer path₁(a) The vector representation of (c) can be calculated using the following formula:

wherein | · | purple sweet₁Represents L₁Norm, u_p(a)∈R^|R|×1A binary vector is used to indicate whether each association of the answer path exists,and the | R | is the number of the incidence relations.

2) For Answer Context information (Answer Context): which acts as a constraint for dealing with the problem. We take the answer entity corresponding to the entity relationships and entities in the range of 1 hop (hop) as the context information of the answer entity. In the same way we can get a distributed expression of answer context information as:

wherein u is_c(a)∈R^|C|×1A binary vector representing whether an inode exists, | C | represents the number of entity-pair relationships in which an answer context exists,is a parameter matrix.

3) For Answer Type (Answer Type): a type is a special entity relationship, such as a time 2018-01-01 type is datetime. In the same way we can get the corresponding distributed expression as:

wherein,for embedded type of matrix, u_t(a)∈R^|T|×¹A binary variable representing whether an answer type exists, | T | is the number of answer types.

In conjunction with the above description, we have obtained the final output function of MCCNNs as:

and step 3: obtaining the most accurate answer a

Step 3.1 training MCCNNs neural network model

For each correct answer a ∈ A to the question q_qRandomly from the answer candidate set C_qAnd extracting k wrong answers a' as samples, and taking the samples as negative samples of the parameters of the training model. Then the Hinge loss function can be defined as:

l(q,a,a')＝(m-S(q,a)+S(q,a'))₊

where S (·,) is the scoring function defined in step 2, m is an interval parameter used to normalize the interval between the two scores S (q, a) and S (q, a'), and (z)₊Max {0, z }. The subjective function is then:

wherein | A_qL represents the number of exact answers,is a set of k wrong answers.

Here we train the model using the back-propagation algorithm of the neural network, updating it continuously with the gradient descent algorithm by calculating the gradient values of the hyper-parameters.

Step 3.2 semantic matching and reasoning

During the test, we have retrieved all answer candidate sets C for the question q_q. For each candidate answerCalculate its scoreThe answer a with the highest score in the candidate answer set is the most accurate answer to the question q.

For a natural language question q, there may be more than one answer, such as "model type of device", due to its accuracy. Therefore, a criterium is needed to determine the final answer. The formula of the rating criterion is as follows:

where m is the interval defined in step 3.1.

By combining the steps, a specific process of the question-answering system algorithm based on deep learning is completed. Compared with the traditional vector modeling method, the vector modeling algorithm based on deep learning solves the semantic sequence of the problem which is not considered, and improves the capability of the algorithm for extracting features.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the invention relates may modify, supplement or substitute the specific embodiments described, without however departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims (4)

1. A construction method of a knowledge graph question-answering system in the electric power communication field based on deep learning is characterized in that natural language questions put forward by users are expressed as q ═ omega₁...ω_nAll candidate answers to the question are represented as answer candidate set C_qThe method specifically comprises the following steps:

step 1: semantic analysis, specifically, preprocessing a question q which is provided by a user through natural language, extracting keywords inquired by the user from the question q, and taking the focus concerned by an inquiry sentence and the like as an entity w of the inquiry sentence;

step 2: performing semantic representation, specifically vectorizing the preprocessed natural language questions, and vectorizing a candidate set of the answer a at the same time, wherein the candidate set is used for subsequently calculating the matching degree of the question q and the answer a;

and step 3: and finding out the most accurate answer a which is most matched with the question q through semantic matching degree calculation, query and reasoning so that the score S (q, a) of the question-answer pair (q, a) is the highest.

2. The method for constructing the power communication field knowledge graph question-answering system based on deep learning according to claim 1, wherein the step 1 specifically comprises:

step 1.1, problem word segmentation and part of speech tagging: performing problem word segmentation and part-of-speech tagging by adopting a Chinese lexical analysis system; after word segmentation, the natural language question is expressed as q ═ ω₁...ω_nWherein w is_nIs a Chinese word;

step 1.2, problem named entity recognition: screening out the named entities described in the question q according to the part of speech marked in the step 1.1, and using the named entities as keywords and focus of attention of the question; the common method for extracting the keywords is a statistical method, namely, the weights of the candidate words are determined, and the candidate words with large weights are screened out to be used as final keywords; the specific method for extracting the keywords comprises the following steps:

step 1.21, performing word segmentation processing on the text;

step 1.22, filtering stop words, namely, fictional words, tone words, punctuation marks and the like;

step 1.23, respectively calculating word frequency factors and position factors of the words according to the counted word frequency and position information;

step 1.24, calculating the weights of the words by using a specific word weight function, sequencing the words, and selecting the word with the larger weight as a keyword;

step 1.23, after the extraction process, identifying named entities so as to calculate the subject and the object of the sentence;

step 1.3, generating answer candidate set: according to the keywords and named entities of the question q, the entity nodes and the nodes within the range of 2-hops are searched in the knowledge baseNeighboring nodes, together forming a candidate set C of answers to the question q_q。

3. The method for constructing the electric power communication field knowledge graph question-answering system based on deep learning according to claim 1, wherein the step 2 specifically comprises:

step 2.1, problem q vectorization: for preprocessed problem q ═ ω₁...ω_nEach word ω thereof is expressed_jVector w mapped into a low dimensional space_jI.e. vectorization of the word; embedding matrix W by words_vConvert it into a d-dimensional distributed vector, i.e.

w_j＝W_vu(ω_j)

Wherein (u (ω)_j)∈{0,1}^|V|) As the word omega_jThe corresponding one-hot form of the corresponding one,is a word embedding matrix (word embedding matrix), V represents the size of the vocabulary; in the training process of the MCCNNs network, W_vIs continuously updated as a hyper-parameter;

then, a representation of the problem q is computed using a sliding window at the convolutional layer of the neural network; for example, for column i of MCCNNs, an n-dimensional vector of the problem q is calculated as follows:

wherein 2s +1 is the size of the sliding window,is a weight matrix of the convolutional layer,representing deviation vectors, h (-) is a nonlinear function (such as sigmoid function, tanh function, etc.);

finally, training at the maximum pooling level outputs a vector representation of the problem q of fixed size, where the output of the i-th column of active units of MCCNNs is:

wherein max is a maximum function; this results in a low-dimensional space vector f for the problem q_i(q)；

Step 2.2, vectorization of the candidate answer set: for three features of the answer, the answer path, the answer context information and the answer type, the vectorization process is respectively shown as follows:

for Answer Path (Answer Path): the represented is the incidence relation between the answer node and the named entity in the question; then the distributed representation g of the answer path₁(a) The vector representation of (c) can be calculated using the following formula:

wherein | · | purple sweet₁Denotes the L1 norm, u_p(a)∈R^|R|×1A binary vector is used to indicate whether each association of the answer path exists,is a parameter matrix, | R | is the number of incidence relations;

for Answer Context information (Answer Context): its role is a constraint to deal with the problem; taking the entity relationship and the entity corresponding to the answer entity in the range of 1 hop (hop) as the context information of the answer entity; the distributed expression of answer context information obtained in the same way is:

wherein u is_c(a)∈R^|C|×1A binary vector representing whether an inode exists, | C | represents that an entity pair exists for an answer contextThe number of the series is,is a parameter matrix;

for Answer Type (Answer Type): type is a special entity relationship, such as the type of time 2018-01-01 is datetime; in the same way, the corresponding distributed expression is obtained as:

wherein,for embedded type of matrix, u_t(a)∈R^|T|×1A binary variable representing whether an answer type exists, | T | is the number of answer types;

in combination with the above description, the final output function of the MCCNNs is obtained as:

4. the method for constructing the electric power communication field knowledge graph question-answering system based on deep learning according to claim 1, wherein the step 3 specifically comprises:

step 3.1, training MCCNNs neural network models: for each correct answer a ∈ A to the question q_qRandomly from the answer candidate set C_qExtracting k wrong answers a' as samples, and taking the samples as negative samples of the training model parameters; then the Hinge loss function can be defined as:

l(q,a,a')＝(m-S(q,a)+S(q,a'))₊

wherein S (-) is the score function defined in step 2, m is the interval parameter for normalizing the two scores S (q, a) and S (q, a')

A spacing therebetween, and (z)₊Max {0, z }; then subjective letterThe number is as follows:

wherein | A_qL represents the number of exact answers,is a set of k wrong answers;

training the model by adopting a back propagation algorithm of a neural network, and continuously updating the model by utilizing a gradient descent algorithm by calculating a gradient value of a hyper-parameter;

step 3.2, semantic matching and reasoning: all answer candidate sets C related to the question q are retrieved in the testing process_q(ii) a For each candidate answerCalculate its scoreThe answer a with the highest score in the candidate answer set is the most accurate answer of the question q;

for a natural language question q, there may be more than one answer due to its accuracy, such as "model type of device"; therefore, a scale criterion is needed to determine the final answer; the formula of the rating criterion is as follows:

where m is the interval defined in step 3.1.