CN106126596A - A kind of answering method based on stratification memory network - Google Patents

Abstract

The present invention provides a question answering method based on a hierarchical memory network. Firstly, sentence granularity memory coding is performed, and under the stimulation of question semantic coding, the information reasoning of the sentence granularity memory unit is completed through a multi-round iterative attention mechanism. The maximum sampling is used to screen the sentences, and word-granularity memory encoding is also performed on the basis of sentence-granularity memory encoding, that is, memory encoding is performed at two levels to form a hierarchical memory encoding, and the sentence-granularity and word-granularity memory units are used to jointly predict the output word Probability distribution improves the accuracy of automatic question answering and effectively solves the problem of answer selection for low-frequency words and unregistered words.

Description

A Question Answering Method Based on Hierarchical Memory Network

technical field

The present invention relates to the technical field of building an automatic question answering system, and more particularly relates to an end-to-end question answering method based on a hierarchical memory network.

Background technique

Automatic question answering has long been one of the most challenging tasks in natural language processing, which requires deep understanding of text and screening candidate answers as system responses. The existing traditional methods include: using the pipeline mode to independently train each module in the text processing process, and then merging the output mode; building a large-scale structured knowledge base, and based on this knowledge base for information reasoning and answer prediction. In recent years, end-to-end systems based on deep learning methods have been widely used to solve various tasks, these methods do not need to manually construct features, and do not require individual tuning of each module.

The question answering system can be roughly divided into two steps: firstly locate relevant semantic information, this step is called "activation phase", and then generate a response based on relevant information, this step is called "generation phase". Recently, neural memory network models have achieved good results in question answering system tasks. However, the biggest disadvantage of these models is that they use single-level sentence granular memory units, which cannot solve the problem of low-frequency words or unregistered words well. And usually, in order to reduce the time complexity of the model, it is often necessary to reduce the size of the dictionary. At this time, the existing end-to-end neural network model cannot well select low-frequency or unregistered words as the answer output. That is, when the target answer word is outside the training dictionary, the existing method cannot select the correct answer as the model output in the online test phase. Take the following dialogue text as an example:

1. Hello sir, what's your name?

2. Uh, my name is Williamson.

3. Please tell me your passport number.

4. OK, it's 577838771.

5. And your phone number?

6. The number is 0016178290851.

Assuming that "Williamson", "577838771" and "0016178290851" are low-frequency words or unregistered words, if traditional methods discard these words or replace them with "unk" symbols, these methods cannot select accurate words from the dialogue text. user information. However, in practical applications, most of the answer information comes from low-frequency words or long-tail words. How to design an answer selection method that can effectively solve unregistered words is an urgent task in the field of automatic question answering systems.

Contents of the invention

(1) Technical problems to be solved

In order to solve the problems in the prior art, the present invention provides a question answering method based on a hierarchical memory network.

(2) Technical solution

The present invention provides a question answering method based on a hierarchical memory network, including: Step S101: Fusion word position and sentence time sequence information, carry out sentence granularity memory encoding on the sentences in the sentence set, and obtain the double sentence granularity memory unit Channel memory encoding; step S102: Under the stimulation of question semantic encoding, complete the information reasoning of the sentence granularity memory unit through the attention mechanism of multiple iterations, and obtain the output word probability distribution of the sentence granularity memory unit in the dictionary dimension ; Step S103: Carry out k-maximum sampling on the information reasoning results of the sentence granularity memory unit, and filter out the k-maximum sampling important sentence set from the sentence set; Step S104: Utilize the bidirectional recurrent neural network model to perform Word granularity memory encoding to obtain the memory encoding of the word granularity memory unit; Step S105: Based on the semantic encoding of the problem, the memory encoding of the word granularity memory unit and the k-maximum sampling important sentence set, the word granularity output word probability distribution is obtained through the attention mechanism ; and step S106 : jointly predict the output word probability distribution from the sentence granularity and word granularity memory units, and use cross-entropy for supervised training.

(3) Beneficial effects

It can be seen from the above technical solution that the question answering method based on the hierarchical memory network of the present invention has the following beneficial effects:

(1) The present invention first carries out sentence granular memory coding, and under the stimulation of problem semantic coding, completes the information reasoning of sentence granular memory unit through the attention mechanism of multiple iterations, can improve the accuracy and timeliness of automatic question answering, has Conducive to the answer selection of low-frequency words and unregistered words;

(2) Screening sentences by k-maximum sampling can improve the efficiency of automatic question answering and reduce computational complexity;

(3) On the basis of sentence granular memory coding, word granular memory coding is also carried out, that is, memory coding is carried out at two levels to form a hierarchical memory coding, which can further improve the accuracy of automatic question answering;

(4) When the recurrent neural network is used for word granular memory coding, it is operated on the whole sentence set X. This method can introduce the context semantic information of words in the full sentence set during the word granular memory coding process, which can improve the automatic the accuracy and timeliness of questions and answers;

(5) The attention mechanism on the word granularity memory unit is operated on the subset of word granularity memory units after k sampling, which avoids the interference information in the memory encoding and reduces the calculation amount of the word granularity attention mechanism;

(6) Using sentence granularity and word granularity memory units to jointly predict the output word probability distribution can further improve the accuracy of automatic question answering, and effectively solve the problem of answer selection for low-frequency words and unregistered words.

Description of drawings

FIG. 1 is a flowchart of a question answering method based on a hierarchical memory network according to an embodiment of the present invention;

FIG. 2 is a schematic framework diagram of a question answering method based on a hierarchical memory network according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of sentence granularity memory coding and information reasoning based on sentence granularity memory coding according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of word granularity memory coding and attention activation based on word granularity memory coding according to an embodiment of the present invention;

FIG. 5 is a performance diagram 1 of a question answering method based on a hierarchical memory network according to an embodiment of the present invention;

FIG. 6 is another performance diagram of a question answering method based on a hierarchical memory network according to an embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

The invention discloses a question answering method based on a hierarchical memory network, which is based on an end-to-end model of a full neural network structure, which can realize information reasoning, screening and word granularity selection in a sentence set, and effectively solve question answering under big data The system selects answers to low-frequency words or unregistered words. In the question answering method of the present invention, sentence sets with time series information are respectively subjected to two hierarchical memory encodings, namely: sentence granularity memory encoding and word granularity memory encoding. Then information reasoning, screening and activation are performed based on the hierarchical memory network, and the probability distribution of candidate answer words is jointly predicted.

In the question-answering method of the present invention, sentence vectorized memory encoding is first performed on the sentence set through a hierarchical memory network, considering the position information of words in the sentence and the sequence time information of the sentence in the sentence set, and then through the attention mechanism of multiple rounds of iteration Complete the information reasoning of the sentence granularity memory unit, and perform k-maximum sampling based on the reasoning result to screen out important sentence information. Then use the two-way recurrent network model to encode the word-granularity sequence of the sentence set, and use the attention mechanism to activate the information of the word-granularity memory unit from the screened information, and finally predict the output word probability from the sentence-granularity and word-granularity memory units respectively Distributed and jointly supervised training by Softmax to learn an end-to-end automatic question answering model.

A question answering method based on a hierarchical memory network as an embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

Fig. 1 is the flowchart of the question answering method based on hierarchical memory network of the embodiment of the present invention, with reference to Fig. 1, this question answering method comprises:

Step S101: Fusing word positions and sentence time series information, performing sentence-granularity memory encoding on the sentences in the sentence set to obtain a dual-channel memory encoding of the sentence-granularity memory unit.

Referring to Fig. 3, step S101 includes:

Sub-step S101a: Perform dual-channel word vector mapping on the sentences with time series information in the sentence set to obtain the dual-channel word vectorization encoding of the sentences.

Sub-step S101a includes: Given a set of sentences with time series information X={ _xi } _{i=(1, 2, ..., n)} , where i is the current time series of sentences, and n is the largest sentence set Time series length; randomly initialize two word vector matrices and Among them, | _V | is the dimension of the dictionary, d is the dimension of the word vector, A and C respectively use the normal distribution with a standard deviation of 0.1 and a mean of 0 as random initialization parameters, and perform dual-channel Word vector mapping, then the two-channel vectorization encoding of word x _ij in sentence x _i is and j is the position information of the word in the sentence x _i .

Sub-step S101b: update the dual-channel word vectorization encoding according to the position information of the word in the sentence.

Sub-step S101b includes: generating an update matrix l according to the position information j of the word in the sentence and the dimension d of the word vector, then the updated two-channel word vectorization encoding is l _gj (Ax _ij ) and l _gj (Cx _ij ),in:

l _gj ＝(1-j/J _i )-(g/d)(1-2j/J _i ) (1)

Among them, J _i is the number of words in the sentence x _i , and g is the current dimension value in the word vector with dimension d, and 1≤j≤J and 1≤g≤d.

Sub-step S101c: Integrating the time series information of the sentence to perform sentence-granularity memory encoding on the sentence to obtain the dual-channel memory encoding of the sentence-granularity memory unit.

Substep S101c includes: randomly initializing the vectorized matrix of two sentence time series and Among them, n is the maximum time series length of the sentence set, d is the dimension of the time vector, which is the same as the dimension of the word vector, T _A and T _C respectively adopt the normal distribution with standard deviation of 0.1 and mean value of 0 as random initialization parameters, then The dual-channel memory encoding of the sentence granularity memory unit is M ^(S) = {{a _i }, { _ci }}, where:

${\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Ax</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Cx</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, l _j is the update vector of the jth word in the sentence x _i in the update matrix l, and the operator · is the element multiplication operation between vectors, such as l _j · (Ax _ij ) in the formula (2) represents the vector l _j and Vector(Ax _ij ) performs element-wise multiplication.

Step S102: Under the stimulation of the semantic encoding of the question, the information reasoning of the sentence granular memory unit is completed through the multi-round iterative attention mechanism, and the output word probability distribution of the sentence granular memory unit in the dictionary dimension is obtained.

Step S102 includes:

Sub-step S102a: Perform vectorized representation of the question text to obtain the semantic code of the question.

Sub-step S102a includes: using the word vector matrix Vectorized representation of the jth word q _j in the question text q And based on the position j of the word in the question text, the vectorized representation is updated to obtain the semantic encoding of the question:

${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Same as the formulas (2) and (3), l _j is the update vector of the jth word of the update matrix l in the sentence x _i .

Sub-step S102b: Under the stimulation of the semantic encoding of the question, use the attention mechanism to activate information in the dual-channel memory encoding of the sentence granularity memory unit;

Sub-step S102b includes: using the dot product method to calculate the attention weight of the question semantic code in the sentence granularity memory unit:

${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Then, under the stimulation of question semantic coding, the activation information of the dual-channel memory coding of the sentence granularity memory unit is:

Sub-step S102c: complete the information reasoning in the sentence granular memory unit through the multi-round iterative attention mechanism, and obtain the output word probability distribution in the dictionary dimension on the sentence granular memory unit.

Sub-step S102c includes: performing R rounds of information activation on the sentence granularity memory unit, finding a set of candidate sentences, and obtaining the Rth round of activation information OR , wherein, in the _r +1th round of information activation,

${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; A</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

Among them, 1≤r≤(R-1), the independent word vector matrix A ^r+1 and C ^r+1 and the time vector matrix are used in the r+1 round of information activation and A vectorized representation of a collection of sentences, and C ^r and A normal distribution with a standard deviation of 0.1 and a mean of 0 was used as random initialization parameters.

The information reasoning in the sentence granularity memory unit is completed through the attention mechanism of R rounds of iteration, and the output word probability distribution in the dictionary dimension on the sentence granularity memory unit is obtained as:

${\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}^{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

in, is the set of dictionary dimension words, is the matrix of word embeddings activated for the R-th round information, and T is the transposition operator.

The present invention first performs sentence granularity memory coding, and under the stimulation of question semantic coding, completes the information reasoning of sentence granularity memory units through the attention mechanism of multiple rounds of iterations, which can improve the accuracy and timeliness of automatic question and answer, and is beneficial to low-frequency words and unregistered word answer choices.

Step S103: Perform k-maximum sampling on the information reasoning results of the sentence-granularity memory unit, and select the k-maximum-sampling important sentence set from the sentence set.

Step S103 includes:

Sub-step S103a: the attention weight vector activated on the R-th round of information on the sentence granularity memory unit Select the k largest attention weight sub-sets from it

Sub-step S103b: Select the k largest attention weight sub-sets The corresponding sentence set is used as the k-maximum sampling important sentence set Sentences in the set of important sentences for important sentences.

The k-maximum sampling of the present invention screens sentences, which can improve the efficiency of automatic question answering, reduce computational complexity, and be more conducive to the answer selection of low-frequency words and unregistered words.

Step S104: use the bidirectional recurrent neural network model to perform word-granularity memory encoding on the sentence set, and obtain the memory encoding of the word-granularity memory unit.

Referring to FIG. 4, step S104 includes:

Sub-step S104a: use the bidirectional recurrent network model to encode the words in the important sentence set in time series, and obtain the hidden state of the bidirectional recurrent network model. There are many existing bidirectional recurrent network models, one of which is adopted in this embodiment: the gated recurrent network model (GRU).

Sub-step S104a includes: using the Gated Recurrent Network Model (GRU) to respectively classify all words in the sentence set X Forward and reverse encoding according to time series, for word features at time t, the hidden state of forward GRU encoding is The hidden state of the backward GRU encoding is Among them, |t| is the maximum sequence length of words after arranging all the words in the sentence set X according to the time sequence, and The dimension of is the same as the dimension d of the word vector, and C ^R is the word vector matrix during the R-th round of information activation in the sentence granularity memory unit.

Sub-step S104b: Fusion the hidden state of the bidirectional recurrent network model to obtain the memory encoding of the word granularity memory unit.

Sub-step S104b includes: adding the hidden states of the two-way recurrent network model directly to obtain the memory encoding M ^(W) ={m _t } _{t=1, 2, ..|t|)} of the memory unit at the word granularity, where

When the present invention utilizes the recurrent neural network to perform word granularity memory encoding, it operates on the entire sentence set X. This method can introduce the context and semantic information of words in the full set of sentences during the word granularity memory encoding process, which can improve automatic question answering. The accuracy and timeliness of the method are beneficial to the answer selection of low-frequency words and unregistered words.

Step S105: Based on the semantic encoding of the question, the memory encoding of the memory unit at the word granularity, and the k-maximum sampling important sentence set, the output word probability distribution at the word granularity is obtained through the attention mechanism.

Step S105 includes:

Sub-step S105a: Calculate the attention weight of the word-granularity memory unit according to the semantic code of the question and the memory code of the word-granularity memory unit;

Sub-step S105a includes: based on the semantic encoding of the question during the R-th round of information activation on the sentence granularity memory unit Memory encoding M ^(W) = {m _t } _{t = 1, 2, .., |t|)} and k largest sampling important sentence set of word granularity memory unit Obtain the attention weight vector of the normalized word granularity memory unit in:

${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; tanh</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Wu</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; m</span>}}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

in, is the k-maximally sampled set of important sentences set of words in The corresponding word-granularity memory code M ^(W) = {m _t } _{t = (1, 2, ..., |t|)} in the subset The dimension of the attention weight vector α ^(W) and the set of important sentences according to the time series all words in The maximum sequence length of the arranged words is the same, that is, and All are learning parameters, v, W and U are randomly initialized with a normal distribution with a standard deviation of 0.1 and a mean of 0, and are updated during the training phase.

Sub-step S105b: Obtain the word granularity output word probability distribution according to the attention weight of the word granularity memory unit. In an embodiment of the present invention, the attention weight α ^(W) of the normalized word granularity memory unit is directly adopted as the word granularity output word probability distribution:

${\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; w</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

At this time, the word granularity output word probability distribution is consistent with the dimension of attention weight, namely is the set of all words in the important sentence set

The present invention also performs word granularity memory coding on the basis of sentence granularity memory coding, that is, performs memory coding at two levels to form hierarchical memory coding, which can further improve the accuracy of automatic question and answer, and is more conducive to low-frequency words and unregistered words. word answer choice. At the same time, the attention mechanism on the word granularity memory unit is operated on the subset of word granularity memory units after k sampling, which avoids the interference information in the memory encoding and reduces the calculation amount of the word granularity attention mechanism.

Step S106: Jointly predict the output word probability distribution from the sentence-granularity and word-granularity memory units, and use cross-entropy for supervised training.

Step S106 includes:

Sub-step S106a: Based on the output word probability distribution in the dictionary dimension and the word granularity output word probability distribution on the sentence granularity memory unit, the output word joint prediction is performed, and the expression of the joint prediction output word distribution p(w) is:

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span>\\ <span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; w</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 14</span>}<span\; class="notranslate">\; )</span>$

Among them, trans( ) indicates that the word granularity of the sub-set is output to the word probability distribution Word granularity output word probability distribution mapped to the full set of dictionary dimensions The mapping operation specifically refers to the output word probability distribution The medium probability value according to its corresponding word sub-set The words in the dictionary dimension word set The position in the probability value mapping, if some words in the full set do not appear in the sub-set, the output probability is set to 0, and the output probability distribution of the word after mapping is obtained

Sub-step S106b: Perform cross-entropy supervised training on the joint prediction output word distribution by using the target answer word distribution. Given that the target answer word distribution of the training set is y, the joint optimization is performed based on the cross-entropy function of the target answer word distribution y and the joint prediction output word distribution p(w).

In an exemplary embodiment of the present invention, the stochastic gradient descent method is used to carry out error backpropagation to optimize the objective function in the joint optimization, and the optimization parameters include the word vector matrix {A ^r } _{r=(1 , 2,...,R)} and {C ^r } _{t=1, 2,...,R)} and time vector matrix and All parameter sets {θ _GRU } of the two-way GRU model used in the encoding process of word granularity memory and v, W and U in calculating the attention weight (formula (12)) of the word granularity memory unit.

The present invention jointly predicts the output word probability distribution in the sentence granularity and word granularity memory units, can further improve the accuracy of automatic question answering, and is more conducive to the answer selection of low-frequency words and unregistered words.

FIG. 2 is a schematic framework diagram of a question answering method based on a hierarchical memory network as an embodiment of the present invention. Referring to Figure 2, the question answering method based on the hierarchical memory network has two levels of memory network units, which are:

Memory unit 1: Sentence sets are coded and memorized at sentence granularity in time series;

Memory unit 2: All words in the sentence set are coded and memorized according to the time sequence of word granularity.

Between different memory unit levels, k-maximum adoption is used to screen and filter important information.

There are two information activation mechanisms in the model information processing stage, which are:

Activation mechanism 1: Use the reasoning mechanism to activate information on the sentence granularity memory unit;

Activation mechanism 2: Use the attention mechanism on the word granularity memory unit for word selection.

During the entire model training phase, there are two supervisory information for guidance, namely:

Supervision information 1: the output vector after the information reasoning of the sentence granularity memory unit is decoded and the fitting information of the target word is output by Softmax;

Supervision information 2: the fitting information of the target word after the attention mechanism activation and Softmax output of the word granular memory unit.

In order to accurately evaluate the automatic question and answer response performance of the method of the present invention, the present invention compares the performance of the method of the present invention by comparing the number of error samples in which the answer words output by the model selection and the real answer words of the data are inconsistent.

Table 1

data field train/test question-answer pairs Dictionary size (all/train/test) Unregistered target words (percentage) pre-book fly ticket 7,000/7,000 10,682/5,612/5,618 5,070 (72.43%)

In the experiment of the present invention, a text data set in the field of Chinese airline ticket booking is adopted, which contains 2,000 complete dialogue histories and 14,000 question-answer pairs, which are divided into training set and test set 5:5. For these text data sets, the present invention does not do any processing (including operations such as removing stop words and word stem restoration). The specific statistical information of the data set is shown in Table 1. It can be seen that the unregistered target words in the test set account for 72.43%, which will have a relatively large impact on traditional model training.

Adopt following comparative method in the experiment of the present invention:

Comparison method 1: Pointer network model based on attention mechanism, which encodes all words in the sentence set as a long sentence according to time sequence, and directly uses the attention mechanism of question and word encoding to generate answers;

Contrast method 2: Neural memory network model, this method encodes sentence sets at sentence granularity, and uses the encoding vector of the question for semantically activated information to directly match the answer in the full dictionary space.

Adopt parameter setting in the experiment of the present invention as shown in table 2:

Table 2

no d R k lr bs 16 100 3 1 0.01 10

In Table 2, the parameter n is the maximum sentence time series of the sentence set of the experimental data, d is the word vector dimension and the hidden layer coding dimension, R is the number of iterations of the reasoning mechanism on the sentence granularity memory unit, and k is the maximum The number of samples, lr is the learning rate when the stochastic gradient descent method is used to optimize the model parameters, and bs is the number of samples in each batch when the model is trained.

In the experiment of the present invention, 15 rounds of iterative training are carried out, and all methods converge as shown in Figure 5, and the experimental results after the final convergence are shown in Table 3:

table 3

method error samples Comparison method one 109 Comparison method two 56 The method of the invention 0

Figure 5 and Table 3 are the evaluation results of the number of error samples on the data set for the method of the present invention, the first comparison method and the second comparison method. Experimental results show that the convergence speed of the method of the present invention is significantly superior to other methods. And according to the final convergence results in Table 3, it can be seen that the method of the present invention is obviously superior to other methods, and can completely solve the problem of answer selection on unregistered word sets, reaching a 100% correct rate.

At the same time, the experiment of the present invention verifies the performance impact of the maximum number of samples k for information screening among hierarchical memory units on the number of wrong samples in answer selection questions, and the experimental results are shown in Figure 6 and Table 4. It can be seen that when the maximum number of samples is 1, the convergence speed and final convergence result of the performance of the method of the present invention can be optimal, which further illustrates the importance of information selection among hierarchical memory units.

Table 4

Maximum number of samples error samples 3 5 2 4 1 0

So far, the embodiments of the present invention have been described in detail with reference to the accompanying drawings. Based on the above description, those skilled in the art should have a clear understanding of the question answering method based on the hierarchical memory network of the present invention.

A question-and-answer method based on a hierarchical memory network of the present invention first performs sentence-granularity memory encoding, and under the stimulation of question semantic encoding, completes the information reasoning of sentence-granularity memory units through a multi-round iterative attention mechanism, which can improve automatic question-answering The accuracy and timeliness of the method are beneficial to the answer selection of low-frequency words and unregistered words; and the k-maximum sampling is used to screen sentences, which can improve the efficiency of automatic question answering and reduce computational complexity. On the basis of sentence granularity memory coding, it can also perform Word-granular memory encoding, that is, memory encoding at two levels to form a hierarchical memory encoding, which can further improve the accuracy of automatic question and answer; when using a recurrent neural network for word-granular memory encoding, it is operated on the entire sentence set X , this method can introduce the context semantic information of words in the full set of sentences in the process of word granular memory encoding, which can improve the accuracy and timeliness of automatic question answering; the attention mechanism on the word granular memory unit is after k sampling The operation on the sub-set of word granularity memory unit avoids the interference information in the memory encoding and reduces the calculation amount of the word granularity attention mechanism; using the sentence granularity and word granularity memory unit to jointly predict the output word probability distribution can further improve the automatic The accuracy of the question and answer effectively solves the problem of answer selection for low-frequency words and unregistered words.

It should be noted that, in the accompanying drawings or in the text of the specification, implementations that are not shown or described are forms known to those of ordinary skill in the art, and are not described in detail. In addition, the above definitions of each element are not limited to the various methods mentioned in the embodiments, and those of ordinary skill in the art can easily modify or replace them, for example:

(1) The directional terms mentioned in the embodiments, such as "up", "down", "front", "back", "left", "right", etc., are only referring to the directions of the drawings, and are not used to limit The protection scope of the present invention;

(2) The above embodiments can be mixed and matched with each other or with other embodiments based on design and reliability considerations, that is, technical features in different embodiments can be freely combined to form more embodiments.

The specific embodiments described above have described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the present invention. Within the spirit and principles of the present invention, any modifications, equivalent replacements, improvements, etc., shall be included in the protection scope of the present invention.

Claims (10)

1. A question answering method based on a hierarchical memory network, characterized in that, comprising: Step S101: Fusing the position of the word and the time series information of the sentence, performing sentence-granularity memory encoding on the sentences in the sentence set, and obtaining the dual-channel memory encoding of the sentence-granularity memory unit; Step S102: Under the stimulation of the semantic encoding of the question, complete the information reasoning of the sentence granular memory unit through the attention mechanism of multiple iterations, and obtain the output word probability distribution of the sentence granular memory unit in the dictionary dimension; Step S103: performing k-maximum sampling on the information reasoning results of the sentence-granularity memory unit, and selecting the k-maximum-sampling important sentence set from the sentence set; Step S104: using a bidirectional recurrent neural network model to perform word-granularity memory encoding on the sentence set to obtain memory encoding of word-granularity memory units; Step S105: Based on the semantic encoding of the problem, the memory encoding of the word granularity memory unit and the k-maximum sampling important sentence set, the word granularity output word probability distribution is obtained through the attention mechanism; and Step S106: Jointly predict the output word probability distribution from the sentence-granularity and word-granularity memory units, and use cross-entropy for supervised training. 2. The question answering method according to claim 1, characterized in that said step S101 comprises: Sub-step S101a: Given a sentence set X={ _xi } _{i=(1, 2,...,n)} with time series information, randomly initialize the word vector matrix and The two-channel vectorization encoding of word x _ij in sentence x _i is and Among them, i is the current time series of the sentence; n is the maximum time series length of the sentence set; |V| is the dictionary dimension; d is the dimension of the word vector; j is the position information of the word in the sentence x _i ; Sub-step S101b: update the dual-channel word vectorization encoding according to the position information of the word in the sentence; and Sub-step S101c: Integrating the time series information of the sentence to perform sentence-granularity memory encoding on the sentence to obtain the dual-channel memory encoding of the sentence-granularity memory unit. 3. The question answering method according to claim 2, characterized in that, said substep S101b comprises: The updated two-channel word vectorization is encoded as l _gj (Ax _ij ) and l _gj (Cx _ij ), where, l _gj ＝(1-j/J _i )-(g/d)(1-2j/J _i ) (1) Among them, J _i is the number of words in the sentence x _i , and g is the current dimension value in the word vector with dimension d, and 1≤j≤J and 1≤g≤d. 4. The question answering method according to claim 3, characterized in that, said sub-step S101c comprises: Randomly initialize the matrix of time vectors for sentences and Then the dual-channel memory encoding of the sentence granularity memory unit is M ^(S) = {{a _i }, { _ci }}, where, a _i ＝∑ _j l _j ·(Ax _ij )+T _A (i) (2) c _i ＝∑ _j l _j ·(Cx _ij )+T _C (i) (3) Among them, l _j is the update vector of the jth word in the sentence x _i in the update matrix l; the operator is the element multiplication operation between vectors; n is the maximum time series length of the sentence set; d is the time vector dimension, and the word The vectors have the same dimensions. 5. The question answering method according to claim 4, characterized in that said step S102 comprises: Sub-step S102a: use word vector matrix Vectorized representation of the jth word q _j in the question text q Get the problem semantic encoding: ${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$ Among them, l _j is the update vector of the jth word in the sentence x _i of the update matrix l; Sub-step S102b: Calculating the attention weight of the question semantic code in the sentence granularity memory unit ${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$ Under the stimulation of question semantic encoding, the activation information of the dual-channel memory encoding of the sentence granularity memory unit is: as well as Sub-step S102c: complete the information reasoning in the sentence granular memory unit through the multi-round iterative attention mechanism, and obtain the output word probability distribution in the dictionary dimension on the sentence granular memory unit. 6. The question answering method according to claim 5, wherein said sub-step S102c comprises: Perform R rounds of information activation on the sentence granularity memory unit to obtain the Rth round of activation information OR , where, in the _r +1th round of information activation, ${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; A</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$ Among them, 1≤r≤(R-1); A ^r+1 ＝C ^r , The output word probability distribution in the dictionary dimension on the sentence granularity memory unit is: ${\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}^{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$ Wherein, w={w _t } _{t=(1, 2, ..., |V|)} is a dictionary dimension word set; The word vector matrix activated for the R-th round of information; T is the transpose operator. 7. The question answering method according to claim 6, characterized in that said step S103 comprises: Sub-step S103a: the attention weight vector activated on the R-th round of information on the sentence granularity memory unit Select the k largest attention weight sub-sets from it as well as Sub-step S103b: Select the k largest attention weight sub-sets The corresponding sentence set is used as the k-maximum sampling important sentence set 8. The question answering method according to claim 7, wherein said step S104 comprises: Sub-step S104a: Use the gated recurrent network model to classify all the words in the sentence set X Forward and reverse encoding according to time series, for word features at time t, the hidden state of forward GRU encoding is The hidden state of the backward GRU encoding is Among them, |t| is the maximum sequence length of words after arranging all the words in the sentence set X according to the time sequence; and The dimension of is the same as the dimension d of the word vector; Sub-step S104b: Obtain the memory code M ^(W) ={m _t } _{t=(1, 2, ..., |t|)} of the word granularity memory unit, where 9. The question answering method according to claim 8, characterized in that said step S105 comprises: Sub-step S105a: Calculating the attention weight vector of the normalized word granularity memory unit in: ${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; tanh</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Wu</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; m</span>}}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$ in, is the k-maximally sampled set of important sentences set of words in The corresponding word-granularity memory code M ^(W) = {m _t } _{t = (1, 2, ..., |t|)} in the subset The dimension of the attention weight vector α ^(W) is and is the learning parameter; Sub-step S105b: word granularity output word probability distribution for: ${\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; w</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$ in, is the set of all words in the important sentence set 10. The question answering method according to claim 9, wherein said step S106 comprises: Sub-step S106a: Based on the output word probability distribution in the dictionary dimension and the word granularity output word probability distribution on the sentence granularity memory unit, the output word joint prediction is performed, and the expression of the joint prediction output word distribution p(w) is: $\begin{array}{c}\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span>\\ <span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; w</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 14</span>}<span\; class="notranslate">\; )</span>$ Among them, trans( ) indicates that the word granularity of the sub-set is output to the word probability distribution Word granularity output word probability distribution mapped to the full set of dictionary dimensions Sub-step S106b: Perform cross-entropy supervised training on the joint prediction output word distribution by using the target answer word distribution.