CN109740158A - Text semantic parsing method and device - Google Patents
Text semantic parsing method and device Download PDFInfo
- Publication number
- CN109740158A CN109740158A CN201811644576.6A CN201811644576A CN109740158A CN 109740158 A CN109740158 A CN 109740158A CN 201811644576 A CN201811644576 A CN 201811644576A CN 109740158 A CN109740158 A CN 109740158A
- Authority
- CN
- China
- Prior art keywords
- vector
- text
- attention
- coding
- given
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 239000013598 vector Substances 0.000 claims abstract description 368
- 238000004364 calculation method Methods 0.000 claims abstract description 42
- 239000011159 matrix material Substances 0.000 claims description 70
- 230000017105 transposition Effects 0.000 claims description 27
- 238000004458 analytical method Methods 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 19
- 238000013527 convolutional neural network Methods 0.000 claims description 15
- 230000002708 enhancing effect Effects 0.000 claims description 14
- 230000002452 interceptive effect Effects 0.000 claims description 12
- 238000003780 insertion Methods 0.000 claims description 11
- 230000037431 insertion Effects 0.000 claims description 11
- 239000000284 extract Substances 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 3
- 235000013399 edible fruits Nutrition 0.000 claims description 2
- 230000001537 neural effect Effects 0.000 claims 1
- 230000008569 process Effects 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000003058 natural language processing Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 239000004615 ingredient Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 229940050561 matrix product Drugs 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Machine Translation (AREA)
Abstract
The application discloses a text semantic parsing method and a text semantic parsing device, wherein the method comprises the following steps: acquiring vector representation of a given text, and generating a coding vector of the given text according to the vector representation; the given text comprises a first text and a second text; the encoded vectors include a first encoded vector and a second encoded vector; generating a first attention and a second attention according to the first encoding vector and the second encoding vector, and generating a global information vector according to the first attention and the second attention; obtaining a semantic enhancement vector from the global information vector; and analyzing a target text corresponding to the second text in the first text according to the semantic enhancement vector. Compared with the prior art, the technical scheme of the application enriches the semantic representation of the vector to the text, accelerates the calculation convergence speed, and improves the accuracy and the efficiency by acquiring the semantic enhancement vector from the global information vector and resolving the target text corresponding to the second text in the first text according to the semantic enhancement vector.
Description
Technical field
This application involves natural language processing technique field more particularly to a kind of text semantic analysis method and devices.
Background technique
Machine reads understand it is current manual's intelligence hot spot research direction, and machine reads understanding task, refers to given one
Section chapter sentence (context) and a corresponding query statement (query), then machine is by reading chapter sentence and inquiry
Sentence parses corresponding answer.
In the prior art, the corresponding answer of query statement is determined by the semanteme of parsing chapter sentence and query statement.
Wherein, the answer need to be the one section of word (it can be appreciated that continuous several words) that can be found in chapter sentence, and
Reading the final goal understood is to determine two subscripts, corresponds respectively to starting position and stop bits of the answer in chapter sentence
It sets.The semantic reading with the corresponding answer of determination of parsing understands task, usually understands mould by the machine reading based on deep learning
Type is realized.The frame of this class model is essentially identical, specifically include that embedded coding module, context-query interactive module and
Export prediction module.Wherein, embedded coding module is used to given text representation be the acceptable term vector of model, context-
Query interactive module is used to give the semantic feature of text, and output prediction module is for exporting the position of answer in the text.
However, existing semanteme analytic method accuracy is lower, calculating process convergence rate is slower, this is simultaneously but also base
Understand performance that model shows also still wait advanced optimize and promoted in the reading of said frame.
Summary of the invention
This application provides a kind of text semantic analysis method and devices, to solve semantic analytic method in the prior art
The low problem of accuracy rate.
In a first aspect, this application provides a kind of text semantic analytic methods, this method comprises:
The vector for obtaining given text indicates, the coding vector for generating the given text is indicated according to the vector；Institute
Stating given text includes the first text and the second text；The coding vector include corresponding first coding vector of the first text and
Corresponding second coding vector of second text；
The first attention and the second attention are generated according to first coding vector and the second coding vector, and according to institute
It states the first attention and the second attention generates global information vector；
Semantically enhancement vector is obtained from the global information vector；
According to the semantically enhancement vector, target corresponding with second text text in first text is parsed
This.
Second aspect, present invention also provides a kind of text semantic analytic methods, this method comprises:
The vector for obtaining the given text indicates；The given text includes the first text and the second text；It is described to
Amount indicates to include that the corresponding primary vector of the first text indicates that secondary vector corresponding with the second text indicates；
It indicates from the primary vector respectively and obtains first the second semanteme of semantically enhancement vector sum in secondary vector expression
Enhance vector；
The first attention and the second attention are generated according to the first semantically enhancement vector sum the second semantically enhancement vector,
And global information vector is generated according to first attention and the second attention；
According to the global information vector, target corresponding with second text text in first text is parsed
This.
The third aspect, this application provides a kind of text semantic resolver, which includes:
Insertion and coding module, the vector for obtaining given text indicate, give according to vector expression generation
Determine the coding vector of text；The given text includes the first text and the second text；The coding vector includes the first text
Corresponding first coding vector and corresponding second coding vector of the second text；
Interactive module, for generating the first attention and the second note according to first coding vector and the second coding vector
Meaning power, and global information vector is generated according to first attention and the second attention；
Semantically enhancement module, for obtaining semantically enhancement vector from the global information vector；
Parsing module, for according to the semantically enhancement vector, parse in first text with second text
This corresponding target text.
Fourth aspect, present invention also provides a kind of text semantic resolver, which includes:
It is embedded in module, the vector for obtaining the given text indicates；The given text includes the first text and the
Two texts；The vector indicates to include that the corresponding primary vector of the first text indicates secondary vector table corresponding with the second text
Show；
Semantically enhancement module, for obtaining the first semantic increasing from primary vector expression and secondary vector expression respectively
Dominant vector and the second semantically enhancement vector；
Coding and interactive module, for generating first according to the first semantically enhancement vector sum the second semantically enhancement vector
Attention and the second attention, and global information vector is generated according to first attention and the second attention；
Parsing module, for according to the global information vector, parse in first text with second text
This corresponding target text.
From the above technical scheme, the embodiment of the present application provides a kind of text semantic analysis method and device, wherein
Method includes: to obtain the vector expression of given text, and the coding vector for generating given text is indicated according to vector；Given text packet
Include the first text and the second text；Coding vector includes corresponding first coding vector of the first text and the second text corresponding the
Two coding vectors；The first attention and the second attention are generated according to the first coding vector and the second coding vector, and according to the
One attention and the second attention generate global information vector；Semantically enhancement vector is obtained from global information vector；According to language
Justice enhancing vector, parses the target text corresponding with the second text in the first text.Compared with prior art, the application skill
Art scheme is gone out in the first text by obtaining semantically enhancement vector from global information vector according to semantically enhancement vector analysis
Target text corresponding with the second text, enrich vector to the semantic expressiveness of text, accelerate to calculate convergence rate, accuracy rate
It is improved with efficiency.
Detailed description of the invention
In order to illustrate more clearly of the technical solution of the application, letter will be made to attached drawing needed in the embodiment below
Singly introduce, it should be apparent that, for those of ordinary skills, without any creative labor,
It is also possible to obtain other drawings based on these drawings.
Fig. 1 is one embodiment method flow diagram of the application text semantic analytic method；
Fig. 2 is a kind of implementation flow chart of the application embodiment illustrated in fig. 1 step 110；
Fig. 3 is another implementation flow chart of the application embodiment illustrated in fig. 1 step 110；
Fig. 4 (a) is that the application one implements the convolutional calculation process schematic exemplified；
Fig. 4 (b) is that the application one implements the convolutional calculation process schematic exemplified；
Fig. 5 is that the application one implements the transposition convolutional calculation process schematic exemplified；
Fig. 6 is another embodiment method flow diagram of the application text semantic analytic method；
Fig. 7 is one embodiment schematic diagram of the application text semantic resolver；
Fig. 8 is the refinement block diagram that the application one implements the text semantic resolver exemplified；
Fig. 9 is the refinement block diagram that the application one implements the text semantic resolver exemplified；
Figure 10 is another embodiment schematic diagram of the application text semantic resolver.
Specific embodiment
Technical solution in order to enable those skilled in the art to better understand the present invention, below in conjunction with of the invention real
The attached drawing in example is applied, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described implementation
Example is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, this field is common
Technical staff's every other embodiment obtained without making creative work, all should belong to protection of the present invention
Range.
The scheme of the application for ease of understanding, below first to convolution, transposition convolution, the attention mechanism etc. in neural network
Several related notions are briefly introduced.
The input of natural language processing task is a Duan Wenben.Text is by indicating generator matrix, each column pair of matrix
Ying Yuyi element, generally a word, indicate the vector of a word.The columns of weight matrix is usually and term vector length
It is identical, guarantee that the term vector for participating in convolution operation is a complete expression.
Convolutional neural networks (Convolutional Neural Networks, CNN) have powerful extraction local feature
Ability.Convolution (Convolution) calculating process sees Fig. 4 (a), it is assumed that the convolution kernel (weight for the use of size being 3 × 3
Matrix) convolutional calculation, stride 1, the then square that convolutional calculation result is one 2 × 2 are carried out for 4 × 4 matrix to a size
Battle array.Also the mode of matrix multiplication can be used to indicate for convolution operation, refering to Fig. 4 (b), it is necessary first to by aforementioned 3 × 3 convolution kernel
It is rearranged into 4 × 16 convolution matrix, wherein each row represents a convolution kernel；Aforementioned 4 × 4 matrix is rearranged into one
The column of 1 dimension；Then 4 × 16 convolution matrix is multiplied with the column of 1 dimension, the matrix that output is one 2 × 2.Wherein, stride indicates
Convolution kernel translates primary distance in sliding window.Further, it is possible to use multiple convolution kernels are to extract different information.
Transposition convolution (Transposed Convolution) is also referred to as deconvolution (Deconvolution) and spans
Convolution (Fractionally-strided Convolution), in order to avoid causing unnecessary misunderstanding, is hereafter all referred to as
Transposition convolution.Transposition convolution can be considered as a kind of method for obtaining up-sampling, compared to conventional method such as arest neighbors interpolation
(Nearest neighbor interpolation), bilinear interpolation (Bi-Linear interpolation), double cubes insert
It is worth (Bi-Cubic interpolation) etc., is more preferably up-sampled by training transposition convolution acquisition.This method will not
Using interpolation method predetermined, it has the parameter that can learn.
Transposition convolution is one kind of convolution, and the back-propagation process of calculation and convolution is essentially identical.Refering to Fig. 5, such as
Above-mentioned 2 × 2 matrix is changed into 4 × 4 matrix by fruit, needs the matrix using one 16 × 4.At this point, can be by convolution matrix
Transposition (16 × 4) be multiplied with the column vector form (4 × 1) of convolution results, one 16 × 1 matrix of output is further arranged
4 × 4 form is arranged into, convolutional calculation result is just made to be restored to the same structure of input matrix of convolutional calculation.In natural language
In processing, transposition convolution operation and convolution operation are substantially similar, and weight matrix line number is usually identical as term vector length.
Attention (Attention) mechanism: attention mechanism obtains inspiration from human visual attention's mechanism.Mankind's view
Feel when perceiving thing, often observation pays attention to specific a part according to demand.Machine learning method uses for reference the thought,
Intermediate portions in input content are learnt, and attention is put on the part when there is similar scene again in the future.
Attention is to be applied to image domains earliest.With the development of deep learning, work of the attention mechanism in natural language processing
With being increasingly taken seriously, the frame of a large amount of newest deep learnings all introduces attention mechanism, and achieves good performance
Index.Attention mechanism is now widely used for text classification, machine translation, reads in the models such as understanding.
Fig. 1 is one embodiment of the application text semantic analytic method.As shown in Figure 1, this method may include as follows
Step:
Step 110, the vector for obtaining given text indicates, the coding for generating the given text is indicated according to the vector
Vector；The given text includes the first text and the second text；The coding vector includes that the first text corresponding first is compiled
Code vector and corresponding second coding vector of the second text；
First text can be chapter sentence (context), and the first text includes several vocabulary and symbol.Second text can
Think query statement (query), the second text includes several vocabulary and symbol.
Fig. 2 is a kind of implementation of step 110.As shown in Fig. 2, step 110 may include:
Step 111, the given text is handled, the vector for obtaining given text indicates.
The vector of given text indicates that usable word, word correspond to vector and other semantic informations, such as syntax, part of speech,
In a kind of selectable embodiment, word vector can be used, term vector obtains the vector of given text indicates:
First text and the second text are segmented to obtain word finder, for example, the first text or the second text are one
Sentence comprising n vocabulary obtains the word finder { w of n dimension after participle1, w2..., wn, wherein wiRepresent a vocabulary.So
Afterwards, on the one hand, search the corresponding index of each vocabulary in word finder in default vocabulary, word finder is converted into one according to index
A n-dimensional vector { idw1, idw2..., idwn, wherein idwiRepresent vocabulary wiCorresponding index, further according to the index in n-dimensional vector
Corresponding term vector is searched, a n × d is obtained1The term vector matrix of size, wherein d1Indicate term vector dimension.On the other hand,
A point word is carried out to each vocabulary each in word finder, the element for obtaining each position is the word collection of word:
Wherein ci-jIndicate vocabulary wiJ-th of word, the maximum value for the number of words that m includes by a vocabulary；Again in predetermined word
Word is searched in table and concentrates the corresponding index of each word, and Jiang Ziji is converted into n × m × d2The matrix of size；One is carried out to the matrix
Secondary convolutional calculation exports a n × d2The word vector matrix of size.Wherein, d2Indicate the dimension of word vector.
Term vector matrix and word vector matrix are spliced, the vector for obtaining given text indicates, including the first text is corresponding
Vector indicate the expression of corresponding with the second text vector.
Step 112, point of addition encoded information, which obtains the input matrix of convolutional neural networks, to be indicated to the vector, with benefit
Local message vector is extracted from input matrix with convolutional neural networks.
It is position encoded to the vocabulary progress of given text, obtain position encoded matrix.Given text is carried out position encoded
Purpose be to make neural network obtain the opposite or absolute location information between the different participles of given text.It compiles position
The dimension of code vector is equal to the dimension of above-mentioned spliced matrix, so that position encoded vector be enable to be added with it.It compiles position
Code vector can obtain simultaneously training in a model with random initializtion, or be generated by SIN function or cosine function position encoded
Vector.
In a kind of selectable embodiment, it is position encoded to the progress of given text that following formula can be used:
Wherein, pos indicates position of the participle in given text, d1Indicating the dimension of term vector, C is periodic coefficient,
PE(pos, 2i)Indicate position encoded, the PE of the 2i dimension of the participle of os position of pth(pos, 2i+1)Indicate os position of pth
Participle the 2i+1 dimension it is position encoded.
It include the position of the term vector ingredient, word vector component and word of given text in corresponding text in input matrix
Ingredient is encoded, therefore, even if being directed to identical vocabulary, also due to the difference of their positions in the text, and extract difference
Local message vector, avoid the loss of sequence of words information in text, the information for extracting convolution kernel is more accurate.
Step 113, the attention force information of given text is obtained according to local message vector.
In the application, it is based on attention mechanism, the attention force information of given text is calculated using Attention function.
Step 114, it is weighted according to attention force information portion's information vector of playing a game, generates the volume of the given text
Code vector.
Fig. 3 is another implementation of step 110.It is different from implementation shown in Fig. 2, it is in this implementation, right
The term vector matrix and word vector matrix of given text have carried out semantically enhancement processing.As shown in figure 3, step 110 may include:
Step 115, the given text is handled, the vector for obtaining given text indicates.
Step 116, preliminary enhancing matrix is obtained from the expression of the vector of given text.
The mode for obtaining preliminary enhancing matrix can be with are as follows: indicates the vector of given text to carry out convolutional calculation first, so
Transposition convolutional calculation is carried out to convolutional calculation result afterwards, obtains tentatively enhancing matrix, so that tentatively enhancing matrix and given text
Vector indicate structure it is identical.Alternatively, aforementioned convolution calculated result and transposition convolutional calculation result are spliced, preliminary enhancing is formed
Matrix.
It in step 117, similarly with abovementioned steps 112, is added by matrix, preliminary enhancing matrix point of addition is compiled
Code information, obtains the input matrix of convolutional neural networks, and convolutional neural networks is recycled to extract part from the input matrix
Information vector.
In step 118, the attention force information of the given text is obtained according to the local message vector；And step
119, the coding vector of the given text is generated according to the attention force information.
Two kinds of implementations difference of step 110 is, in the second implementation, indicates the vector of given text
Semantically enhancement processing has been carried out, the semantic expressiveness of vector is enriched, has improved the accuracy of semantic parsing.
Step 120, the first attention and the second attention, and root are generated according to the first coding vector and the second coding vector
Global information vector is generated according to the first attention and the second attention.
First attention characterizes the attention of the first text to the second text, and the second attention characterizes the second text to first
The attention of text.
In step 120 in the specific implementation, first according to first coding vector and the generation of the second coding vector
The attention matrix of first text and the second text.It specifically can use following similarity function and generate the first text and the second text
This attention matrix:
Wherein, StjIndicate the attention force value between t-th of chapter sentence word and j-th of query statement word, CtIt indicates
T-th of column vector of chapter sentence, QjIndicating j-th of column vector of query statement, ⊙ indicates to press element multiplication, [；] indicate to
Measure the splicing on being expert at；Expression can training parameter.
Then the row and column of the attention matrix is normalized respectively using normalization exponential function, is obtained
Row processing result S ' and column processing result S ".
Again by the transposed vector Q of row processing result S ' and the second coding vectorTMultiplication obtains the first attention A, A=S ' QT；
By row processing result S ', the transposition S " of column processing resultTAnd first coding vector transposition CTMultiplication obtains the second attention B,
B=S ' S "TCT。
Global information vector [C is finally generated according to the first attention A and the second attention B；A；C⊙A；C⊙B].
Step 130, semantically enhancement vector is obtained from global information vector.
The mode that semantically enhancement vector is obtained from global information vector can be with are as follows: first with convolution kernel to global information
Vector carries out convolutional calculation；Then transposition convolutional calculation is carried out to convolutional calculation result, obtained and global information vector structure phase
Same semantically enhancement vector.Alternatively, the centre that the convolutional calculation result of global information vector and transposition convolutional calculation are obtained to
Amount splicing, obtains semantically enhancement vector.By this implementation, the semantic information for enriching given text is indicated, is improved semantic
Parse accuracy rate.
It is no longer superfluous herein about step 130 referring also to the specific implementation for obtaining preliminary enhancing matrix in step 110
It states.
Step 140, according to semantically enhancement vector, the target text corresponding with the second text in the first text is parsed.
In the specific implementation, can use model based coding layer (model encoder layer) and the output of QAnet model
Layer, parses the target text corresponding with second text in first text.
Using semantically enhancement vector as the input of model based coding layer, wherein model based coding layer includes 3 models from bottom to top
Coding module, output are respectively M0, M1And M2.Make M0, M1And M2As the input of output layer, predict respectively every in the first text
A lexical position is the end point that each lexical position is target text in the probability and the first text of the starting point of target text
Probability, to parse target text corresponding with the second text in the first text.
From the above technical scheme, the embodiment of the present application provides a kind of text semantic analytic method, and method includes: to obtain
The vector expression for determining text is drawn, the coding vector for generating given text is indicated according to vector；Given text includes the first text
With the second text；Coding vector include corresponding first coding vector of the first text and the second text corresponding second encode to
Amount；The first attention and the second attention are generated according to the first coding vector and the second coding vector, and according to the first attention
Global information vector is generated with the second attention；Semantically enhancement vector is obtained from global information vector；According to semantically enhancement to
Amount, parses the target text corresponding with the second text in the first text.Compared with prior art, technical scheme is logical
Cross from global information vector acquisition semantically enhancement vector, and according to semantically enhancement vector analysis go out in the first text with second
The corresponding target text of text enriches vector to the semantic expressiveness of text, accelerates to calculate convergence rate, accuracy rate and efficiency are equal
It is improved.
Fig. 6 is another embodiment of the application text semantic analytic method.As shown in fig. 6, this method may include:
Step 610, the vector for obtaining given text indicates；The given text includes the first text and the second text；Institute
Stating vector indicates to include that the corresponding primary vector of the first text indicates that secondary vector corresponding with the second text indicates.
First text can be chapter sentence (context), and the first text includes several vocabulary and symbol.Second text can
Think query statement (query), includes several vocabulary and symbol.
Specifically, being segmented to obtain word finder to the first text and the second text, for example, the first text or the second text
It is the sentence comprising n vocabulary, obtains the word finder { w of n dimension after participle1, w2..., wn, wherein wiRepresent one
Vocabulary；Then, on the one hand, search the corresponding index of each vocabulary in word finder in default vocabulary, word finder is converted into one
A n-dimensional vector { idw1, idw2..., idwn, wherein idwiRepresent vocabulary wiCorresponding index, further according to the index in n-dimensional vector
Corresponding term vector is searched, a n × d is obtained1The term vector matrix of size；On the other hand, to each vocabulary each in word finder
A point word is carried out, the element for obtaining each position is the word collection of word:
Wherein ci-jIndicate vocabulary wiJ-th of word, the maximum value for the number of words that m includes by a vocabulary；Again in predetermined word
Word is searched in table and concentrates the corresponding index of each word, obtains a n × m × d2The matrix of size carries out a secondary volume to the matrix
Product calculates, and exports a n × d2The word vector matrix of size.
Above-mentioned term vector matrix and word vector matrix are spliced, the vector for obtaining given text indicates.
Step 620, it indicates to obtain the first semantically enhancement vector sum second in secondary vector expression from primary vector respectively
Semantically enhancement vector.It is real to the possibility of step 620 below for obtaining the first semantically enhancement vector in indicating from primary vector
Existing mode is illustrated.
Refering to Fig. 4 and Fig. 5, primary vector can be indicated to carry out convolutional calculation, then convolutional calculation result is carried out again
Transposition convolutional calculation obtains indicating the first semantically enhancement vector with same structure with primary vector.Alternatively, by first to
The convolutional calculation result indicated and the splicing of transposition convolutional calculation result are measured, the first semantically enhancement vector is obtained.
Referring to the treatment process indicated primary vector, secondary vector expression is handled, is increased with obtaining the second semanteme
Dominant vector.
The semantic information that step 620 enriches given text indicates, improves semantic parsing accuracy rate.
Step 630, the first attention and the are generated according to the first semantically enhancement vector sum the second semantically enhancement vector
Two attentions, and global information vector is generated according to first attention and the second attention.
In act 630, it is added first by matrix, respectively to first semantically enhancement vector sum the second semantically enhancement vector
Point of addition encoded information obtains two input matrixes for two convolutional neural networks, passes through one of convolutional Neural
Network extracts the corresponding first partial information vector of the first semantically enhancement vector, extracts second by another convolutional neural networks
The corresponding second local information vector of semantically enhancement vector；Further according to first partial information vector and the second local information acquisition institute
The attention force information for stating given text, according to the attention force information generate the first semantically enhancement vector corresponding first encode to
Measure the second coding vector corresponding with the second semantically enhancement vector；Finally generated according to the first coding vector and the second coding vector
First attention and the second attention, and global information vector is generated according to the first attention and the second attention.
Step 640, according to the global information vector, target corresponding with the second text text in the first text is parsed
This.
The present embodiment technical solution indicates to obtain the first semantically enhancement vector sum in secondary vector expression from primary vector
Second semantically enhancement vector generates the first attention and the second note according to first semantically enhancement vector sum the second semantically enhancement vector
Meaning power, and global information vector is generated according to the first attention and the second attention；According to the global information vector, parse
Target text corresponding with the second text in first text.Compared with prior art, vector is enriched to the semantic table of text
Show, accelerate to calculate convergence rate, parses semantic accuracy rate and efficiency and be improved.
It should be noted that the Overall Steps that the application above method embodiment is related to can be by constructing a text
Semantic analytic modell analytical model is realized.
For example, the model may include input layer, expression layer, alternation of bed, semantically enhancement layer and output layer from bottom to top.
The given text of input is received by input layer；It is term vector by given text representation by expression layer and term vector is compiled
Code obtains the coding vector of given text, that is, realizes the step 110 in embodiment illustrated in fig. 1 by expression layer；Then with table
Show input of the output of layer as alternation of bed, to capture global information by alternation of bed, output is global information vector,
That is, realizing the step 120 in embodiment illustrated in fig. 1 by alternation of bed；Again using the output of alternation of bed as semantically enhancement layer
Input, to enrich the semantic expressiveness of input vector by semantically enhancement layer, output is semantically enhancement vector, that is, passes through language
Adopted enhancement layer realizes the step 130 in embodiment illustrated in fig. 1；Finally by output layer provide in first text with institute
State the corresponding target text of the second text.
Compared with prior art, the model built based on the application text semantic analytic method exports accuracy rate and receipts
Speed is held back to be significantly improved.
It should be noted that can be constructed different according to the different embodiments of the application text semantic analytic method
The text semantic analytic modell analytical model of hierarchical structure, above-mentioned example do not constitute the restriction to the application protection scope and implementation.
Fig. 7 is one embodiment of the application text semantic resolver.The device can be applied to server, PC (PC),
In the plurality of devices such as tablet computer, mobile phone, virtual reality device and intelligent wearable device.
As shown in fig. 7, the apparatus may include: insertion and coding module 710, interactive module 720, semantically enhancement module
730 and parsing module 740.
Wherein, the vector that insertion and coding module 710 are used to obtain given text indicates, indicates to generate according to the vector
The coding vector of the given text；The given text includes the first text and the second text；The coding vector includes the
Corresponding first coding vector of one text and corresponding second coding vector of the second text；Interactive module 720 is used for according to
First coding vector and the second coding vector generate the first attention and the second attention, and according to first attention and the
Two attentions generate global information vector；Semantically enhancement module 730 is for obtaining semantically enhancement from the global information vector
Vector；Parsing module 740 be used for according to the semantically enhancement vector, parse in first text with second text
Corresponding target text.
Specifically, as shown in figure 8, the apparatus may include two insertions and coding module 710, one of them is used for the
One text is handled, another is for handling the second text.Each insertion and coding module 710 may include insertion
Layer 711 and coding layer 712.Wherein, embeding layer 711 is specifically used for handling the given text, obtains the vector table of given text
Show；Coding layer 712 is specifically used for indicating that point of addition encoded information obtains the input matrix of convolutional neural networks to the vector,
To extract local message vector from input matrix using convolutional neural networks；It is given according to local message vector acquisition
Determine the attention force information of text；The coding vector of the given text is generated according to the attention force information.
Such as Fig. 9, may be used also between its embeding layer 711 and coding layer 712 for above-mentioned each insertion and coding module 710
To include semantically enhancement layer 713；In this implementation, embeding layer 711 obtains given text for handling the given text
This vector indicates；Semantically enhancement layer 713 is used to from the expression of the vector of given text obtain preliminary enhancing matrix；Coding layer
712 for obtaining the input matrix of convolutional neural networks to the preliminary enhancing matrix point of addition encoded information, to utilize volume
Product neural network extracts local message vector from the input matrix；The given text is obtained according to the local message vector
This attention force information；The coding vector of the given text is generated according to the attention force information.
Wherein semantically enhancement layer 713 is specifically used for indicating to carry out convolutional calculation to the vector of given text；To the convolution
Calculated result carries out transposition convolutional calculation, obtains preliminary enhancing matrix identical with the vector of given text expression structure.
Interactive module 720 is specifically used for: generating first text according to first coding vector and the second coding vector
The attention matrix of this and the second text；The row and column of the attention matrix is returned respectively using normalization exponential function
One change processing, obtains row processing result and column processing result；By the transposition of the row processing result and second coding vector
Multiplication of vectors obtains the first attention, by the row processing result, the transposed matrix of the column processing result and described first
The transposed vector of coding vector is multiplied to obtain the second attention.
Semantically enhancement module 730 may include: convolutional layer 731 and transposition convolutional layer 732.Convolutional layer 731 is used for described
Global information vector carries out convolutional calculation；Transposition convolutional layer 732 is used to carry out transposition convolutional calculation to convolutional calculation result, obtains
Semantically enhancement vector identical with the global information vector structure.In another implementation, transposition convolutional layer 732 is used for
Transposition convolutional calculation is carried out to convolutional calculation result, obtains intermediate vector identical with the global information vector structure；By institute
It states convolutional calculation result and the intermediate vector is spliced, form the semantically enhancement vector.
As can be seen from the above embodiments, technical scheme is by insertion and coding module 710, obtain given text to
Amount indicates, the coding vector for generating the given text is indicated according to the vector；By interactive module 720, according to described
One coding vector and the second coding vector generate the first attention and the second attention, and according to first attention and second
Attention generates global information vector；By semantically enhancement module 730, semantically enhancement vector is obtained from global information vector；
By parsing module 740, the target text corresponding with the second text in the first text is gone out according to semantically enhancement vector analysis.With
The prior art is compared, and technical scheme enriches vector to the semantic expressiveness of text, accelerates to calculate convergence rate, accuracy rate
It is improved with efficiency.
Figure 10 is another embodiment of the application text semantic resolver.The device can be applied to server, PC
In the plurality of devices such as (PC), tablet computer, mobile phone, virtual reality device and intelligent wearable device.
As shown in Figure 10, the apparatus may include: insertion module 100, semantically enhancement module 200, coding and interactive module
300, parsing module 400.Wherein, the vector that insertion module 100 is used to obtain the given text indicates；The given text packet
Include the first text and the second text；The vector indicates to include that the corresponding primary vector of the first text indicates corresponding with the second text
Secondary vector indicate；Semantically enhancement module 200 from the primary vector for indicating to obtain in secondary vector expression respectively
First semantically enhancement vector sum the second semantically enhancement vector；Coding and interactive module 300 are used for according to first semantically enhancement
Vector sum the second semantically enhancement vector generates the first attention and the second attention, and according to first attention and the second note
Power of anticipating generates global information vector；Parsing module 400 is used to be parsed in first text according to the global information vector
Target text corresponding with second text.
Compared with prior art, the embodiment of the present application device enriches vector to the semantic expressiveness of text, accelerates to calculate and receive
Speed is held back, semantic accuracy rate and efficiency is parsed and is improved.
The application can be used in numerous general or special purpose computing system environments or configuration.Such as: personal computer, service
Device computer, handheld device or portable device, laptop device, multicomputer system, microprocessor-based system, top set
Box, programmable consumer-elcetronics devices, network PC, minicomputer, mainframe computer, including any of the above system or equipment
Distributed computing environment etc..
The application can describe in the general context of computer-executable instructions executed by a computer, such as program
Module.Generally, program module includes routines performing specific tasks or implementing specific abstract data types, programs, objects, group
Part, data structure etc..The application can also be practiced in a distributed computing environment, in these distributed computing environments, by
Task is executed by the connected remote processing devices of communication network.In a distributed computing environment, program module can be with
In the local and remote computer storage media including storage equipment.
It should be noted that, in this document, the relational terms of such as " first " and " second " or the like are used merely to one
A entity or operation with another entity or operate distinguish, without necessarily requiring or implying these entities or operation it
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant are intended to
Cover non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or setting
Standby intrinsic element.
Those skilled in the art will readily occur to its of the application after considering specification and practicing application disclosed herein
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the application, these modifications, purposes or
Person's adaptive change follows the general principle of the application and including the undocumented common knowledge in the art of the application
Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the application are by following
Claim is pointed out.
It should be understood that the application is not limited to the precise structure that has been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.Scope of the present application is only limited by the accompanying claims.
Claims (10)
1. a kind of text semantic analytic method, which is characterized in that the described method includes:
The vector for obtaining given text indicates, the coding vector for generating the given text is indicated according to the vector；It is described to give
Determining text includes the first text and the second text；The coding vector includes corresponding first coding vector of the first text and second
Corresponding second coding vector of text；
The first attention and the second attention are generated according to first coding vector and the second coding vector, and according to described the
One attention and the second attention generate global information vector；
Semantically enhancement vector is obtained from the global information vector；
According to the semantically enhancement vector, the target text corresponding with second text in first text is parsed.
2. the method according to claim 1, wherein it is described from global information vector obtain semantically enhancement to
Amount, comprising:
Convolutional calculation is carried out to the global information vector；
To convolutional calculation result carry out transposition convolutional calculation, obtain semantically enhancement identical with the global information vector structure to
Amount.
3. the method according to claim 1, wherein it is described from global information vector obtain semantically enhancement to
Amount, comprising:
Convolutional calculation is carried out to the global information vector；
Transposition convolutional calculation is carried out to convolutional calculation result, obtains intermediate vector identical with the global information vector structure；
Convolutional calculation result and the intermediate vector are spliced, the semantically enhancement vector is formed.
4. method according to claim 1-3, which is characterized in that the vector expression for obtaining given text,
The coding vector of the generation given text is indicated according to the vector, comprising:
The given text is handled, the vector for obtaining given text indicates；
Point of addition encoded information, which obtains the input matrix of convolutional neural networks, to be indicated to the vector, to utilize convolutional Neural net
Network extracts local message vector from input matrix；
The attention force information of the given text is obtained according to the local message vector；
The coding vector of the given text is generated according to the attention force information.
5. method according to claim 1-3, which is characterized in that the vector expression for obtaining given text,
The coding vector of the generation given text is indicated according to the vector, comprising:
The given text is handled, the vector for obtaining given text indicates；
Preliminary enhancing matrix is obtained from the expression of the vector of given text；
The input matrix of convolutional neural networks is obtained to the preliminary enhancing matrix point of addition encoded information, to utilize convolution mind
Local message vector is extracted from the input matrix through network；
The attention force information of the given text is obtained according to the local message vector；
The coding vector of the given text is generated according to the attention force information.
6. according to the method described in claim 5, it is characterized in that, described obtain preliminary increase from the expression of the vector of given text
Strong matrix, comprising:
The vector of given text is indicated to carry out convolutional calculation；
Transposition convolutional calculation is carried out to convolutional calculation result, obtains preliminary enhancing identical with the vector of given text expression structure
Matrix.
7. the method according to claim 1, wherein described raw according to the first coding vector and the second coding vector
At the first attention and the second attention, comprising:
The attention matrix of first text and the second text is generated according to first coding vector and the second coding vector；
The row and column of the attention matrix is normalized respectively using normalization exponential function, obtains row processing knot
Fruit and column processing result；
The row processing result is multiplied to obtain the first attention with the transposed vector of second coding vector, at the row
Reason result, the transposed matrix of column processing result and the transposed vector of first coding vector are multiplied to obtain the second attention.
8. a kind of text semantic analytic method, which is characterized in that the described method includes:
The vector for obtaining the given text indicates；The given text includes the first text and the second text；The vector table
Showing indicates that secondary vector corresponding with the second text indicates including the corresponding primary vector of the first text；
It indicates to obtain first the second semantically enhancement of semantically enhancement vector sum in secondary vector expression from the primary vector respectively
Vector；
The first attention and the second attention, and root are generated according to the first semantically enhancement vector sum the second semantically enhancement vector
Global information vector is generated according to first attention and the second attention；
According to the global information vector, the target text corresponding with second text in first text is parsed.
9. a kind of text semantic resolver, which is characterized in that described device includes:
Insertion and coding module, the vector for obtaining given text indicates, is indicated to generate the given text according to the vector
This coding vector；The given text includes the first text and the second text；The coding vector is corresponding including the first text
The first coding vector and corresponding second coding vector of the second text；
Interactive module pays attention to for generating the first attention and second according to first coding vector and the second coding vector
Power, and global information vector is generated according to first attention and the second attention；
Semantically enhancement module, for obtaining semantically enhancement vector from the global information vector；
Parsing module, for according to the semantically enhancement vector, parse in first text with second text pair
The target text answered.
10. a kind of text semantic resolver, which is characterized in that described device includes:
It is embedded in module, the vector for obtaining the given text indicates；The given text includes the first text and the second text
This；The vector indicates to include that the corresponding primary vector of the first text indicates that secondary vector corresponding with the second text indicates；
Semantically enhancement module, for respectively from the primary vector indicate and secondary vector expression in obtain the first semantically enhancement to
Amount and the second semantically enhancement vector；
Coding and interactive module pay attention to for generating first according to the first semantically enhancement vector sum the second semantically enhancement vector
Power and the second attention, and global information vector is generated according to first attention and the second attention；
Parsing module, for according to the global information vector, parse in first text with second text pair
The target text answered.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811644576.6A CN109740158B (en) | 2018-12-29 | 2018-12-29 | Text semantic parsing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811644576.6A CN109740158B (en) | 2018-12-29 | 2018-12-29 | Text semantic parsing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109740158A true CN109740158A (en) | 2019-05-10 |
| CN109740158B CN109740158B (en) | 2023-04-07 |
Family
ID=66362781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811644576.6A Active CN109740158B (en) | 2018-12-29 | 2018-12-29 | Text semantic parsing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109740158B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110297889A (en) * | 2019-06-28 | 2019-10-01 | 南京冰鉴信息科技有限公司 | A kind of enterprise's Sentiment orientation analysis method based on Fusion Features |
| CN110443863A (en) * | 2019-07-23 | 2019-11-12 | 中国科学院深圳先进技术研究院 | Method, electronic equipment and the storage medium of text generation image |
| CN110516253A (en) * | 2019-08-30 | 2019-11-29 | 苏州思必驰信息科技有限公司 | Chinese spoken language semantic understanding method and system |
| CN111309891A (en) * | 2020-03-16 | 2020-06-19 | 山西大学 | System for reading robot to automatically ask and answer questions and application method thereof |
| CN111382574A (en) * | 2020-03-11 | 2020-07-07 | 中国科学技术大学 | Semantic parsing system combining syntax under virtual reality and augmented reality scenes |
| CN112883295A (en) * | 2019-11-29 | 2021-06-01 | 北京搜狗科技发展有限公司 | Data processing method, device and medium |
| CN114020881A (en) * | 2022-01-10 | 2022-02-08 | 珠海金智维信息科技有限公司 | Topic positioning method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018058994A1 (en) * | 2016-09-30 | 2018-04-05 | 华为技术有限公司 | Dialogue method, apparatus and device based on deep learning |
| CN108595590A (en) * | 2018-04-19 | 2018-09-28 | 中国科学院电子学研究所苏州研究院 | A kind of Chinese Text Categorization based on fusion attention model |
| CN108959246A (en) * | 2018-06-12 | 2018-12-07 | 北京慧闻科技发展有限公司 | Answer selection method, device and electronic equipment based on improved attention mechanism |
| CN109033068A (en) * | 2018-06-14 | 2018-12-18 | 北京慧闻科技发展有限公司 | It is used to read the method, apparatus understood and electronic equipment based on attention mechanism |
| CN109086423A (en) * | 2018-08-08 | 2018-12-25 | 北京神州泰岳软件股份有限公司 | A kind of text matching technique and device |
| WO2018232699A1 (en) * | 2017-06-22 | 2018-12-27 | 腾讯科技（深圳）有限公司 | Information processing method and related device |
-
2018
- 2018-12-29 CN CN201811644576.6A patent/CN109740158B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018058994A1 (en) * | 2016-09-30 | 2018-04-05 | 华为技术有限公司 | Dialogue method, apparatus and device based on deep learning |
| WO2018232699A1 (en) * | 2017-06-22 | 2018-12-27 | 腾讯科技（深圳）有限公司 | Information processing method and related device |
| CN108595590A (en) * | 2018-04-19 | 2018-09-28 | 中国科学院电子学研究所苏州研究院 | A kind of Chinese Text Categorization based on fusion attention model |
| CN108959246A (en) * | 2018-06-12 | 2018-12-07 | 北京慧闻科技发展有限公司 | Answer selection method, device and electronic equipment based on improved attention mechanism |
| CN109033068A (en) * | 2018-06-14 | 2018-12-18 | 北京慧闻科技发展有限公司 | It is used to read the method, apparatus understood and electronic equipment based on attention mechanism |
| CN109086423A (en) * | 2018-08-08 | 2018-12-25 | 北京神州泰岳软件股份有限公司 | A kind of text matching technique and device |
Non-Patent Citations (2)
| Title |
|---|
| 张仰森等: "基于双重注意力模型的微博情感分析方法", 《清华大学学报(自然科学版)》 * |
| 胡朝举等: "基于词向量技术和混合神经网络的情感分析", 《计算机应用研究》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110297889A (en) * | 2019-06-28 | 2019-10-01 | 南京冰鉴信息科技有限公司 | A kind of enterprise's Sentiment orientation analysis method based on Fusion Features |
| CN110297889B (en) * | 2019-06-28 | 2020-10-23 | 南京冰鉴信息科技有限公司 | Enterprise emotional tendency analysis method based on feature fusion |
| CN110443863A (en) * | 2019-07-23 | 2019-11-12 | 中国科学院深圳先进技术研究院 | Method, electronic equipment and the storage medium of text generation image |
| CN110516253A (en) * | 2019-08-30 | 2019-11-29 | 苏州思必驰信息科技有限公司 | Chinese spoken language semantic understanding method and system |
| CN110516253B (en) * | 2019-08-30 | 2023-08-25 | 思必驰科技股份有限公司 | Chinese spoken language semantic understanding method and system |
| CN112883295A (en) * | 2019-11-29 | 2021-06-01 | 北京搜狗科技发展有限公司 | Data processing method, device and medium |
| CN112883295B (en) * | 2019-11-29 | 2024-02-23 | 北京搜狗科技发展有限公司 | Data processing method, device and medium |
| CN111382574A (en) * | 2020-03-11 | 2020-07-07 | 中国科学技术大学 | Semantic parsing system combining syntax under virtual reality and augmented reality scenes |
| CN111309891A (en) * | 2020-03-16 | 2020-06-19 | 山西大学 | System for reading robot to automatically ask and answer questions and application method thereof |
| CN111309891B (en) * | 2020-03-16 | 2022-05-31 | 山西大学 | System for reading robot to automatically ask and answer questions and application method thereof |
| CN114020881A (en) * | 2022-01-10 | 2022-02-08 | 珠海金智维信息科技有限公司 | Topic positioning method and system |
| CN114020881B (en) * | 2022-01-10 | 2022-05-27 | 珠海金智维信息科技有限公司 | Topic positioning method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109740158B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109947912B (en) | Model method based on intra-paragraph reasoning and joint question answer matching | |
| CN109740158A (en) | Text semantic parsing method and device | |
| CN110019701B (en) | Method for question answering service, question answering service system and storage medium | |
| CN109948691B (en) | Image description generation method and device based on depth residual error network and attention | |
| CN110083705A (en) | A kind of multi-hop attention depth model, method, storage medium and terminal for target emotional semantic classification | |
| CN108875074A (en) | Based on answer selection method, device and the electronic equipment for intersecting attention neural network | |
| CN110134946A (en) | A kind of machine reading understanding method for complex data | |
| CN112800205B (en) | Method and device for obtaining question-answer related paragraphs based on semantic change manifold analysis | |
| Liu et al. | Sign language recognition from digital videos using feature pyramid network with detection transformer | |
| CN118093834B (en) | AIGC large model-based language processing question-answering system and method | |
| CN117236410B (en) | Trusted electronic file large language model training and reasoning method and device | |
| CN109145083B (en) | Candidate answer selecting method based on deep learning | |
| CN114282528A (en) | Keyword extraction method, device, equipment and storage medium | |
| CN113159187B (en) | Classification model training method and device and target text determining method and device | |
| Hong et al. | Selective residual learning for visual question answering | |
| CN117494815A (en) | File-oriented credible large language model training and reasoning method and device | |
| CN111767697B (en) | Text processing method and device, computer equipment and storage medium | |
| CN110852066A (en) | Multi-language entity relation extraction method and system based on confrontation training mechanism | |
| CN113741759B (en) | Comment information display method and device, computer equipment and storage medium | |
| CN113761192B (en) | Text processing method, text processing device and text processing equipment | |
| CN115203388A (en) | Machine reading understanding method and device, computer equipment and storage medium | |
| Kumari et al. | Context-based question answering system with suggested questions | |
| CN109446518B (en) | Decoding method and decoder for language model | |
| CN117932066A (en) | Pre-training-based 'extraction-generation' answer generation model and method | |
| CN113157892B (en) | User intention processing method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |