CN115238705A - Semantic analysis result reordering method and system - Google Patents
Semantic analysis result reordering method and system Download PDFInfo
- Publication number
- CN115238705A CN115238705A CN202210731235.2A CN202210731235A CN115238705A CN 115238705 A CN115238705 A CN 115238705A CN 202210731235 A CN202210731235 A CN 202210731235A CN 115238705 A CN115238705 A CN 115238705A
- Authority
- CN
- China
- Prior art keywords
- natural language
- sample
- candidate
- reordering
- semantic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000004458 analytical method Methods 0.000 title claims abstract description 38
- 238000012549 training Methods 0.000 claims abstract description 104
- 238000013528 artificial neural network Methods 0.000 claims abstract description 40
- 230000014509 gene expression Effects 0.000 claims abstract description 23
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 19
- 239000013598 vector Substances 0.000 claims description 35
- 230000015654 memory Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000010845 search algorithm Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 3
- 239000010410 layer Substances 0.000 description 22
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000003058 natural language processing Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012163 sequencing technique Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- 244000290333 Vanilla fragrans Species 0.000 description 1
- 235000009499 Vanilla fragrans Nutrition 0.000 description 1
- 235000012036 Vanilla tahitensis Nutrition 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/338—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention provides a method and a system for reordering semantic analysis results, wherein the method comprises the following steps: analyzing a target natural language sentence through a semantic analyzer to obtain a candidate logic expression set of the target natural language sentence; inputting the target natural language statement and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result; determining a target logic representation of the target natural language statement according to the candidate semantic reordering result; the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results. According to the invention, the semantic analysis results are reordered, so that the probability space is further searched, the final semantic analysis result is determined according to the reordered result, and the performance and the accuracy of the inference algorithm in the semantic analysis are improved.
Description
Technical Field
The invention relates to the technical field of natural language processing, in particular to a method and a system for reordering semantic analysis results.
Background
Semantic parsing refers to mapping a natural language sentence into a logical representation form, which is a structured semantic representation form applied to many scenarios, such as scenarios like knowledge question answering, search engines, system control and database query interfaces. The natural language sentences are analyzed and converted into specially designed semantic forms (including general first-order logic, configuration languages in specific fields and the like), so that the semantic analysis of the natural language sentences is completed.
In the existing semantic parsing technology, inference algorithm (Inference) is one of the important components in the semantic parsing technology, and is used to select the semantic representation with the highest probability for output to the user. The traditional reasoning algorithm uses the well-known CKY (Cocke Young Kasami) algorithm in natural language processing, and the new neural network system uses vector representation and model parameters to execute forward calculation to complete reasoning.
Because of the objective reasons of huge semantic representation combined space, high model complexity, limited training data and the like, the existing reasoning algorithm uses a greedy strategy or a dynamic programming mode, and approximately finds the result with the maximum global probability by decomposing the problem and obtaining the optimal solution of each local part. However, the existing semantic parsing technology provides the output result with the highest probability, and has the problems of huge cost and inaccurate output result. Therefore, a method and system for reordering semantic parsing results are needed to solve the above problems.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a system for reordering semantic analysis results.
The invention provides a method for reordering semantic analysis results, which comprises the following steps:
analyzing a target natural language sentence through a semantic analyzer to obtain a candidate logic expression set of the target natural language sentence;
inputting the target natural language statement and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result;
determining a target logic representation of the target natural language statement according to the candidate semantic reordering result;
the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
According to the reordering method of semantic parsing results provided by the present invention, before the target natural language sentence and the candidate logical representation set are input to a reordering model to obtain a candidate semantic reordering result, the method further comprises:
acquiring a target similar natural language corresponding to the target natural language sentence based on a preset semantic vector index; acquiring target similar logic representations corresponding to the candidate logic representations in the candidate logic representation set based on a preset structural index;
performing fine tuning training on the reordering model through the target similar natural language and the target similar logic representation to obtain a re-ordering model after fine tuning training;
inputting the target natural language statement and the candidate logical representation set into a reordering model to obtain a candidate semantic reordering result, including:
and inputting the target natural language sentence and the candidate logic expression set into the reordering model after the fine training to obtain a candidate semantic reordering result.
According to the reordering method of the semantic analysis result provided by the invention, the reordering model is obtained by training the following steps:
constructing a first training sample set according to a sample natural language statement and a sample logic representation corresponding to the sample natural language statement;
training a neural network through the first training sample set, outputting a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence, and obtaining the semantic parser;
marking a positive error label on each sample candidate logic representation prediction result, and constructing a second training sample set according to the sample natural language statement and the sample candidate logic representation prediction result marked with the positive error label;
training the deep neural network through the second training sample set to obtain a pre-trained deep neural network;
acquiring a natural language similar sample through a preset semantic vector index and the sample natural language sentence; acquiring a logic representation similar sample through a preset structural index and the sample candidate logic representation prediction result;
constructing a third training sample set according to the natural language similar sample and the logic representation similar sample;
and training the pre-trained deep neural network through the third training sample set to obtain the reordering model.
According to the reordering method of semantic analysis results provided by the present invention, after the training of the neural network by the first training sample set, the method further comprises:
acquiring a sample candidate logic representation prediction result output by the neural network;
and determining a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence based on the column search algorithm and the preset candidate result number.
According to the reordering method of the semantic parsing result provided by the invention, a natural language similar sample is obtained through a preset semantic vector index and the sample natural language statement; before obtaining a logic representation similar sample through presetting a structural index and the sample candidate logic representation prediction result, the method further comprises:
obtaining a sample vector representation corresponding to the sample natural language sentence through a pre-training BERT model;
constructing a preset semantic vector index according to the sample vector representation and the similarity algorithm;
and constructing a preset structural index by using a tree edit distance algorithm and the sample candidate logic to represent a prediction result.
According to the semantic analysis result reordering method provided by the invention, an alignment layer of the deep neural network is provided with a concern mechanism, and an encoding layer is provided with a long-time memory network.
According to the reordering method of semantic parsing results provided by the present invention, before parsing a target natural language sentence through a semantic parser to obtain a candidate logical representation set of the target natural language sentence, the method further comprises:
preprocessing a target natural language sentence to obtain a standard sentence with the same meaning as the target natural language sentence;
the parsing a target natural language sentence through a semantic parser to obtain a candidate logical representation set of the target natural language sentence includes:
and analyzing the standard statement through a semantic analyzer to obtain a candidate logic representation set.
The invention also provides a system for reordering semantic analysis results, which comprises:
the candidate logic representation generation module is used for analyzing the target natural language sentences through a semantic analyzer to obtain a candidate logic representation set of the target natural language sentences;
the candidate semantic reordering module is used for inputting the target natural language sentence and the candidate logic representation set into a reordering model to obtain a candidate semantic reordering result;
a semantic parsing result generation module, configured to determine a target logical representation of the target natural language sentence according to the candidate semantic reordering result;
the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the semantic analysis result reordering method.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a semantic parsing result reordering method as described in any of the above.
According to the semantic analysis result reordering method and system provided by the invention, the semantic analysis result is reordered, so that the probability space is further searched, the final semantic analysis result is determined according to the reordering result, and the performance and accuracy of an inference algorithm in semantic analysis are improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a reordering method for semantic parsing results according to the present invention;
FIG. 2 is a schematic diagram of an overall architecture of a reordering model according to the present invention;
FIG. 3 is a schematic structural diagram of a semantic parsing result reordering system according to the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The existing mature system for semantic analysis is probabilistic, on one hand, because natural language inherently has ambiguity, ambiguity is eliminated in daily life, and semantic understanding is completed mainly by combining two parties of communication with personal knowledge background under specific occasions through modes such as inquiry confirmation and the like; on the other hand, as the statistical model is widely applied to natural language processing, the performance of the statistical model is obviously superior to that of the traditional technologies such as expert systems, artificial templates, deterministic logic reasoning and induction and the like.
A specific example of semantic parsing is a code generation task, which takes as an example the input question "define the function entries with 2 definitions. Table 1 shows candidate results and reordered samples for semantic parsing using the column search algorithm, as shown in table 1:
TABLE 1
| Logic Forms | Reranking Score | |
| 1 | def iterkeys(d):pass | -5.8956 |
| 2 | def iterkeys(d,*kw):pass | -5.5826 |
| 3 | def iterkeys(d,**kw):pass | -1.5059 |
| 4 | def iterkeys(d,*kw,**kwargs):pass | -6.8337 |
| 5 | def iterkeys(d,*kw,**options):pass | -6.8315 |
In table 1, the first 5 candidate results are obtained by using the column search algorithm, and the best output is arranged at the third position, so that it can be seen that the corresponding logical representation form between the second candidate result and the third candidate result is very small, but the final reordering results of the two candidate results are very different.
In the existing reordering method of semantic parsing results, for the input of a natural language sentence, a plurality of candidate outputs can be obtained by a column search method through an arbitrary probability semantic parsing tool, and a relevance score is calculated for each candidate output together with the input through designing a sorting model for sorting. Specifically, the prior art first aligns each input and output with each other; secondly, comparing the aligned parts pairwise, and calculating the score of the alignment characteristic; finally, the different alignment scores are aggregated to obtain a total score.
The key point of the prior art is in the design of a sequencing model, but the two aligned natural languages with different structures and semantic representations thereof are used as word sequences for alignment, so that the performance is poor, and particularly for a sequencing candidate set with similar word faces but larger structural differences, misjudgment is easily generated by the technology. The prior art defect is derived from two steps of model design, alignment and comparison, structural features of semantic representation are not introduced into the model, and only literal similarity of two sequences is compared during alignment.
Aiming at the problems in the prior art, the invention provides a method for reordering semantic parsing results, wherein reordering is beneficial to further searching a probability space and improving the performance of an inference algorithm in semantic parsing. In the reordering process, the invention provides a method for accurately adjusting a model by using similar samples, various indexes of semantics and structures are designed, samples which are similar semantically and structurally are obtained in real time, and the method is favorable for improving the utilization of structural information of an analysis result.
Fig. 1 is a schematic flow diagram of a semantic analysis result reordering method provided by the present invention, and as shown in fig. 1, the present invention provides a semantic analysis result reordering method, which includes:
In the invention, a neural network can be trained by sample natural language sentences and an unsupervised training method to obtain a probabilistic semantic parser. Specifically, a probabilistic semantic parser is obtained through training by using pairs of natural language and logic expression as training data, and a batch of high-probability parsing results can be output after natural language sentences to be parsed (namely target natural language sentences) are input into the semantic parser. It should be noted that the present invention is not limited to the type of the semantic parser, and the semantic parser obtained by other methods may be used alternatively.
And 102, inputting the target natural language sentence and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result.
Preferably, before said inputting said target natural language sentence and said set of candidate logical representations into a reordering model to obtain candidate semantic reordering results, said method further comprises:
acquiring a target similar natural language corresponding to the target natural language sentence based on a preset semantic vector index; acquiring target similar logic representations corresponding to the candidate logic representations in the candidate logic representation set based on a preset structural index;
performing fine training on the reordering model through the target similar natural language and the target similar logic representation to obtain a reordered model after the fine training;
inputting the target natural language statement and the candidate logical representation set into a reordering model to obtain a candidate semantic reordering result, including:
and inputting the target natural language sentence and the candidate logic expression set into the reordering model after the fine training to obtain a candidate semantic reordering result.
In the invention, the candidate logic expression set of the target natural language sentence is obtained after the target natural language sentence acquired each time is input into the semantic parser, because different application fields exist in the knowledge question-answer scene of the natural language, the semantics of the same question-answer in different application fields may be deviated, and the judgment of the subtle difference of the problems with similar semantics is particularly difficult for the model. In order to solve the above problem, in each semantic parsing reordering process, the invention respectively obtains the target similar natural language corresponding to the target natural language sentence and the target similar logic representation corresponding to each candidate logic representation in the candidate logic representation set by using the preset semantic vector index and the preset structural index, so that the reordering model is finely adjusted through the similar natural language sentence and the similar logic representation, and the semantic parsing reordering effect of the reordering model for the natural language under different application field types is further improved.
103, determining a target logic representation of the target natural language statement according to the candidate semantic reordering result;
the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
In the invention, the candidate logical representation set acquired in the embodiment is input into a reordering model to perform more targeted matching scoring, the candidate logical representations in the set are ordered, and finally the highest scoring is selected from the reordered analysis result as the target logical representation.
Specifically, in one embodiment, a probabilistic semantic parser may be obtained by training using existing techniques; then, outputting a batch of possible candidate results of the natural language training data by using the semantic parser, and marking the correctness of each candidate result; then, establishing a semantic vector index for input natural language training data, and establishing a structural index for output candidate results; further, constructing a reordering model by using the prior art, and training the reordering model by using natural language training data and candidate results; then, outputting a batch of candidate outputs from the semantic parser for each input natural language training data, and acquiring similar samples from corresponding indexes through semantic features corresponding to the natural language training data and structural features corresponding to candidate results; and finally, carrying out fine tuning training on the reordering model by using a similar sample to obtain a trained reordering model, reordering candidate results of the target natural language sentence to be analyzed by using the trained reordering model, and selecting the analysis result with the highest score after sequencing to serve as a target analysis result.
The semantic analysis result reordering method provided by the invention is beneficial to further searching the probability space by reordering the semantic analysis result, thereby determining the final semantic analysis result according to the reordering result and improving the performance and accuracy of the inference algorithm in the semantic analysis.
On the basis of the above embodiment, the reordering model is obtained by training through the following steps:
constructing a first training sample set according to a sample natural language statement and a sample logic representation corresponding to the sample natural language statement;
and training the neural network through the first training sample set, outputting a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence, and obtaining the semantic parser.
In the present invention, a training data set for the reordering stage needs to be constructed. In the existing semantic parsing data set D, only one standard semantic representation (logical representation) y is provided for each input natural language x, and by representing the natural language x and the semantic representation y, a semantic parsing tool can be trained, typically using neural network modeling, including the parameter θ. The training mode uses standard cross entropy, i.e. optimizes the loss function as follows:
and outputting a batch of sample candidate logic representation prediction results by the semantic parser obtained by training in the mode.
On the basis of the foregoing embodiment, after the training of the neural network by the first training sample set, the method further includes:
acquiring a sample candidate logic representation prediction result output by the neural network;
and determining a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence based on the column search algorithm and the preset candidate result number.
In the invention, a candidate set corresponding to each sample natural language statement is obtained by a Beam Search algorithm, so that the capability of an inference algorithm is improved with the lowest cost. Specifically, the number of columns is generally determined empirically, assuming that K is used, K results of the previous step are sequentially analyzed at each local decision, K best results are retained in combination with the current decision, naturally, K results are obtained when the algorithm is executed to the last step, and the other one with the highest retention probability is generally discarded. Therefore, the reordering technique provided by the present invention selects one of the K better candidates for the final semantic resolution, rather than selecting the first result by default. Further, in the present invention, the number 5 is selected as the size of the candidate set, that is, each sample natural language statement corresponds to 5 sample candidate logics representing the prediction result.
And marking a positive error label for each sample candidate logic representation prediction result, and constructing a second training sample set according to the sample natural language statement and the sample candidate logic representation prediction result marked with the positive error label.
In the present invention, a batch of candidate results with higher probability are output by the semantic analyzer of the above embodiment for the same training data. In order to mark the positive error of each candidate result (namely, the positive sample label is marked for the correct candidate result, and the negative sample label is marked for the wrong candidate result), the invention uses two methods to determine the positive error of the candidate result, wherein one method is to require that the output candidate result is completely consistent with the standard output to be correct, and otherwise, the output candidate result is wrong; the other is to execute the logical representation in the candidate result on the interpreter, whose result is consistent with the standard output. Only one of the above two methods of determining whether the candidate is correct or incorrect may be used.
Specifically, taking as an example that each sample natural language statement corresponds to 5 sample candidate logical representation prediction results, according to standard semantic representation in training data, the correctness of the 5 sample candidate logical representation prediction results is evaluated, and if the correctness is correct, the sample candidate logical representation prediction result y is marked as 1, otherwise, the sample candidate logical representation prediction result y is 0. This results in a reordered 0-1 sorted task data set D' = { x } i ,y ij ,c ij } ij Wherein x is i Denotes the ith natural language sentence, y ij J-th candidate logical representation representing the i-th natural language sentence, c ij The jth candidate logic representing the ith natural language statement represents a corresponding label, and the value is only 0 or 1.
Training the deep neural network through the second training sample set to obtain a pre-trained deep neural network;
in the invention, a prediction result is expressed by sample natural language sentences and sample candidate logics marked with correct and wrong labels, a naive matching model is used for training, so that the natural language sentences and the logic expressions are respectively encoded, a cross attention mechanism is used for information communication, finally, a pooling layer is used for output expression, and matching scores are output after splicing, so that matching ordering of the natural language sentences and the candidate results is realized, and a pre-trained deep neural network is obtained.
Acquiring a natural language similar sample through a preset semantic vector index and the sample natural language sentence; and acquiring a logic representation similar sample through a preset structural index and the sample candidate logic representation prediction result.
In the invention, for the reordering model, the input comprises two parts of natural language sentences and logic representation, semantic vectors are needed to index for natural language, and structural indexes are needed to be designed for logic representation. The semantic vector index is realized by vector representation output by a large-scale language model, and needs to be calculated off line by using a natural language input domain and stored; the method using the structure index is mainly online calculation, but needs offline storage by using an information retrieval tool.
On the basis of the embodiment, a natural language similar sample is obtained through the preset semantic vector index and the sample natural language sentence; before obtaining a logic representation similar sample through presetting a structural index and the sample candidate logic representation prediction result, the method further comprises:
obtaining a sample vector representation corresponding to the sample natural language sentence through a pre-training BERT model;
constructing a preset semantic vector index according to the sample vector representation and the similarity algorithm;
and constructing a preset structural index by using a tree edit distance algorithm and the sample candidate logic to represent a prediction result.
In the present invention, an input sample natural language sentence x, via a semantic parser, gets a set of candidate results. For each sample candidate in the candidate result, representing the prediction result y logically, a respective set of similarity samples is retrieved from the preset structural index.
Specifically, before obtaining similar samples, semantic vector indexes and structural indexes of candidate sets are established first. Each sample in the reordered data set comprises a sample natural language statement x and a sample candidate logical representation prediction result y, and two indexes are required to be established respectively for the purpose. For a sample natural language statement x, a character index can be established by using information retrieval tools such as Lucene and the like, and the character index is recorded as NL-NGRAM; the vector representation obtained by the pre-training BERT model can be used for calculating the cosine similarity to obtain the similarity, so that the vector representation given by the pre-training BERT model is also recorded into a KD tree index, and a preset semantic vector index is constructed and recorded as NL-BERT. For the sample candidate logic representation prediction result y, when a preset structural index is constructed, an ordinary character index is established and recorded as LF-NGRAM by using Lucene and other information retrieval tools; the similarity of the tree structure of semantic representation analysis can be evaluated by using technologies such as APTED algorithm of tree editing distance and the like, and is recorded as LF-TED. In actual use, LF-NGRAM index can be used to screen out a large amount of semantic representations, and then tree editing distance is calculated, so that the calculation amount is greatly reduced.
In the present invention, for the various index construction methods of the above embodiments, the similarity sample can also be obtained by various methods in the search. When the LF-TED is used for indexing, because the efficiency of an algorithm for calculating the edit distance of the tree is relatively low, similar samples are difficult to search in real time on the whole training set, therefore, the method needs to firstly use the LF-NGRAM index to obtain a batch of small samples, and then calculate the edit distance to obtain similar samples.
And constructing a third training sample set according to the natural language similar sample and the logic representation similar sample.
In the invention, a batch of similar samples are respectively inquired from the preset semantic vector index and the preset structural index constructed in the embodiment according to the sample natural language sentence and the sample candidate logic expression prediction result output by the semantic analyzer. The semantic vector index is obtained by performing Euclidean distance calculation on a natural language part of a candidate sample according to the index of the vector; the structural index requires first querying a batch of samples from the offline storage and then calculating the tree edit distance of the set from the current sample online. The similar samples obtained by retrieval are used for carrying out fine training on the subsequent reordering model, the similar samples are similar to each sample no matter whether the sample is correct or incorrect in the set to be ordered, different indexes are respectively inquired by using the natural language part and the logic representation part of the samples, and the similarity has better diversity.
And training the pre-trained deep neural network through the third training sample set to obtain the reordering model.
In the invention, the pre-trained deep neural network obtained by the embodiment utilizes the similar samples to perform parameter optimization training, thereby obtaining the trained reordering model and enabling the model to be more inclined to the samples. And finally, using the trained reordering model to rank and score the candidate logic representation set input in practical application, and taking the candidate logic representation result with the highest ranking score as final output.
On the basis of the above embodiment, the alignment layer of the deep neural network is provided with an attention mechanism, and the coding layer is provided with a long-term memory network.
In the invention, a reordering model is constructed through a deep network model, the model is input into a natural language statement x and a candidate logic representation y, and the probability of a predicted value c =1 is output. Specifically, the reordering model of the present invention mainly comprises: an embedding layer for converting the natural language sentence x and the semantic representation y into a numerical vector; the encoding layer is used for converting the numerical value vector into a hidden layer vector, wherein the conversion mode uses a multi-layer Long Short Term Memory (LSTM) network, and the obtained features are spliced with the features of the embedding layer; the alignment layer is used for splicing the characteristics of the coding layer and the results of the embedding layer, modeling the characteristics through a bidirectional Attention Mechanism (Attention Mechanism) and outputting the characteristics; the fusion layer is used for constructing heuristic characteristics [ h, k, h x k, h-k ] by utilizing the output h of the alignment layer and the output k of the coding layer, and splicing the characteristics for linear transformation fusion; and the pooling layer is used for fusing the outputs of the previous layers into a single layer and then calculating the value of the probability P (c = 1) of the prediction layer. Among them, the encoding layer, the alignment layer, and the fusion layer are overlapped a plurality of times, and 4 times are used in this embodiment. In addition, the ordering of the output of the semantic parser is also an important feature, which helps to prompt the reordering model, what the order of the candidate sets was originally, and to modify it. The ordering attribute is defined as follows:
where rank (y) represents the original order, starting with 0, and taking the value of 2 if the 3 rd bit. The reordering model of the invention is optimized by using the minimum cross entropy loss, which is specifically shown as the following formula:
wherein the weight w i Are equal normalized weights. Fig. 2 is a schematic diagram of an overall architecture of a reordering model provided in the present invention, and the structure and the overall workflow of the reordering model can be referred to fig. 2.
On the basis of the foregoing embodiment, before the parsing, by the semantic parser, the target natural language sentence to obtain the candidate logical representation set of the target natural language sentence, the method further includes:
preprocessing a target natural language sentence to obtain a standard sentence with the same meaning as the target natural language sentence;
the parsing a target natural language sentence through a semantic parser to obtain a candidate logical representation set of the target natural language sentence includes:
and analyzing the standard statement through a semantic analyzer to obtain a candidate logic representation set.
In the invention, aiming at the spoken language existing in the actual scene, the spoken language sentence can be firstly converted into the standard sentence through a sentence conversion model, for example, the natural language sentence is 'how many people are in company', the standard sentence obtained through the sentence conversion model is 'the number of staff in company', and further the subsequent semantic analysis efficiency and accuracy are improved. The sentence conversion model can be obtained by training in an unsupervised training mode.
In one embodiment, in order to verify the effectiveness of the semantic parsing reordering method provided by the present invention, the present invention performs verification on two data sets, namely an ATIS data set and a Django data set. The ATIS data set is a small-scale data set, is input as a natural language question, and is output as a first-order logic expression expressed in the form of LISP sentences; the Django data set is large in scale, a sentence with code intention expressed in English is input, and a python statement fragment with corresponding semantics of the sentence is output. Two data set sizes are shown in table 2:
TABLE 2
| Dataset | ATIS | Django |
| Training | 4434 | 16000 |
| Validation | 448 | 1000 |
| Testing | 491 | 1805 |
| Candidates per instance | 5 | 15 |
The method for comparison includes: seq2Tree, coarse2Fine, tranX, baseparr (basic semantic analysis tool, recurrent TranX model without ranking performance), pseudo reranker (ranking model trained only with raw data without using candidate semantic representation set), and VanillaReranker (ranking model trained with candidate semantic representation set but without using similarity samples for Fine tuning).
Aiming at different similar indexing methods, the reordering model of the invention embodies different performances, and it can be seen that in the ATIS data set, the same performances of the invention and the ranking model Vanilla Reranker are obtained, which are all superior to other comparison methods. On the Django dataset, the reordering model of the invention surpasses other various methods, and among different similarity measures, the use of the Tree edit distance index (LF TED) shows the best performance, showing the effectiveness of adding structural information in the invention. The final semantic parsing specific results of each method are shown in table 3:
TABLE 3
Experiments on another set of Django data sets showed that as shown in table 4:
TABLE 4
When the size of the candidate set is gradually increased, the method proposed by the present invention has a stable improvement because there is more possibility to find the candidate sample. And the performance of the common sorting model which does not use similar samples is jittered from 75.85 to 75.12, which causes the performance of the candidate set to be increased.
The semantic analysis result reordering system provided by the invention is described below, and the semantic analysis result reordering system described below and the semantic analysis result reordering method described above can be referred to correspondingly.
Fig. 3 is a schematic structural diagram of a semantic parsing result reordering system provided by the present invention, and as shown in fig. 3, the present invention provides a semantic parsing result reordering system, which includes a candidate logical representation generating module 301, a candidate semantic reordering module 302, and a semantic parsing result generating module 303, where the candidate logical representation generating module 301 is configured to parse a target natural language sentence through a semantic parser to obtain a candidate logical representation set of the target natural language sentence; the candidate semantic reordering module 302 is configured to input the target natural language sentence and the candidate logical representation set into a reordering model to obtain a candidate semantic reordering result; the semantic parsing result generating module 303 is configured to determine a target logical representation of the target natural language sentence according to the candidate semantic reordering result;
the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
The semantic parsing result reordering system provided by the invention is beneficial to further searching the probability space by reordering the semantic parsing result, so that the final semantic parsing result is determined according to the reordering result, and the performance and accuracy of a reasoning algorithm in the semantic parsing are improved.
On the basis of the embodiment, the system further comprises a first training set constructing module, a first training module, a second training set constructing module, a second training module, a similar sample obtaining module, a third training set constructing module and a third training module, wherein the first training set constructing module is used for constructing a first training sample set according to a sample natural language sentence and a sample logic representation corresponding to the sample natural language sentence; the first training module is used for training a neural network through the first training sample set, outputting a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence, and obtaining the semantic parser; the second training set construction module is used for marking a correct and wrong label on each sample candidate logic representation prediction result and constructing a second training sample set according to the sample natural language sentences and the sample candidate logic representation prediction results marked with the correct and wrong labels; the second training module is used for training the deep neural network through the second training sample set to obtain a pre-trained deep neural network; the similar sample acquisition module is used for acquiring a natural language similar sample through a preset semantic vector index and the sample natural language sentence; acquiring a logic representation similar sample through a preset structural index and the sample candidate logic representation prediction result; the third training set constructing module is used for constructing a third training sample set according to the natural language similar sample and the logic representation similar sample; and the third training module is used for training the pre-trained deep neural network through the third training sample set to obtain the reordering model.
The system provided by the present invention is used for executing the above method embodiments, and for the specific processes and details, reference is made to the above embodiments, which are not described herein again.
Fig. 4 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 4, the electronic device may include: a Processor (Processor) 401, a communication Interface (communication Interface) 402, a Memory (Memory) 403 and a communication bus 404, wherein the Processor 401, the communication Interface 402 and the Memory 403 complete communication with each other through the communication bus 404. The processor 401 may call logic instructions in the memory 803 to perform a semantic parsing result reordering method, which includes: analyzing a target natural language sentence through a semantic analyzer to obtain a candidate logic expression set of the target natural language sentence; inputting the target natural language statement and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result; determining a target logic representation of the target natural language statement according to the candidate semantic reordering result; the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
In addition, the logic instructions in the memory 403 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the semantic parsing result reordering method provided by the above methods, the method including: analyzing a target natural language sentence through a semantic analyzer to obtain a candidate logic expression set of the target natural language sentence; inputting the target natural language statement and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result; determining a target logic representation of the target natural language statement according to the candidate semantic reordering result; the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to execute the semantic parsing result reordering method provided in the above embodiments, the method including: analyzing a target natural language sentence through a semantic analyzer to obtain a candidate logic expression set of the target natural language sentence; inputting the target natural language statement and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result; determining a target logic representation of the target natural language statement according to the candidate semantic reordering result; the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A reordering method of semantic analysis results is characterized by comprising the following steps:
analyzing a target natural language sentence through a semantic analyzer to obtain a candidate logic expression set of the target natural language sentence;
inputting the target natural language statement and the candidate logic expression set into a reordering model to obtain a candidate semantic reordering result;
determining a target logic representation of the target natural language statement according to the candidate semantic reordering result;
the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
2. The method for reordering semantic parsing results according to claim 1, wherein before the inputting the target natural language sentence and the set of candidate logical representations into a reordering model to obtain candidate semantic reordering results, the method further comprises:
acquiring a target similar natural language corresponding to the target natural language sentence based on a preset semantic vector index; acquiring target similar logic representations corresponding to the candidate logic representations in the candidate logic representation set based on a preset structural index;
performing fine tuning training on the reordering model through the target similar natural language and the target similar logic representation to obtain a re-ordering model after fine tuning training;
inputting the target natural language statement and the candidate logic representation set into a reordering model to obtain a candidate semantic reordering result, including:
and inputting the target natural language sentence and the candidate logic expression set into the reordering model after the fine training to obtain a candidate semantic reordering result.
3. The method for reordering semantic parsing results of claim 1, wherein the reordering model is trained by the following steps:
constructing a first training sample set according to a sample natural language statement and a sample logic representation corresponding to the sample natural language statement;
training a neural network through the first training sample set, outputting a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence, and obtaining the semantic parser;
marking a positive error label on each sample candidate logic representation prediction result, and constructing a second training sample set according to the sample natural language statement and the sample candidate logic representation prediction result marked with the positive error label;
training the deep neural network through the second training sample set to obtain a pre-trained deep neural network;
acquiring a natural language similar sample through a preset semantic vector index and the sample natural language sentence; acquiring a logic representation similar sample through a preset structural index and the sample candidate logic representation prediction result;
constructing a third training sample set according to the natural language similar sample and the logic representation similar sample;
and training the pre-trained deep neural network through the third training sample set to obtain the reordering model.
4. The method for reordering semantic parsing results of claim 3, wherein after the training of the neural network by the first set of training samples, the method further comprises:
acquiring a sample candidate logic representation prediction result output by the neural network;
and determining a plurality of sample candidate logic representation prediction results corresponding to each sample natural language sentence based on a column search algorithm and a preset candidate result number.
5. The method for reordering semantic parsing results of claim 3, wherein a natural language similar sample is obtained from the sample natural language sentence indexed by a predetermined semantic vector; before obtaining a logic representation similar sample through presetting a structural index and the sample candidate logic representation prediction result, the method further comprises:
obtaining a sample vector representation corresponding to the sample natural language sentence through a pre-training BERT model;
constructing a preset semantic vector index according to the sample vector representation and the similarity algorithm;
and constructing a preset structural index by using a tree edit distance algorithm and the sample candidate logic to represent a prediction result.
6. The method for reordering semantic parsing results according to claim 3, wherein an alignment layer of the deep neural network is provided with a focus mechanism, and an encoding layer is provided with a long-time memory network.
7. The reordering method of semantic parsing results according to any one of claims 1 to 6, wherein before the parsing the target natural language sentence by the semantic parser to obtain the candidate logical representation set of the target natural language sentence, the method further comprises:
preprocessing a target natural language sentence to obtain a standard sentence with the same meaning as the target natural language sentence;
the parsing a target natural language sentence through a semantic parser to obtain a candidate logical representation set of the target natural language sentence includes:
and analyzing the standard statement through a semantic analyzer to obtain a candidate logic representation set.
8. A system for reordering semantic parsing results, comprising:
the candidate logic representation generation module is used for analyzing the target natural language sentences through a semantic analyzer to obtain a candidate logic representation set of the target natural language sentences;
the candidate semantic reordering module is used for inputting the target natural language sentence and the candidate logic representation set into a reordering model to obtain a candidate semantic reordering result;
a semantic parsing result generation module, configured to determine a target logical representation of the target natural language sentence according to the candidate semantic reordering result;
the reordering model is obtained by training a pre-trained deep neural network based on natural language similar samples corresponding to sample natural language sentences and logic representation similar samples corresponding to sample candidate logic representation prediction results.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the semantic analysis result reordering method according to any one of claims 1 to 7 when executing the computer program.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the semantic parsing result reordering method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210731235.2A CN115238705A (en) | 2022-06-24 | 2022-06-24 | Semantic analysis result reordering method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210731235.2A CN115238705A (en) | 2022-06-24 | 2022-06-24 | Semantic analysis result reordering method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115238705A true CN115238705A (en) | 2022-10-25 |
Family
ID=83669403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210731235.2A Pending CN115238705A (en) | 2022-06-24 | 2022-06-24 | Semantic analysis result reordering method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115238705A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116048454A (en) * | 2023-03-06 | 2023-05-02 | 山东师范大学 | Code rearrangement method and system based on iterative comparison learning |
-
2022
- 2022-06-24 CN CN202210731235.2A patent/CN115238705A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116048454A (en) * | 2023-03-06 | 2023-05-02 | 山东师范大学 | Code rearrangement method and system based on iterative comparison learning |
| CN116048454B (en) * | 2023-03-06 | 2023-06-16 | 山东师范大学 | Code rearrangement method and system based on iterative comparison learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109271505B (en) | Question-answering system implementation method based on question-answer pairs | |
| CN109992782B (en) | Legal document named entity identification method and device and computer equipment | |
| CN112800170A (en) | Question matching method and device and question reply method and device | |
| CN112035730B (en) | Semantic retrieval method and device and electronic equipment | |
| CN112069298A (en) | Human-computer interaction method, device and medium based on semantic web and intention recognition | |
| CN110765277B (en) | Knowledge-graph-based mobile terminal online equipment fault diagnosis method | |
| CN115599902B (en) | Oil-gas encyclopedia question-answering method and system based on knowledge graph | |
| CN112328800A (en) | System and method for automatically generating programming specification question answers | |
| CN116127095A (en) | Question-answering method combining sequence model and knowledge graph | |
| CN115470338B (en) | Multi-scenario intelligent question answering method and system based on multi-path recall | |
| CN116150335A (en) | Text semantic retrieval method under military scene | |
| CN112463944B (en) | Search type intelligent question-answering method and device based on multi-model fusion | |
| CN111966810A (en) | Question-answer pair ordering method for question-answer system | |
| CN116166782A (en) | Intelligent question-answering method based on deep learning | |
| CN112632250A (en) | Question and answer method and system under multi-document scene | |
| CN112925918A (en) | Question-answer matching system based on disease field knowledge graph | |
| CN115658846A (en) | Intelligent search method and device suitable for open-source software supply chain | |
| CN115905487A (en) | Document question and answer method, system, electronic equipment and storage medium | |
| CN110851584A (en) | Accurate recommendation system and method for legal provision | |
| CN116842126B (en) | Method, medium and system for realizing accurate output of knowledge base by using LLM | |
| CN115238705A (en) | Semantic analysis result reordering method and system | |
| CN111581365B (en) | Predicate extraction method | |
| CN117454898A (en) | Method and device for realizing legal entity standardized output according to input text | |
| CN115982338A (en) | Query path ordering-based domain knowledge graph question-answering method and system | |
| CN115203206A (en) | Data content searching method and device, computer equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |