CN111274784A - Automatic verification method for air-ground communication repeating semantics based on BilSTM-Attention - Google Patents
Automatic verification method for air-ground communication repeating semantics based on BilSTM-Attention Download PDFInfo
- Publication number
- CN111274784A CN111274784A CN202010042043.1A CN202010042043A CN111274784A CN 111274784 A CN111274784 A CN 111274784A CN 202010042043 A CN202010042043 A CN 202010042043A CN 111274784 A CN111274784 A CN 111274784A
- Authority
- CN
- China
- Prior art keywords
- air
- repeating
- land
- sentence
- bilstm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 85
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012795 verification Methods 0.000 title claims description 27
- 239000013598 vector Substances 0.000 claims abstract description 106
- 230000007246 mechanism Effects 0.000 claims description 26
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 230000014509 gene expression Effects 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 10
- 238000002474 experimental method Methods 0.000 description 10
- 241000282414 Homo sapiens Species 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000003058 natural language processing Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 102100034761 Cilia- and flagella-associated protein 418 Human genes 0.000 description 1
- 101100439214 Homo sapiens CFAP418 gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 208000013409 limited attention Diseases 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
A method for automatically checking the repeating semantics of air-ground communication based on BilSTM-Attention. The method comprises the steps of making a Chinese-English land-air communication repeating corpus; obtaining a word vector sequence of two sentences in each air-ground communication repeating sentence pair; obtaining a feature vector of a land-air communication repeating statement pair output by a BilSTM network; and obtaining and splicing the instruction statement semantic feature vector sent by the controller and the repeating statement semantic feature vector of the pilot, inputting the spliced statement semantic feature vector into the multilayer perceptron, and obtaining the matching score of the statement pair and the classification result of the repeating statement which is consistent or inconsistent. The invention has the advantages that: can catch the contact between land and air conversation instruction and reciting effectively, and then can improve the check-up result, can reduce controller's working strength, improve the check-up precision, have the significance to civil aviation transportation safety.
Description
Technical Field
The invention belongs to the technical field of automatic verification of air-ground call repeating semantics in civil aviation transportation, and particularly relates to an air-ground call repeating semantic automatic verification method based on BilSTM-Attention.
Background
Radio land-air telephony is one of the important ways in which communication between current air traffic controllers and aircraft pilots occurs. Because there are differences of languages, accents, semantic expressions and understanding modes between the members of the land-air conversation and at the same time, the members are influenced by factors of working intensity, mental stress, emotion and the like, radio land-air conversation errors occur. During the actual flight, a seemingly small talk mistake can cause a fatal flight accident. The types of air-ground call errors mainly comprise incorrect content, irregular phrases, careless information, wrong reciting or no reciting. The air-ground call repeating process means that after the controller sends an instruction, the pilot repeats the instruction and feeds back the instruction to the controller, and the controller manually verifies the instruction, so that the working intensity of the controller is increased, and the verification process is easily ignored.
The task of repeated verification of the land-air communication can be summarized into a sentence matching problem in a natural language processing task, and has important application in a plurality of tasks such as text classification, information retrieval, machine translation and the like. In recent years, a Recurrent Neural Network (RNN) in deep learning has good application in processing sequence data, a deformed LSTM and a BilSTM network of the RNN solve the problem of gradient disappearance in the RNN training process and are widely applied to natural language processing, and an Attention mechanism (Attention) can select information which is more key and useful for a current target task from a plurality of information and can capture important matching feature information among texts, so that the Attention mechanism can be widely applied to sentence matching tasks. But no relevant method for combining the BiLSTM network and the attention mechanism for the task of automatically verifying the speech repeating semantics of the air-ground call has been found at present.
Disclosure of Invention
In order to solve the above problems, the present invention aims to provide an automatic verification method for the speech repeating semantics of air-ground calls based on BilSTM-Attention.
In order to achieve the purpose, the automatic verification method of the air-ground call repeating semantics based on the BilSTM-Attention provided by the invention comprises the following steps in sequence:
step 1) making a Chinese-English land-air communication repeating corpus;
step 2) preprocessing the Chinese-English air-land call repeating corpus to obtain word vector sequences of two sentences in each air-land call repeating sentence pair;
step 3) respectively inputting the word vector sequences of the two sentences in each land-air communication repeating sentence pair into two parallel BilSTM networks to obtain the feature vectors of the land-air communication repeating sentence pairs output by the BilSTM networks;
step 4) adding an Attenttion layer behind a BilSTM network output layer, carrying out weight distribution on the feature vector of the air-ground communication repeating statement pair output by the BilSTM network, and finally obtaining an instruction statement semantic feature vector Vp sent by a controller and a repeating statement semantic feature vector V of a pilotATC；
Step 5) the command sentence semantic feature vector Vp sent by the controller and the repeating sentence semantic feature vector V of the pilotATCAnd splicing, and inputting the spliced sentence semantic feature vectors into the multilayer perceptron to obtain the classification result of the matching score and the repeating of the sentence pair which are consistent or inconsistent.
In step 1), the method for making the Chinese-English land-air communication repeating corpus comprises the following steps:
step 1.1) converting the real land-air call recording into a text form, referring to civil aviation air traffic control land-air call standards, selecting a repeating type sentence pair from the text form as a land-air call repeating sentence pair, wherein each repeating sentence pair consists of two sentences, namely an instruction sent by a controller and a repeating sentence of a pilot, and storing the sentence in a text txt format;
step 1.2) taking the land-air communication repeating sentence pair as a sample and marking positive and negative samples according to land-air communication rules and civil aviation bureau related requirements in air traffic controller radio land-air communication, marking the land-air communication repeating sentence pair with correct repeating as a positive sample, and setting a label as 1; marking the land-air communication repeating sentence pair with the repeating error as a negative sample, and setting a label as 0; forming a land-air call repeating corpus by all positive and negative samples;
step 1.3) checking the sample; the air-ground communication repeating corpus comprises a Chinese air-ground communication repeating corpus and an English air-ground communication repeating corpus.
In step 2), the method for preprocessing the Chinese-english air-terrestrial call repeating corpus to obtain the word vector sequences of the two sentences in each air-terrestrial call repeating sentence pair includes:
step 2.1) preprocessing the Chinese air-ground communication repeating corpus, including Chinese word segmentation, word list making and word vector generation; the Chinese word segmentation is to divide a sentence into word sequence forms; then all words in the Chinese air-ground communication repeating language database with the divided words are counted to obtain a word list special for Chinese air-ground communication repeating; training and generating two word vectors of one-hot and word2vec according to the obtained special word list for Chinese air-land communication repeating to obtain a word vector sequence of each sentence;
step 2.2) preprocessing the English air-land call repeating corpus, including stem extraction, conversion into lower case, word list making and word vector generation; the stem extraction process is a process of removing affixes to obtain roots, namely, the original form of a word is to be found, and words in different forms are mapped into the same stem; the used stemming extraction method is a step algorithm carried by an NLTK library in PYTHON; all texts in the English land-air communication repeating corpus after the word stem is extracted are converted into lowercase, so that the repeating check task cannot be influenced by the capitalization reason; then all words in the English land-air communication repeating corpus after conversion into the sketch are counted to obtain a word list special for English land-air communication repeating; finally, training and generating two word vectors of one-hot and word2vec according to the obtained special word list for English air-land communication repeating to obtain a word vector sequence of each sentence;
thus, the instruction sequence sent by the controller is obtained
Repeating sequence with pilot
Wherein xiFor the word vector of the ith word, L1 and L2 respectively are the sentence lengths of the commands sent by the controller and the repeat of the pilot, the longest length N of all sentences is taken as the length of the input sentence, and sentences smaller than the sentence length are filled up by using a zero filling method.
In step 3), the method for obtaining the feature vector of the air-ground communication repeating sentence pair output by the BiLSTM network by inputting the word vector sequences of the two sentences in each air-ground communication repeating sentence pair into the two parallel BiLSTM networks respectively comprises the following steps:
the calculation formula of the feature vector of each state of the LSTM network is as follows:
i(t)＝σ(W3x(t)+Wrec3h(t-1)+Wp2c(t-1)+bi)(1)
f(t)＝σ(W2x(t)+Wrec2h(t-1)+Wp3c(t-1)+bf)(2)
o(t)＝σ(W1x(t)+Wrec1h(t-1)+Wp1c(t-1)+bo)(3)
l(t)＝tanh(W4x(t)+Wrec4h(t-1)+bl) (4)
c(t)＝f(t)c(t-1)+i(t)l(t)(5)
h(t)＝tanh(c(t))o(t) (6)
wherein, WiAnd Wreci(i ═ 1,2,3,4) are the connection weight matrices of the inputs and outputs of the input gate, the forgetting gate, the output gate and the memory cell, respectively, (. sigma.) is the sigmoid activation function, bi、bf、bo、blIs a bias term;
the BilSTM network extracts the characteristics of the statement to obtain the calculation process of the characteristic vector of each moment of the statement as shown in the formulas (7) to (9):
In step 4), adding an Attention layer behind a BilSTM network output layer, performing weight distribution on the feature vector of the air-ground communication repeating statement pair output by the BilSTM network, and finally obtaining an instruction statement semantic feature vector Vp sent by a controller and a repeating statement semantic feature vector V of a pilotATCThe method comprises the following steps:
the calculation formula of the attention mechanism structure is shown in formulas (10) to (12):
ut＝tanh(Wwht+bw)(10)
wherein, WwAnd bwAre respectively asAttention to the weight and offset of the mechanism, atV is a sentence semantic feature vector obtained by weighted summation for the calculated weight at the time t; firstly, the feature vector h of each time instant output by the BilSTM networktObtaining its implicit representation u by means of a non-linear transformation tanh (-)tAnd randomly initializing to generate attention mechanism uw(ii) a Then, for the implicit representation utAttention mechanism uwAfter the dot product operation is carried out, normalization operation is carried out by utilizing a softmax function to obtain the weight output by the word-level BilTM network at each moment, namely the weight W of the attention mechanismw。
The automatic verification method of the air-ground communication repeating semantics based on the BilSTM-Attention provided by the invention has the following advantages:
the BilSTM network can automatically learn the semantic features of the sentence sequence and extract the advantages of rich context semantic features, so that the overhead of manually designing features can be avoided, the extracted feature vectors are weighted by an attention mechanism, useful information for the verification result is reserved, and useless information for the verification result is inhibited, so that the matching verification task between the land-air call repeated sentence pairs can be better realized. The method can effectively capture the relation between the land-air communication instruction and the repeating, further improve the verification result, reduce the working strength of a controller, improve the verification precision and have important significance on the civil aviation transportation safety.
Drawings
FIG. 1 is a flow chart of an automatic verification method for air-ground call repeating semantics based on BilSTM-Attention provided by the invention.
Fig. 2 is a time-series development diagram of a conventional RNN.
Fig. 3 is a diagram of an LSTM network architecture.
Fig. 4 is a diagram of a BiLSTM network architecture.
Fig. 5 is a view of the attention mechanism.
FIG. 6 is a block diagram of an automatic verification method of air-ground call repeating semantics based on BilSTM-Attention.
FIG. 7 is a block diagram of the semantic check of the air-to-ground call repetition using the traditional Siemese-RNN/LSTM model.
FIG. 8 is a block diagram of a speech over air duplication check using the Siamese-BilSTM model.
Detailed Description
The automatic checking method for the air-ground call repeating semantic based on BilSTM-Attention provided by the invention is described in detail below with reference to the accompanying drawings and specific embodiments.
As shown in FIG. 1, the automatic verification method for the air-ground speech repeating semantics based on BilSTM-authorization provided by the invention comprises the following steps in sequence:
step 1) making a Chinese-English land-air communication repeating corpus;
step 1.1) converting the real land-air call recording into a text form, referring to civil aviation air traffic control land-air call standards, selecting a repeating type sentence pair from the text form as a land-air call repeating sentence pair, wherein each repeating sentence pair consists of two sentences, namely an instruction sent by a controller and a repeating sentence of a pilot, and storing the sentence in a text txt format;
step 1.2) taking the land-air communication repeating sentence pair as a sample and marking positive and negative samples according to land-air communication rules and civil aviation bureau related requirements in air traffic controller radio land-air communication, marking the land-air communication repeating sentence pair with correct repeating as a positive sample, and setting a label as 1; marking the land-air communication repeating sentence pair with the repeating error as a negative sample, and setting a label as 0; forming a land-air call repeating corpus by all positive and negative samples;
step 1.3) the sample is verified, so that the instruction coverage in the corpus is more comprehensive, the sample types are richer, and the sample is more practical;
the air-ground communication repeating corpus comprises a Chinese air-ground communication repeating corpus and an English air-ground communication repeating corpus.
Step 2) preprocessing the Chinese-English air-land call repeating corpus to obtain word vector sequences of two sentences in each air-land call repeating sentence pair;
step 2.1) preprocessing the Chinese air-ground communication repeating corpus, including Chinese word segmentation, word list making and word vector generation; the Chinese word segmentation is to divide the sentence into word sequence forms for the convenience of processing the material library by a computer; then all words in the Chinese air-ground communication repeating language database with the divided words are counted to obtain a word list special for Chinese air-ground communication repeating; training and generating two word vectors of one-hot and word2vec according to the obtained special word list for Chinese air-land communication repeating to obtain a word vector sequence of each sentence;
step 2.2) preprocessing the English air-land call repeating corpus, including stem extraction, conversion into lower case, word list making and word vector generation; the stem extraction process is a process of removing affixes to obtain roots, namely, the original form of a word is to be found, and words in different forms are mapped into the same stem; the stem extraction method used in the invention is a stemming algorithm carried by an NLTK library in PYTHON; the English word letters have capital and small case marks, so that all texts in the English land-air communication repeated language database after the stem extraction are required to be converted into lowercase, and repeated verification tasks cannot be influenced by capital and small case reasons; then all words in the English land-air communication repeating corpus after conversion into the sketch are counted to obtain a word list special for English land-air communication repeating; finally, training and generating two word vectors of one-hot and word2vec according to the obtained special word list for English air-land communication repeating to obtain a word vector sequence of each sentence;
wherein, a one-hot word vector represents a word by using a vector, the dimension of the vector is equal to the size of a word list, wherein, the value of only one dimension is 1, and the rest are 0. The position of 1 represents the position of the word in the vocabulary; the word2vec word vector is a word vector obtained by training by using a deep learning method based on a neural network language model. The word2vec word vector can convert a one-hot word vector into a low-dimensional continuous value, namely a dense vector, and words with similar meanings are mapped to similar positions in a vector space, so that the semantic relationship between two words can be measured.
Thus, the instruction sequence sent by the controller can be obtained
Repeating sequence with pilot
Wherein xiFor the word vector of the ith word, L1 and L2 are respectively the sentence length of the command sent by the controller and the repeat of the pilot, and for the convenience of the processing of the following step 3), the invention takes the longest length N of all sentences as the length of the input sentence, and sentences smaller than the sentence length are filled up by using a zero filling method.
Step 3) respectively inputting the word vector sequences of the two sentences in each land-air communication repeating sentence pair into two parallel BilSTM networks to obtain the feature vectors of the land-air communication repeating sentence pairs output by the BilSTM networks;
sending the command sequence of the controller in each air-ground call repeating statement
Repeating sequence with pilot
Respectively input into two parallel BilSTM networks to obtain the characteristic vector expressions of the command sent by the controller and the repeat sequence of the pilot
And
wherein h istThe feature vector output at the moment t in the statement.
The BilSTM network described above is a variation of the LSTM network. FIG. 2 is an expanded view of a conventional RNN in time series, and it can be seen that the RNN forward propagation process is similar to a generic perceptron, except that the state of the hidden layer of the RNN at the current time is not only related to the input at the current time, but also related to the hidden layer output at the previous time. The LSTM network is a modified structure of RNN, the input state of the LSTM network is determined by the input of the node at the time t and the output state of the node at the time t-1, and the difference is that the LSTM network adds an LSTM unit in the hidden layer structure of RNN. The structure of the LSTM network is shown in fig. 3. It can be seen that the LSTM network is added with three gate units of an input gate, an output gate and a forgetting gate and a memory cell, where the memory cell is a core of the LSTM network and is used for storing history information. It is because of these elements that the LSTM network is able to extract information over longer distances. The calculation formula of the feature vector of each state of the LSTM network is as follows:
i(t)＝σ(W3x(t)+Wrec3h(t-1)+Wp2c(t-1)+bi)(1)
f(t)＝σ(W2x(t)+Wrec2h(t-1)+Wp3c(t-1)+bf)(2)
o(t)＝σ(W1x(t)+Wrec1h(t-1)+Wp1c(t-1)+bo)(3)
l(t)＝tanh(W4x(t)+Wrec4h(t-1)+bl) (4)
c(t)＝f(t)c(t-1)+i(t)l(t)(5)
h(t)＝tanh(c(t))o(t) (6)
wherein, WiAnd Wreci(i ═ 1,2,3,4) are the connection weight matrices of the inputs and outputs of the input gate, the forgetting gate, the output gate and the memory cell, respectively, (. sigma.) is the sigmoid activation function, bi、bf、bo、blIs the bias term.
The one-way LSTM network is trained from front to back, and semantic information after time t cannot be learned, so there is a problem of positional bias, i.e. the latter words are more important than the former words. But all words from front to back in the controllers 'commands and pilot's repeat are equally important in the air-ground talk repeat semantic consistency auto-check mission. Therefore, the invention uses a bidirectional long-time memory cycle network (BilSTM network) to extract the semantic features of the sentences. The structure diagram of the BilTM network is shown in FIG. 4, which is a combination of two opposite LSTM networks, a forward LSTM network and a backward LSTM network process the input word vector sequence from the beginning and the end respectively, the forward LSTM network is used to capture the above feature information, the backward LSTM network is used to capture the below feature information, the output is a combination of two LSTM network outputs, and the output at each time contains the complete past and future context information corresponding to that time in the input word vector sequence. The BilSTM network extracts the characteristics of the statement to obtain the calculation process of the characteristic vector of each moment of the statement as shown in the formulas (7) to (9):
compared with a unidirectional LSTM network, the BilTM network can extract richer semantic information at each moment, so that a better statement feature representation can be constructed.
Step 4) adding Attention behind the output layer of the BilSTM networkn layers, weight distribution is carried out on the characteristic vector of the air-ground communication repeating statement pair output by the BilSTM network, so as to highlight the important information useful for verification of the air-ground communication repeating statement, inhibit the useless characteristic information, and finally obtain the instruction statement semantic characteristic vector Vp sent by the controller and the repeating statement semantic characteristic vector V of the pilotATC；
The Attention Mechanism (Attention Mechanism) is proposed according to the human visual Attention Mechanism, human vision obtains a target area needing important Attention by rapidly scanning a global image, namely a focus of Attention, and then puts more Attention resources into the area to obtain more detailed information of the target needing Attention and suppress other useless information, which is a means for rapidly screening high-value information from a large amount of information by using limited Attention resources by human beings, is a survival Mechanism formed in the long-term evolution of human beings, and greatly improves the efficiency and accuracy of visual information processing. Based on the advantages of the attention mechanism, the method is widely applied to image processing and natural language processing tasks at present, and achieves outstanding results. Therefore, the invention introduces Attention mechanism (Attention) to perform the air-ground call repeating and checking work.
The attention mechanism in deep learning is similar to the selective visual attention mechanism of human beings in nature, and the core target is to select information which is more critical to the current task target from a plurality of information. The attention mechanism used in the present invention is shown in fig. 5, and the calculation formulas are shown in formulas (10) to (12):
ut＝tanh(Wwht+bw)(10)
wherein, WwAnd bwRespectively as attention mechanismWeight and offset of atV is a sentence semantic feature vector obtained by weighted summation for the calculated weight at the time t. Firstly, the feature vector h of each time instant output by the BilSTM networktObtaining its implicit representation u by means of a non-linear transformation tanh (-)tAnd randomly initializing to generate attention mechanism uw(ii) a Then, for the implicit representation utAttention mechanism uwAfter the dot product operation is carried out, normalization operation is carried out by utilizing a softmax function to obtain the weight output by the word-level BilTM network at each moment, namely the weight W of the attention mechanismw。
Step 5) the command sentence semantic feature vector Vp sent by the controller and the repeating sentence semantic feature vector V of the pilotATCSplicing, and inputting the spliced sentence semantic feature vectors into a multilayer perceptron (MLP) to obtain a classification result with matching scores and repeating numbers of sentence pairs consistent or inconsistent;
FIG. 6 is a structural diagram of an automatic verification method for the air-ground communication repeating semantics based on BilSTM-Attention provided by the present invention.
Experiments and analyses
The experimental data of the invention comprises a Chinese air-land call repeating corpus and an English air-land call repeating corpus. The Chinese air-ground communication repeating corpus comprises 5000 repeating sentence pairs, wherein the positive sample number is 2500 pairs (the sentence semantics of the controller instruction and the pilot repeating are consistent), and the negative sample number is 2500 pairs (the sentence semantics of the controller instruction and the pilot repeating are inconsistent); the English air-land call repeating corpus comprises 2500 repeating sentence pairs, wherein the positive sample number is 1500 pairs, and the negative sample number is 1000 pairs. The corpus covers various phases of aircraft flight and control. In each experiment, the land-air call repeating corpus is randomly divided into a training set, a verification set and a test set according to the ratio of 8:1:1 for model training and testing.
In order to measure the performance of the method of the invention more accurately, the inventor conducts 30 times of random training and testing on the Chinese and English air-land call repeating corpus, calculates the testing precision of each time and records the testing precision. Test essenceThe formula of degree (Acc) is shown as formula (13), after 30 experiments, the inventor utilizes the average test precision of 30 random experiments
The effectiveness of the method of the present invention was evaluated by the calculation formula shown in formula (14).
Wherein Acc is the test precision value of one experiment, NaccThe number of correct samples, N, is judged in the testing processallFor the number of all samples involved in the test, N is the total number of experiments, and the value of N in the present invention is 30, so
The average test precision is calculated after N times of experiments.
In order to verify the stability of the method, the inventor calculates the mean square error of the test precision of 30 random experiments, and the calculation formula of the mean square error (Mse) is shown as the formula (15):
to verify the classification effect of the method of the present invention, F1 values were calculated, and F1 values were calculated from the accuracy (P) and recall (R). The accuracy rate can be described as the proportion of the number of positive samples with correct prediction to the number of samples with positive prediction results, and the recall rate can be described as the proportion of the number of positive samples with correct prediction to the number of actual positive samples. Ideally, the higher the accuracy and the better the recall rate are, but in fact, the accuracy and the recall rate are contradictory under certain conditions, and the recall rate is high and is very low, whereas the accuracy is low and the recall rate may be very high. The results of the two can be combined by calculating the F1 value, and thus the F1 value can be used for evaluating the performance of the model as a whole. The calculation formula of the F1 value is shown in equation (16):
in order to verify the performance of the method, the inventor compares the experimental results of the traditional two-channel statement semantic matching model (Siamese-RNN/LSTM/BiLSTM) on the task of automatically checking the repeated semantics of the air-ground call. FIG. 7 is a block diagram of semantic checks for speech retrieval over the air using the traditional Siemese-RNN/LSTM model. The model uses two parallel LSTM networks to map a land-air conversation instruction and a repeating instruction statement pair to the same semantic space, two semantic vectors are respectively obtained to be used as the expression of the whole statement, and then the matching score between the two semantic vectors is directly calculated to obtain the matching score between the instruction and the repeating instruction, so that whether the repeating semantics are consistent or not is judged. FIG. 8 is a block diagram of a speech over air duplication check using the Siamese-BilSTM model.
The inventor respectively performs 30 experiments on the method and the traditional double-channel Siamese-RNN/LSTM/LSTM model method, and calculates
Mse and F1 values. Table 1 and Table 2 show the experimental results of the method of the present invention and the conventional Simase-RNN/LSTM/BiLSTM model when word2vec word vectors and one-hot word vectors are used as model inputs, respectively.
TABLE 1 air-ground talking repeating semantic check experiment result with Word2vec Word vector as input
TABLE 2 semantic verification test results of air-ground talk using one-hot word vectors as input
As can be seen from the experimental results of tables 1 and 2: compared with the traditional double-channel Siamese-RNN/LSTM/BiLSTM model, the method provided by the invention has the advantages of higher test accuracy and better stability. The introduction of the attention mechanism in the BilSTM network is effective for repeating semantic check results, and the semantic relation between the instruction and the repeated instruction can be well matched through the attention mechanism. Meanwhile, the experimental part respectively compares the one-hot word vector and the word2vec word vector as the experimental results obtained by model input, so that the experimental result of the word2vec word vector is better, and the word2vec word vector contains the semantic relation of words, so that the matching result is more favorable.
The experimental results show that the method is effective for checking tasks, the performance of the method is superior to that of the traditional Simese-RNN/LSTM/BiLSTM model, and the practicability and the feasibility of the method are also proved.
Claims (5)
1. A land-air communication repeating semantic automatic checking method based on BilSTM-Attention is characterized by comprising the following steps: the automatic verification method for the air-ground call repeating semantics based on the BilSTM-Attention comprises the following steps in sequence:
step 1) making a Chinese-English land-air communication repeating corpus;
step 2) preprocessing the Chinese-English air-land call repeating corpus to obtain word vector sequences of two sentences in each air-land call repeating sentence pair;
step 3) respectively inputting the word vector sequences of the two sentences in each land-air communication repeating sentence pair into two parallel BilSTM networks to obtain the feature vectors of the land-air communication repeating sentence pairs output by the BilSTM networks;
step 4) adding an Attenttion layer behind a BilSTM network output layer, carrying out weight distribution on the feature vector of the air-ground communication repeating statement pair output by the BilSTM network, and finally obtaining an instruction statement semantic feature vector Vp sent by a controller and a repeating statement semantic feature vector V of a pilotATC；
Step 5) the instruction sentence semantic feature vector Vp sent by the controller and the complex of the pilotRecite sentence semantic feature vector VATCAnd splicing, and inputting the spliced sentence semantic feature vectors into the multilayer perceptron to obtain the classification result of the matching score and the repeating of the sentence pair which are consistent or inconsistent.
2. The BiLSTM-attachment based automatic verification method for air-ground call repeating semantics of claim 1, wherein: in step 1), the method for making the Chinese-English land-air communication repeating corpus comprises the following steps:
step 1.1) converting the real land-air call recording into a text form, referring to civil aviation air traffic control land-air call standards, selecting a repeating type sentence pair from the text form as a land-air call repeating sentence pair, wherein each repeating sentence pair consists of two sentences, namely an instruction sent by a controller and a repeating sentence of a pilot, and storing the sentence in a text txt format;
step 1.2) taking the land-air communication repeating sentence pair as a sample and marking positive and negative samples according to land-air communication rules and civil aviation bureau related requirements in air traffic controller radio land-air communication, marking the land-air communication repeating sentence pair with correct repeating as a positive sample, and setting a label as 1; marking the land-air communication repeating sentence pair with the repeating error as a negative sample, and setting a label as 0; forming a land-air call repeating corpus by all positive and negative samples;
step 1.3) checking the sample; the air-ground communication repeating corpus comprises a Chinese air-ground communication repeating corpus and an English air-ground communication repeating corpus.
3. The BiLSTM-attachment based automatic verification method for air-ground call repeating semantics of claim 1, wherein: in step 2), the method for preprocessing the Chinese-english air-terrestrial call repeating corpus to obtain the word vector sequences of the two sentences in each air-terrestrial call repeating sentence pair includes:
step 2.1) preprocessing the Chinese air-ground communication repeating corpus, including Chinese word segmentation, word list making and word vector generation; the Chinese word segmentation is to divide a sentence into word sequence forms; then all words in the Chinese air-ground communication repeating language database with the divided words are counted to obtain a word list special for Chinese air-ground communication repeating; training and generating two word vectors of one-hot and word2vec according to the obtained special word list for Chinese air-land communication repeating to obtain a word vector sequence of each sentence;
step 2.2) preprocessing the English air-land call repeating corpus, including stem extraction, conversion into lower case, word list making and word vector generation; the stem extraction process is a process of removing affixes to obtain roots, namely, the original form of a word is to be found, and words in different forms are mapped into the same stem; the used stemming extraction method is a step algorithm carried by an NLTK library in PYTHON; all texts in the English land-air communication repeating corpus after the word stem is extracted are converted into lowercase, so that the repeating check task cannot be influenced by the capitalization reason; then all words in the English land-air communication repeating corpus after conversion into the sketch are counted to obtain a word list special for English land-air communication repeating; finally, training and generating two word vectors of one-hot and word2vec according to the obtained special word list for English air-land communication repeating to obtain a word vector sequence of each sentence;
thus, the instruction sequence sent by the controller is obtained
Repeating sequence with pilot
Wherein xiFor the word vector of the ith word, L1 and L2 respectively are the sentence lengths of the commands sent by the controller and the repeat of the pilot, the longest length N of all sentences is taken as the length of the input sentence, and sentences smaller than the sentence length are filled up by using a zero filling method.
4. The BiLSTM-attachment based automatic verification method for air-ground call repeating semantics of claim 1, wherein: in step 3), the method for obtaining the feature vector of the air-ground communication repeating sentence pair output by the BiLSTM network by inputting the word vector sequences of the two sentences in each air-ground communication repeating sentence pair into the two parallel BiLSTM networks respectively comprises the following steps:
the calculation formula of the feature vector of each state of the LSTM network is as follows:
i(t)＝σ(W3x(t)+Wrec3h(t-1)+Wp2c(t-1)+bi) (1)
f(t)＝σ(W2x(t)+Wrec2h(t-1)+Wp3c(t-1)+bf) (2)
o(t)＝σ(W1x(t)+Wrec1h(t-1)+Wp1c(t-1)+bo) (3)
l(t)＝tanh(W4x(t)+Wrec4h(t-1)+bl) (4)
c(t)＝f(t)c(t-1)+i(t)l(t) (5)
h(t)＝tanh(c(t))o(t) (6)
wherein, WiAnd Wreci(i ═ 1,2,3,4) are the connection weight matrices of the inputs and outputs of the input gate, the forgetting gate, the output gate and the memory cell, respectively, (. sigma.) is the sigmoid activation function, bi、bf、bo、blIs a bias term;
the BilSTM network extracts the characteristics of the statement to obtain the calculation process of the characteristic vector of each moment of the statement as shown in the formulas (7) to (9):
5. The BiLSTM-attachment based automatic verification method for air-ground call repeating semantics of claim 1, wherein: in step 4), adding an Attention layer behind a BilSTM network output layer, performing weight distribution on the feature vector of the air-ground communication repeating statement pair output by the BilSTM network, and finally obtaining an instruction statement semantic feature vector Vp sent by a controller and a repeating statement semantic feature vector V of a pilotATCThe method comprises the following steps:
the calculation formula of the attention mechanism structure is shown in formulas (10) to (12):
ut＝tanh(Wwht+bw) (10)
wherein, WwAnd bwWeight and offset, a, of attention mechanism, respectivelytV is a sentence semantic feature vector obtained by weighted summation for the calculated weight at the time t; first, the respective time of the output of the BilSTM networkCharacteristic vector h oftObtaining its implicit representation u by means of a non-linear transformation tanh (-)tAnd randomly initializing to generate attention mechanism uw(ii) a Then, for the implicit representation utAttention mechanism uwAfter the dot product operation is carried out, normalization operation is carried out by utilizing a softmax function to obtain the weight output by the word-level BilTM network at each moment, namely the weight W of the attention mechanismw。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010042043.1A CN111274784A (en) | 2020-01-15 | 2020-01-15 | Automatic verification method for air-ground communication repeating semantics based on BilSTM-Attention |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010042043.1A CN111274784A (en) | 2020-01-15 | 2020-01-15 | Automatic verification method for air-ground communication repeating semantics based on BilSTM-Attention |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111274784A true CN111274784A (en) | 2020-06-12 |
Family
ID=70999031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010042043.1A Pending CN111274784A (en) | 2020-01-15 | 2020-01-15 | Automatic verification method for air-ground communication repeating semantics based on BilSTM-Attention |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111274784A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113053366A (en) * | 2021-03-12 | 2021-06-29 | 中国电子科技集团公司第二十八研究所 | Controlled voice repeat consistency checking method based on multi-mode fusion |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920464A (en) * | 2018-07-03 | 2018-11-30 | 中国民航大学 | A kind of land sky call based on CNN is rehearsed error classification method |
| CN109214002A (en) * | 2018-08-27 | 2019-01-15 | 成都四方伟业软件股份有限公司 | A kind of transcription comparison method, device and its computer storage medium |
| CN109299262A (en) * | 2018-10-09 | 2019-02-01 | 中山大学 | A kind of text implication relation recognition methods for merging more granular informations |
| CN109522555A (en) * | 2018-11-16 | 2019-03-26 | 中国民航大学 | A kind of land sky call based on BiLSTM is rehearsed semantic automatic Verification method |
-
2020
- 2020-01-15 CN CN202010042043.1A patent/CN111274784A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920464A (en) * | 2018-07-03 | 2018-11-30 | 中国民航大学 | A kind of land sky call based on CNN is rehearsed error classification method |
| CN109214002A (en) * | 2018-08-27 | 2019-01-15 | 成都四方伟业软件股份有限公司 | A kind of transcription comparison method, device and its computer storage medium |
| CN109299262A (en) * | 2018-10-09 | 2019-02-01 | 中山大学 | A kind of text implication relation recognition methods for merging more granular informations |
| CN109522555A (en) * | 2018-11-16 | 2019-03-26 | 中国民航大学 | A kind of land sky call based on BiLSTM is rehearsed semantic automatic Verification method |
Non-Patent Citations (1)
| Title |
|---|
| 杨飞: ""基于LSTM的文本相似度识别方法研究"", 《中国知网》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113053366A (en) * | 2021-03-12 | 2021-06-29 | 中国电子科技集团公司第二十八研究所 | Controlled voice repeat consistency checking method based on multi-mode fusion |
| CN113053366B (en) * | 2021-03-12 | 2023-11-21 | 中国电子科技集团公司第二十八研究所 | Multi-mode fusion-based control voice duplicate consistency verification method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368565B (en) | Text translation method, text translation device, storage medium and computer equipment | |
| CN108614875B (en) | Chinese emotion tendency classification method based on global average pooling convolutional neural network | |
| CN109359293B (en) | Mongolian name entity recognition method neural network based and its identifying system | |
| CN106547737B (en) | Sequence labeling method in natural language processing based on deep learning | |
| CN111966812B (en) | Automatic question answering method based on dynamic word vector and storage medium | |
| CN112990296B (en) | Image-text matching model compression and acceleration method and system based on orthogonal similarity distillation | |
| CN110019843A (en) | The processing method and processing device of knowledge mapping | |
| CN111008293A (en) | Visual question-answering method based on structured semantic representation | |
| CN111475655B (en) | Power distribution network knowledge graph-based power scheduling text entity linking method | |
| CN110222178A (en) | Text sentiment classification method, device, electronic equipment and readable storage medium storing program for executing | |
| CN110210019A (en) | A kind of event argument abstracting method based on recurrent neural network | |
| CN108959260B (en) | A kind of Chinese grammer error-detecting method based on textual term vector | |
| CN110717341B (en) | Method and device for constructing old-Chinese bilingual corpus with Thai as pivot | |
| CN109711465A (en) | Image method for generating captions based on MLL and ASCA-FR | |
| CN111125333B (en) | Generation type knowledge question-answering method based on expression learning and multi-layer covering mechanism | |
| CN105893354A (en) | Word segmentation method based on bidirectional recurrent neural network | |
| CN110334196B (en) | Neural network Chinese problem generation system based on strokes and self-attention mechanism | |
| CN108984525A (en) | A kind of Chinese grammer error-detecting method based on the term vector that text information is added | |
| CN110276069A (en) | A kind of Chinese braille mistake automatic testing method, system and storage medium | |
| CN113536804B (en) | Natural language feature extraction method based on keyword enhancement GRU and Kronecker | |
| CN106227718A (en) | Land based on CNN sky call semantic consistency method of calibration | |
| Lu et al. | A new method for semantic consistency verification of aviation radiotelephony communication based on LSTM-RNN | |
| CN114925195A (en) | Standard content text abstract generation method integrating vocabulary coding and structure coding | |
| CN114254645A (en) | Artificial intelligence auxiliary writing system | |
| Juven et al. | Cross-situational learning with reservoir computing for language acquisition modelling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200612 |