CN115510869A - An End-to-End Tibetan Lag Shallow Semantic Analysis Method - Google Patents

Abstract

The invention relates to the technical field of Tibetan La lattice shallow semantic analysis, in particular to an end-to-end Tibetan La lattice shallow semantic analysis method, which comprises the following steps: 1. mapping the input character sequence taking the word as a unit and the corresponding mark sequence into a low-dimensional real-valued vector; 2. a gated high-speed connection mechanism GM is arranged in the vertical direction of the LSTM, and the timing sequence characteristics and the context semantic information of the input sentence are learned by adopting the BilSTM; the GM contains linear connections to the cell internal inputs and outputs, allowing information to propagate unobstructed between different layers; 3. calculating local normalized distribution of the semantic label at each moment by using softmax, so as to be used for constraint decoding by an output layer; 4. and when the Viterbi algorithm is used for decoding, the structural relationship between the output semantic labels is normalized by forcibly executing the set BIO and La lattice shallow semantic labeling constraints. The method can better perform the Tibetan La grid shallow semantic analysis.

Description

An End-to-End Tibetan Lag Shallow Semantic Analysis Method

technical field

The present invention relates to the technical field of Tibetan Lag shallow semantic analysis, in particular to an end-to-end Tibetan Lag shallow semantic analysis method.

Background technique

The goal of Tibetan Lag shallow semantic analysis is to first find out the predicate of a given sentence, and then use the predicate as the core to determine the relevant main semantic components and mark the corresponding semantic labels, and then obtain the type of the given sentence and its various meanings. Semantic roles played by semantic components.

中的主要研究内容，所以面向自然语言处理，研究基于机器学习方法的藏文La格浅层语义分析技术可以对很多上层藏语自然语言理解任务起到帮助作用，比如语义角色标注、语义分析、信息抽取、自动问答、阅读理解、机器翻译等。此外，La格是藏语文课本中必学的一个重点知识，唯有熟练掌握其概念和用法，才能准确区分藏文La的类型，找准每个句子的主要语义成分，并进一步分析每个句子的实际含义。可见研究基于机器学习方法的藏文La格浅层语义分析技术在La格的学习中也具备一定的实际应用价值。Lage is the key and difficult point in the Tibetan grammar classics "Thirty Songs" and "Sound Potential", and it is also an eight-form

Therefore, for natural language processing, the study of Tibetan Lag shallow semantic analysis technology based on machine learning methods can help many upper-level Tibetan natural language understanding tasks, such as semantic role labeling, semantic analysis, Information extraction, automatic question answering, reading comprehension, machine translation, etc. In addition, Lage is a key knowledge that must be learned in Tibetan textbooks. Only by mastering its concept and usage can we accurately distinguish the types of Tibetan La, identify the main semantic components of each sentence, and further analyze each sentence actual meaning. It can be seen that the study of Tibetan Lague shallow semantic analysis technology based on machine learning methods also has certain practical application value in Lague learning.

Traditional shallow semantic analysis tasks are closely related to syntactic analysis and rely heavily on syntactic analysis results, which leads to increased complexity of shallow semantic analysis. In recent years, with the continuous maturity of deep learning technology, end-to-end models without syntactic input have achieved good results on shallow semantic analysis tasks. Some literatures have studied the end-to-end LSTM shallow semantic analysis method, which has achieved better results than the traditional method of introducing syntactic information. The success of these end-to-end models reveals the potential ability of LSTM to deal with the latent syntactic structure of sentences. The improvement provides a theoretical basis and a reference basis. At present, there is no reference to the Tibetan shallow semantic analysis method based on the end-to-end model, let alone the literature report on the Lag shallow semantic analysis.

Contents of the invention

The content of the present invention is to provide an end-to-end Tibetan Laga shallow semantic analysis method, which can overcome some or some defects of the prior art.

A kind of end-to-end Tibetan LaGa shallow semantic analysis method according to the present invention, it comprises the following steps:

1. Map the input feature sequence with word as the unit and the corresponding tag sequence into a low-dimensional real-valued vector;

2. Install the gated high-speed connection mechanism GM in the vertical direction of LSTM, and use BiLSTM to learn the timing characteristics and contextual semantic information of the input sentence; GM includes linear connections to the input and output of the unit, so that information can be smoothly transferred between different layers Spread between;

3. Use softmax to calculate the local normalized distribution of the semantic label at each moment for the output layer to perform constraint decoding;

4. By enforcing the set BIO and La lattice shallow semantic labeling constraints when using the Viterbi algorithm for decoding, standardize the structural relationship between the output semantic tags.

表示已训练好的GloVe词向量，用V表示词汇表，

用C∈{0,1}表示标记集合，则最原始的输入序列{w₁,w₂,…,w_T}和标记序列{m₁,m₂,…,m_T}通过查找表lookup table映射成低维实值向量e(w_t)和e(m_t)，其中w_t∈V和对应标记m_t∈C；至此，可将向量e(w_t)和e(m_t)拼接成x_l,t作为LSTM第一层的输入：As a preference, in step one, use

Represents the trained GloVe word vector, and V represents the vocabulary,

Use C ∈ {0,1} to represent the tag set, then the most original input sequence {w ₁ ,w ₂ ,…,w _T } and the tag sequence {m ₁ ,m ₂ ,…,m _T } pass the lookup table lookup table Mapped into low-dimensional real-valued vectors e(w _t ) and e(m _t ), where w _t ∈ V and the corresponding label m _t ∈ C; so far, the vectors e(w _t ) and e(m _t ) can be spliced into x _{l, t is} used as the input of the first layer of LSTM:

x _l,t ＝[e(w _t ),e(m _t )]

Among them, x _l,t is the input to the LSTM at the moment t of the first layer, where l=1, t=[1,T].

Preferably, in step 2, use the first LSTM to process the input sentence forward, and then use the output of this layer as the input of the next layer for reverse processing, in order to improve the learning ability of time series features and fully obtain the context of each moment Semantic information lays the foundation; LSTM is defined as follows:

Among them, δ _l represents the direction of the l-th layer LSTM. When δ _l = -1, the direction of LSTM is forward, and when δ _l = 1, the direction is reverse;

To stack LSTMs in interleaved mode, layer-specific inputs xl _,t and orientation parameters _δl are arranged in the following way:

The input vector xl _,t is the concatenation of the word embedding of the character w _t and the embedding representing whether the word of w _t is a binary feature (t=v) for a given predicate.

As a preference, in step 2, the linear and nonlinear transformation weights between the layers are controlled by installing GM in the vertical direction of the LSTM, which functions to balance the transmission of information in the vertical direction; use λ _{l, t} to represent the gating device of GM , then the output h _l,t of the hidden layer after using GM can be changed to:

h _l ′ _,t = LSTM(h _l-1,t ,h _l,t-1 ).

Preferably, in step 2, in order to reduce overfitting, the dropout rate Dropout is used, and the shared Dropout mask D _l is applied to the hidden state:

Input the feature sequence x={w ₁ ,w ₂ ,…,w _n } of a La-case sentence, and the log likelihood of the corresponding correct semantic label sequence y={y ₁ ,y ₂ ,…,y _n } is:

Preferably, in step 3, according to the hidden state h _l,t of the model, softmax can be used to calculate the local normalized distribution on the output semantic label y _t :

是Kronecker delta，维度与语义标签的个数一致；模型训练目标是在给定输入的时候最大化正确标签的概率。W _o in the above formula is the parameter matrix of softmax,

Is the Kronecker delta, the dimension is consistent with the number of semantic labels; the model training goal is to maximize the probability of the correct label when given the input.

The present invention first learns from the design idea of LSTM, and balances the transmission of information in the vertical direction by installing a gated high-speed connection mechanism (GM) in the vertical direction of the LSTM. Through GM, information can spread more smoothly in space and time dimensions, and there is only a small loss of information. The most important thing is that GM contains a "gating" function that can dynamically select or ignore the propagation of information in the vertical direction, so that different levels of abstract representation can be more conveniently passed to the output layer. Then softmax is used to calculate the local normalized distribution of semantic labels at each moment for constrained decoding at the output layer. Finally, when using the Viterbi algorithm for decoding, by enforcing the BIO and La lattice shallow semantic labeling constraints set in this paper, the structural relationship between the output semantic tags is standardized, and the accuracy of the final predicted semantic tags is improved.

Description of drawings

Fig. 1 is the shallow layer semantic analysis model architecture schematic diagram of La lattice in the embodiment;

Fig. 2 is the difference diagram of GM and LSTM in the embodiment;

Fig. 3 is the schematic diagram of the influence of GM on experimental performance in the embodiment;

FIG. 4 is a schematic diagram of the influence of constraint decoding on experimental performance in an embodiment;

Fig. 5 is a schematic diagram of the influence of the time series feature learning method on the experimental performance in the embodiment.

detailed description

In order to further understand the content of the present invention, the present invention will be described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the examples are only for explaining the present invention and not for limiting it.

Example

The present embodiment provides a kind of end-to-end Tibetan Lag shallow semantic analysis method, which comprises the following steps:

1. Map the input feature sequence with word as the unit and the corresponding tag sequence into a low-dimensional real-valued vector;

2. Install the gated high-speed connection mechanism GM in the vertical direction of LSTM, and use BiLSTM to learn the timing characteristics and contextual semantic information of the input sentence; GM includes linear connections to the input and output of the unit, so that information can be smoothly transferred between different layers Spread between;

3. Use softmax to calculate the local normalized distribution of the semantic label at each moment for the output layer to perform constraint decoding;

4. By enforcing the set BIO and La lattice shallow semantic labeling constraints when using the Viterbi algorithm for decoding, standardize the structural relationship between the output semantic tags.

Since semantic role labeling (semantic role labeling, SRL), as a shallow semantic representation, has the advantages of being simple and easy to use, applicable to multiple languages, and more in-depth research on models and algorithms, and the goal of shallow semantic analysis of Tibetan La grid Similar to the goal of the semantic role labeling task, the model architecture shown in Figure 1 is designed by referring to and drawing on the representative research in the semantic role labeling task, that is, based on the work based on the deep Bi-LSTM and Self-attention framework. It mainly consists of the following parts:

(1) Embedding layer: map the input feature sequence in units of words and the corresponding tag sequence into a low-dimensional real-valued vector;

(2) LSTM layer: for the purpose of improving the learning ability of the model's timing features and enhancing the semantic space expression ability of the model, this embodiment uses BiLSTM to learn the timing features and contextual semantic information of the input sentence;

(3) Gated high-speed link: In order to alleviate the problem of gradient disappearance when training BiLSTM, this embodiment uses GM to control the weight of linear and nonlinear transformation between layers;

(4) Softmax layer: use the Softmax function to map the semantic labels that may be output at each moment into a local normalized distribution of (0,1) for the output layer to perform constraint decoding;

(5) Constrained decoding layer: In order to constrain the structural relationship between the output semantic tags during decoding, this embodiment uses the Viterbi algorithm to enforce the BIO and La grid shallow semantic labeling constraints set in this embodiment when decoding .

embedding layer

表示已训练好的GloVe词向量，用V表示词汇表，

用

表示标记集合，则最原始的输入序列{w₁,w₂,…,w_T}和标记序列{m₁,m₂,…,m_T}通过查找表lookup table映射成低维实值向量e(w_t)和e(m_t)，其中w_t∈V和对应标记m_t∈C；至此，可将向量e(w_t)和e(m_t)拼接成x_l,t作为LSTM第一层的输入：In the shallow semantic analysis task of Tibetan Laga, using

Represents the trained GloVe word vector, and V represents the vocabulary,

use

Represents a tag set, then the most original input sequence {w ₁ ,w ₂ ,…,w _T } and tag sequence {m ₁ ,m ₂ ,…,m _T } are mapped into a low-dimensional real-valued vector e through a lookup table (w _t ) and e(m _t ), where w _t ∈ V and the corresponding label m _t ∈ C; so far, the vectors e(w _t ) and e(m _t ) can be spliced into x _l,t as the first LSTM Layer input:

x _l,t ＝[e(w _t ),e(m _t )]

Among them, x _l,t is the input to the LSTM at the moment t of the first layer, where l=1, t=[1,T].

Bidirectional LSTM

This embodiment transforms the shallow semantic analysis of Tibetan Lag into an end-to-end sequence labeling task. The sequence labeling task is highly dependent on the learning ability of temporal features and text context information, so the first LSTM is used to process the input sentence forward, and then the output of this layer is used as the input of the next layer for reverse processing. Improving the learning ability of time series features and laying the foundation for fully obtaining contextual semantic information at each moment; LSTM is defined as follows:

Among them, δ _l represents the direction of the l-th layer LSTM. When δ _l = -1, the direction of LSTM is forward, and when δ _l = 1, the direction is reverse;

To stack LSTMs in interleaved mode, layer-specific inputs xl _,t and orientation parameters _δl are arranged in the following way:

The input vector xl _,t is the concatenation of the word embedding of the character w _t and the embedding representing whether the word of w _t is a binary feature (t=v) for a given predicate.

LSTM-based GM

In order to alleviate the problem of gradient disappearance when training BiLSTM, GM is installed in the vertical direction of LSTM to control the linear and nonlinear transformation weights between layers, which acts to balance the transmission of information in the vertical direction; λ _l,t represents the GM gating device, the output h _l,t of the hidden layer after using GM can be changed to:

h' _l,t = LSTM(h _l-1,t ,h _l,t-1 ).

For clarity, the difference diagram of LSTM-based GM and plain LSTM is given, see Fig. 2 for details.

In Figure 2, h _l-1,t represents the output of the previous layer and is also the input of the current layer. h′ _l,t represents the candidate output, which is also the output of LSTM. GM linearly connects h _l-1,t and h′ _l,t , which plays a huge role in the high-speed transmission of information in the vertical direction. λ _l,t determines how much information can be transmitted in the next layer to the upper level. During the training process, the closer λ _l,t is to 1, the more information is passed to the next layer. When λ _l,t = 1, the input will be directly copied to the output without any changes, and the GM mechanism , the information at the bottom layer can be transmitted to the top layer more smoothly. Conversely, the closer λ _l,t is to 0, the less information is passed to the next layer. When λ _l,t = 0, GM degenerates into a traditional LSTM. Since GM occurs inside neurons, it does not affect the transmission of underlying information in the time direction.

To reduce overfitting, dropout is used, applied to the hidden state by a shared dropout mask _Dl :

Input the feature sequence x={w ₁ ,w ₂ ,…,w _n } of a La-case sentence, and the log likelihood of the corresponding correct semantic label sequence y={y ₁ ,y ₂ ,…,y _n } is:

Softmax layer

According to the hidden state h _l,t of the model, the local normalized distribution on the output semantic label y _t can be calculated using softmax:

是Kronecker delta，维度与语义标签的个数一致；模型训练目标是在给定输入的时候最大化正确标签的概率。W _o in the above formula is the parameter matrix of softmax,

Is the Kronecker delta, the dimension is consistent with the number of semantic labels; the model training goal is to maximize the probability of the correct label when given the input.

constrained decoding layer

In order to combine constraints on the output structure during decoding, this embodiment sets BIO and La lattice shallow semantic annotation constraints according to the BIO sequence annotation method and the La lattice shallow semantic annotation specification. Finally, the above two constraints are enforced when decoding using the Viterbi algorithm. An example of the constraints is as follows:

(1) BIO constraints

BIO is a commonly used sequence labeling method in NLP. B indicates the beginning of the labeled segment, I indicates the middle or end of the labeled segment, and O indicates other. Its constraints reject any sequence that does not produce a valid BIO transition, such as B-A0 followed by I-A1, etc.

(2) Lag-based shallow semantic annotation constraints

Unique semantic label: Semantic labels A0, A1, A2 and the proprietary labels in Table 1 can only appear once at most for each La-case sentence pattern;

Restricted Semantic Tags: Reject any proprietary tags in Table 1 that cross-appear in different sentence patterns, such as AM-Bas followed by AM-L ₂ , etc.

Sequential semantic labeling: reject any case where a proprietary label AM-L _i sequence in Table 1 appears before another proprietary label, such as AM-L ₁ followed by AM-Bas, etc.

Continuation semantic tag: A continuation semantic tag can only exist if its basic semantic tag is implemented before it, such as B-A0 followed by I-A0, etc.

Table 1 Defining table of proprietary semantic tags and shared semantic tags

在业格、为格、依格、同格和时格五种用法中，原则上可以随机代替其余虚词

和

故统称它们为La格

La格包括自由虚词和不自由虚词两种，其中

为不自由虚词，添接受限于前一音节的后加字，

和

为自由虚词，添接不受前一音节的限制。Function Words in Inlague

In the five usages of ye case, wei case, yi case, tong case and time case, the rest of the function words can be randomly replaced in principle

and

So collectively they are called Lager

La case includes two kinds of free function words and non-free function words, among them

As a non-free function word, Tim accepts the suffix word limited to the previous syllable,

and

As a free function word, the addition is not restricted by the previous syllable.

According to the semantic functions, joining rules and different usages of La-case function words, the usage of La-case function words can be divided into five types of sentences: Biaoye case, Wei case, Yi case, Tong case and Shi case:

实施地动作

和La格虚词

三个主要语义成分，三者缺一不可。如“

(在图书馆学习)”中，实施动作的处所是“

(书店)”，实施地动作是“

(学习)”和La格虚词是

机器若能正确识别并区分这些语义成分，则基本可以正确理解其语义。Dative sentence: A sentence that expresses the action that has been implemented or is being implemented and the action that will be implemented in a certain place of implementation ^[9] . One of the characteristics of karma sentences is that there is an implementation place in the sentence

implement the action

and Lag function words

There are three main semantic components, all of which are indispensable. Such as"

(Learning in the library)", the place to implement the action is "

(bookstore)", the implementation action is "

(Learning)" and La-case function words are

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly.

为达到某一目的而实施地动作行为

和La格虚词

三个主要语义成分，三者缺一不可。如“

(为了获得知识而努力)”中，目的是“

(知识)”，为达到目的而实施地动作行为是“

(努力)”，La格虚词是

机器若能正确识别并区分这些语义成分，则基本可以正确理解其语义。For the clause: a sentence that expresses the implementation of an action to achieve a certain purpose. A characteristic of a case sentence is that there is a purpose in the sentence

an action performed to achieve a purpose

and Lag function words

There are three main semantic components, all of which are indispensable. Such as"

(Efforts to acquire knowledge)", the purpose is "

(knowledge)", the action behavior implemented to achieve the goal is "

(Efforts)", the Lag function word is

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly.

某依存物

和La格虚词

三种主要语义成分。如“

(教室里学生多)”中，地点是“

(教室)”，依存物是“

(学生)”，La格虚词是

机器若能正确识别并区分这些语义成分，则基本可以正确理解其语义；二是当该类句型中的谓词为存在助词

和

等时，可以没有第一个特征中的依存物这一成分，如

中，地点是“

(教室)，存在助词是“

(有)”，La格虚词是

机器若能正确识别并区分这些语义成分，也基本可以正确理解其语义。Accordive sentence: A sentence that expresses that something depends on a certain place where Lag function words are added. There are two main features in the example sentences according to the case. One is that there is a certain place in the sentence where La case function words are added

a dependency

and Lag function words

Three main semantic components. Such as"

(There are many students in the classroom)", the location is "

(Classroom)", the dependency is "

(student)", the Lag function word is

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly;

and

etc., there can be no dependency component in the first feature, such as

, where the location is "

(classroom), the existential particle is "

(Yes)", the Lag function word is

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly.

改变成的另一种事物

和La格虚词

三种主要语义成分。如“

(汉文翻译成藏文)”中，某一事物是“

(汉文)”，变成的另一种事物是“

(藏文)”，La格虚词是

机器若能正确识别并区分这些语义成分，则基本可以正确理解其语义；二是句中有动作行为变化的结果

动作行为

和La格虚词

三种主要语义成分。如“

(让他开心)”中，动作行为变化的结果是“

(藏文)”，动作行为是“

(让)”，La格虚词是

机器若能正确识别并区分这些语义成分，也基本可以正确理解其语义。Congruent sentence: A sentence that expresses the change of one thing into another, so that the two have the property of "identity" in the result of the change of action and behavior. There are two main characteristics of the example sentence in the same case. One is that there is a certain thing in the sentence

changed into something else

and Lag function words

Three main semantic components. Such as"

(Translated from Chinese to Tibetan)", a certain thing is "

(Chinese)", another thing that becomes is "

(Tibetan)", Lag function words are

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly; the second is the result of changes in actions and behaviors in sentences

action behavior

and Lag function words

Three main semantic components. Such as"

(Make him happy)", the result of the action behavior change is "

(Tibetan)", the action behavior is "

(Let)", the Lague function word is

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly.

实施地某一动作行为

和La格虚词

三种主要语义成分，如“

(研究了三年)”中，实施动作行为的时间是“

(三年)”，实施的动作行为是“

(研究)”，La格虚词是

机器若能正确识别并区分这些语义成分，则基本可以正确理解其语义；二是句中只有实施某一动作行为的时间

和La格虚词

两种语义成分，没有实施地某一动作行为

这一语义成分的现象较常见，如“

(他三年)”中，实施动作行为的时间是“

(三年)”，La格虚词是

机器若能正确识别并区分这些语义成分，也基本可以正确理解其语义。Temporal clauses: Sentences expressing the time when an action is performed. There are two main characteristics of this type of sentence. One is that there is a time for performing a certain action in the sentence.

perform an action

and Lag function words

Three main semantic components, such as "

(Three years of research)", the time to implement the action behavior is "

(three years)", the action behavior implemented is "

(research)", the Lag function word is

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly; the second is that there is only time to implement a certain action in a sentence

and Lag function words

Two semantic components, a certain action behavior without implementation

The phenomenon of this semantic component is more common, such as "

(His three years)", the time to implement the action behavior is "

(three years)", the Lag function word is

If the machine can correctly identify and distinguish these semantic components, it can basically understand its semantics correctly.

experiment

Since there is no publicly available Tibetan Lag case shallow semantic analysis data set, this example first extracted 20,000 sentences containing only one Lag case function word from the Tibetan corpus constructed by the laboratory research group. Then, 12,000 sentences were selected after preprocessing these sentences to mark the Lag shallow semantics. Finally, according to the La grid shallow semantic label specification formulated in this embodiment, the construction of the Tibetan La grid shallow semantic analysis data set is completed by manual labeling. In order to facilitate later use, it is referred to as TLSD. During the experiment, the TLSD data set was divided into training set, verification set and test set according to the ratio of 8:1:1.

During the experiment, in order to ensure the comparability of the experimental results, the hyperparameters of all models were adjusted to a limited range. After several times of parameter adjustments, the current optimal hyperparameter combination was finally selected within a limited range. The model The parameters are detailed in Table 2.

Table 2 Model parameters

baseline method

Since there is no reference to the literature on Tibetan Lag shallow semantic analysis, and there is no public Tibetan Lag shallow semantic analysis data set, it is impossible to directly verify the model of this embodiment by comparing it with previous work. Effect.

Based on the above reasons, we will select several methods in the referenced literature when building the model of this embodiment as the baseline model to verify the effect of the model of this embodiment.

(1) LSTM+CRF: It is a semantic role labeling method based on deep bidirectional RNN.

(2) DBLSTM: It is a semantic role labeling method based on deep bidirectional LSTM.

(3) Self-Attention: It is a semantic role labeling method based on the self-attention mechanism.

(4) End-to-end: It is a span-based end-to-end semantic role labeling method.

(5) BiLSTM+GM+CD and BiLSTM+GM+CD(V) are the models of this embodiment, which respectively refer to the shallow semantic analysis model of Tibetan La grid when the predicates in the sentence need to be predicted by the model and given in advance.

Evaluation index

In this embodiment, the accuracy rate (ACC), an evaluation indicator commonly used in sequence labeling tasks, is selected to evaluate the performance of the model. Let TP represent positive samples predicted to be positive, FP to represent negative samples predicted to be positive, FN to represent positive samples predicted to be negative, and TN to represent negative samples predicted to be negative, then the accuracy rate (ACC) is calculated as:

Experimental results and analysis

Performance comparison of models on TLSD dataset

In order to verify the validity and superiority of the model in this embodiment, the Tibetan La grid shallow semantic analysis performance of the baseline model and the model in this embodiment is compared. The experimental comparison results of each model are shown in Table 3.

Table 3 Experimental comparison results of each model

It can be seen from the experimental results in Table 3 that the performance of the model in this example is improved compared with several baseline models. When the model predicts the predicate by itself, the accuracy rate of La lattice shallow semantic analysis on the test set is increased by 3.33, 3.01, 1.58 and 1.87 percentage points, respectively, compared with LSTM+CRF, DBLSTM, Self-Attention and End-to-end. The Tibetan Lag shallow semantic analysis performance of the embodiment model is better. In addition, it can be seen that the model of this embodiment has also achieved good results when jointly predicting predicates and other corresponding semantic labels. On the test set, the accuracy of La lattice shallow semantic analysis is only 0.93 lower than that when predicates are given in advance. %, verified the superiority of the model of this embodiment.

The Tibetan L-case shallow semantic analysis task has achieved good results on several baseline models and the model of this example. There are three main reasons. First, all the sentences in the TLSD dataset are Tibetan single sentences containing only one La-case function word The second is that the sentences in the TLSD dataset are not long, between 4 and 30 words in length; the third is that compared with the semantic role labeling task, Lag's shallow semantic labels are easy to identify and label. In addition, there are two main reasons why the performance of the model in this embodiment is better than that of the baseline model. One is that the device GM balances the transmission of information in the vertical direction, thereby alleviating the problem of gradient disappearance; the other is that the output structure is constrained during decoding. , and a more reasonable output semantic label structure is obtained.

GM Validation Verification

In order to verify the effectiveness of GM, the La lattice shallow semantic analysis effect of the model was examined when using GM's BiLSTM and using ordinary BiLSTM. The first method is to use the ordinary BiLSTM method, and the second method is to use the GM BiLSTM method. The experimental results on the test set are shown in Figure 3.

It can be seen from Figure 3 that the accuracy rate when using GM's BiLSTM is 1.06% higher than when using ordinary BiLSTM, which verifies the effectiveness of GM.

Validation of Constrained Decoding Method

In order to verify the effectiveness of the constrained decoding method, the effect of La lattice shallow semantic analysis of the model is examined with and without constrained decoding. Method 1 is a method without constrained decoding, and method 2 is a method using constrained decoding. The experimental results on the test set are shown in Figure 4.

It can be seen from Figure 4 that the La-lattice shallow semantic analysis accuracy of the model is 0.76% higher when constrained decoding is used than when it is not used, which verifies the effectiveness of the constrained decoding method.

Influence of time series feature learning method on model performance

In order to examine the impact of the time series feature learning method on the performance of the model, the La lattice shallow semantic analysis effect of the model was compared when using LSTM and BiLSTM. Method 1 is the method of using LSTM to learn time-series features, and method 2 is the method of using BiLSTM to learn time-series features. The experimental results on the test set are shown in Figure 5.

It can be seen from Figure 5 that the La lattice shallow semantic analysis accuracy rate of the model when using BiLSTM to learn temporal features is 2.57% higher than that of using LSTM to learn temporal features, indicating that the performance of the model is better when using BiLSTM to learn temporal features.

epilogue

This embodiment proposes an end-to-end long-short memory neural network Tibetan La grid shallow semantic analysis method. In order to alleviate the problem of gradient disappearance and balance the transmission of information in the vertical direction, this embodiment installs GM in the vertical direction of LSTM To mix linear and nonlinear information, so that information spreads more smoothly in space and time dimensions. In order to standardize the structural relationship between the output semantic tags and improve the accuracy of predicted semantic tags, the BIO and La grid shallow semantic tag constraints set in this embodiment are enforced when decoding using the Viterbi algorithm. Experimental results show that, on the test set, the accuracy rate of La lattice shallow semantic analysis of the method in this embodiment reaches 90.59%, and the performance is better than several baseline models.

The above schematically describes the present invention and its implementation, which is not restrictive, and what is shown in the drawings is only one of the implementations of the present invention, and the actual structure is not limited thereto. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the inventive concept of the present invention, without creatively designing a structural mode and embodiment similar to the technical solution, it shall all belong to the protection scope of the present invention .

Claims (6)

1. an end-to-end Tibetan Lag shallow semantic analysis method, is characterized in that: comprise the following steps: 1. Map the input feature sequence with word as the unit and the corresponding tag sequence into a low-dimensional real-valued vector; 2. Install the gated high-speed connection mechanism GM in the vertical direction of LSTM, and use BiLSTM to learn the timing characteristics and contextual semantic information of the input sentence; GM includes linear connections to the input and output of the unit, so that information can be smoothly transferred between different layers Spread between; 3. Use softmax to calculate the local normalized distribution of the semantic label at each moment for the output layer to perform constraint decoding; 4. By enforcing the set BIO and La lattice shallow semantic labeling constraints when using the Viterbi algorithm for decoding, standardize the structural relationship between the output semantic tags. 2. according to a kind of end-to-end Tibetan La grid shallow semantic analysis method described in claim 1, it is characterized in that: in step 1, use

Represents the trained GloVe word vector, and V represents the vocabulary,

Use C∈{0,1} to represent the tag set, then the most original input sequence {w ₁ ,w ₂ ,…,w _T } and the tag sequence {m ₁ ,m ₂ ,…,m _T } are mapped through the lookup table into low-dimensional real-valued vectors e(w _t ) and e(m _t ), where w _t ∈ V and the corresponding marker m _t ∈ C; so far, the vectors e(w _t ) and e(m _t ) can be concatenated into x _{l, t} as the input of the first layer of LSTM: x _l,t ＝[e(w _t ),e(m _t )] Among them, x _l,t is the input to the LSTM at the moment t of the first layer, where l=1, t=[1,T]. 3. according to a kind of end-to-end Tibetan La grid shallow semantic analysis method described in claim 1, it is characterized in that: in step 2, use the sentence of first LSTM forward processing input, then with this layer The output is reversely processed as the input of the next layer, laying the foundation for improving the learning ability of time series features and fully obtaining contextual semantic information at each moment; the definition of LSTM is as follows:

Among them, δ _l represents the direction of the l-th layer LSTM. When δ _l = -1, the direction of LSTM is forward, and when δ _l = 1, the direction is reverse; To stack LSTMs in interleaved mode, layer-specific inputs xl _,t and orientation parameters _δl are arranged in the following way:

The input vector xl _,t is the concatenation of the word embedding of the character w _t and the embedding representing whether the word of w _t is a binary feature (t=v) for a given predicate. 4. according to a kind of end-to-end Tibetan La lattice shallow layer semantic analysis method described in claim 3, it is characterized in that: in step 2, by installing GM on the vertical direction of LSTM to control the linearity and non-linearity between layers Linear transformation weight, its function is to balance the transmission of information in the vertical direction; use λ _l,t to represent the gating device of GM, then the output h _l,t of the hidden layer after using GM can be changed to:

h' _l,t = LSTM(h _l-1,t ,h _l,t-1 ). 5. according to a kind of end-to-end Tibetan La grid shallow semantic analysis method described in claim 4, it is characterized in that: in step 2, in order to reduce overfitting, use dropout rate Dropout, by sharing Dropout mask D _l applied to the hidden state:

Input the feature sequence x={w ₁ ,w ₂ ,…,w _n } of a La-case sentence, and the log likelihood of the corresponding correct semantic label sequence y={y ₁ ,y ₂ ,…,y _n } is:

6. According to a kind of end-to-end Tibetan La grid shallow semantic analysis method described in claim 1, it is characterized in that: in step 3, according to the hidden state h _l,t of the model, use softmax to calculate the output semantic label Locally normalized distribution over y _t :

W _o in the above formula is the parameter matrix of softmax,

Is the Kronecker delta, the dimension is consistent with the number of semantic labels; the model training goal is to maximize the probability of the correct label when given the input.

Images