CN116956944A - An endangered language translation model method integrating syntactic information - Google Patents

Abstract

The invention discloses an endangered language translation model method fusing syntactic information, which comprises the following steps: constructing an endangered language dependency structure tree library in a dependency syntax standard format in a semi-automatic mode; performing dependency syntax analysis on the endangered language based on the double affine classifier, and constructing an endangered language dependency syntax analysis model based on the double affine classifier; and adding the word sequence index, the part of speech label, the dominant word index and the dependency syntax relation label contained in the endangered language dependency structure tree library as syntax features to a machine translation model coding end to construct an endangered language-Chinese neural machine translation model. The translation of the endangered language can be more accurately completed through the syntactic information, the defects that time and labor are wasted, a large amount of expertise is needed, the data size is small, the effect of using a conventional neural machine translation method is poor and the like when the endangered language corpus is marked manually are overcome, and the effectiveness of endangered language translation is greatly improved.

Description

An endangered language translation model method integrating syntactic information

Technical field

The present invention relates to the technical field of artificial intelligence machine translation, and in particular to an endangered language translation model method that integrates syntactic information.

Background technique

Since most endangered languages have scarce corpus resources and no written records, currently the collected recording data can only be transcribed using International Phonetic Alphabets, and then native speakers and linguists can transcribe them into common languages (such as Chinese) can be understood by the public only after being annotated. This phonetic notation method requires a lot of manpower and time, and does not have the scalability of corpus resources.

The technical difficulties faced by existing methods of using neural machine learning to translate endangered languages include:

First, endangered language corpus resources are scarce, there are few native speakers, corpus annotation is difficult and time-consuming, and there is still no standard data set;

Second, the grammatical rules of endangered languages are inconsistent with Chinese, and since there are no written characters, only international phonetic symbols can be used, making it difficult to understand;

Third, manual annotation of endangered language corpora is time-consuming and labor-intensive, requiring a lot of professional knowledge and a small amount of data.

Contents of the invention

In order to overcome the shortcomings of the above-mentioned existing technologies, the present invention provides an endangered language translation model method that integrates syntactic information, constructs an endangered language translation model that integrates syntactic information, and can more accurately complete the translation of endangered languages.

The invention realizes the construction of an endangered language translation model that integrates syntactic information, which includes three aspects: semi-automatic construction of an endangered language dependency structure tree library, realization of endangered language dependency syntax analysis based on a double affine classifier, and establishment of integrated syntax information. An endangered language-Chinese neural machine translation model. The present invention can more accurately complete the translation of endangered languages through syntactic information, and overcomes the problems of manually annotating endangered language corpus, which is time-consuming and laborious, requires a large amount of professional knowledge, has a small amount of data, and has poor effects using conventional neural machine translation methods. .

The invention includes:

A semi-automatic method is used to construct a dependency structure tree library of endangered languages in the CoNLL-U standard format of dependency syntax to promote the construction of language resources for endangered languages; graph-based methods and double affine classifier models integrate multiple language embedding methods and coding model to better understand the language structure of endangered languages; it combines the syntactic information of endangered languages with the machine translation model Transformer to assist in the annotation of endangered languages.

Among them, the endangered language dependency structure treebank is used to analyze the dependency syntax of endangered languages and establish an endangered language-Chinese neural machine translation model;

The endangered language dependency syntax analysis module uses the endangered language dependency structure tree library as experimental data to perform endangered language dependency syntax analysis and support the subsequent establishment of the endangered language-Chinese neural machine translation model, including embedding layer, encoding layer, feature dimensionality reduction layer, double Affine parsing layer and decoding layer;

Establish an endangered language-Chinese neural machine translation model, and add the word order index, part-of-speech tagging, dominant word index, and dependency syntactic relationship annotation contained in the endangered language dependency structure tree library as syntactic features to the encoding end of the Transformer model to build an endangered language-Chinese neural machine translation model. Machine translation model.

The construction of the model of the present invention is divided into the following three steps: semi-automatic construction of endangered language dependency structure tree library; construction of endangered language dependency syntax analysis model based on double affine classifier; construction of endangered language-Chinese neural machine translation that integrates syntactic information Models, specifically including:

1. Semi-automatic construction of endangered language dependency structure treebank: first collect corpus, then pre-process the corpus, annotate part-of-speech and dependency syntactic relationships of the processed corpus, and finally build an endangered language dependency structure treebank and perform manual correction. test;

2. Construct an endangered language dependency syntax analysis model based on a double affine classifier: A graph-based endangered language dependency syntax analysis model TuParser is proposed, which consists of an embedding layer, a coding layer, a feature dimension reduction layer, a double affine parsing layer, The decoding layer completes the dependency syntax analysis of endangered languages;

3. Construct an endangered language-Chinese neural machine translation model: Transformer was selected as the baseline model, and based on it, a neural machine translation model TuSynTRM that integrated syntactic information was designed. On the basis of keeping the structure of the Transformer model unchanged, we focus on strengthening the source-side input features, and combine the word order index, part-of-speech tagging, dominant word index, and dependency tag information contained in the endangered language dependency structure tree library with the coding end of the traditional Transformer model.

Specifically, it includes the following steps:

A semi-automatically constructs an endangered language dependency structure treebank, and obtains an endangered language dependency structure treebank in CoNLL-U format;

B performs dependency syntax analysis on endangered languages based on a double affine classifier, and obtains the dependency structure of sentences in endangered languages.

Specifically, the graph-based endangered language dependency syntax analysis model TuParser is used.

The TuParser model consists of five parts: embedding layer, encoding layer, feature dimensionality reduction layer, biaffine parsing layer and decoding layer;

C established an endangered language-Chinese neural machine translation model that integrates syntactic information, selected Transformer as the baseline model, and designed a neural machine translation model TuSynTRM that integrates syntactic information based on it.

C1 uses the dependency syntax analysis model TuParser in B to determine whether there is a dependency relationship between words in the endangered language sentence, and the corresponding dependency relationship type;

C2 integrates the characteristics of syntactic information (including: endangered language parts of speech, dependencies) embedding and position coding (word order index, dominant word index of endangered languages);

C2.1 uses the two types of syntactic information extracted in A, namely part-of-speech tagging and dependency tagging of endangered languages, as part of the input feature embedding;

C2.2 treats the word order index and dominant word index of endangered languages as position information, and performs position coding on the word order index and dominant word index of endangered languages, allowing the TuSynTRM model to additionally learn syntactic information of endangered languages.

The C3 TuSynTRM model uses the traditional attention mechanism to extract information from endangered languages and Chinese, model the relationship between bilingual languages, and build an endangered language translation model that integrates syntactic information.

C31. Map the input endangered language word vector sequence into three matrices Q, K, and V, and obtain the correlation matrix after matrix multiplication, scaling, and masking operations;

C32. Perform matrix normalization, compare the similarities of Q and K to obtain the weight coefficient, and sum it with V to obtain the self-attention value;

C33. Perform self-attention operations on each self-attention head to obtain single-head output;

C34. Finally, after splicing and linearly transforming the outputs of T self-attention heads, the output of multi-head attention is obtained.

D. Use the constructed endangered language translation model to realize endangered language translation that integrates syntactic information.

Through the above steps, the establishment of the standard database is completed, and the endangered language translation model that integrates syntactic information is completed. This model can more accurately complete the translation of endangered languages through syntactic information, and overcomes the shortcomings of manually annotating endangered language corpus, which is time-consuming and labor-intensive, requires a lot of professional knowledge, has a small amount of data, and has poor results using conventional neural machine translation methods. Greatly improves the effectiveness of endangered language translation.

Description of the drawings

Figure 1 is a structural block diagram of the model TuParser based on the double affine classifier constructed by the present invention.

Figure 2 is a structural block diagram of the TuSynTRM model constructed by the present invention.

Detailed ways

The present invention will be further described below through examples in conjunction with the accompanying drawings.

The method of the invention includes: 1) semi-automatic construction of endangered language dependency structure tree library; 2) endangered language dependency syntax analysis based on double affine classifier; 3) establishing an endangered language-Chinese neural machine translation model that integrates syntactic information.

Specifically, it includes the following steps:

A semi-automatically constructs an endangered language dependency structure treebank, and obtains an endangered language dependency structure treebank in CoNLL-U format. Each endangered language sentence consists of the International Phonetic Alphabet of one or more words, and each endangered language word consists of multiple field information. Representation, including: word order index, part-of-speech tagging, dominant word index, dependency tagging information;

A1 exports corpus, first selects corpus from different categories such as folklore, historical stories, character conversations, etc., and passes these sentences through the ELAN speech annotation software with "International Phonetic Alphabet, Chinese Translation (translate a single word), Chinese Translation (translate the entire sentence) ” is exported as three lines of text.

A2 preprocesses the text exported by A1 including:

A2.1 Remove duplicate corpus, replace all punctuation marks with spaces, and obtain an endangered language-Chinese translation parallel sentence set.

A2.2 Carry out machine automatic segmentation of the endangered language-Chinese parallel sentence pairs obtained in A2.1 using "space" as the separator to obtain single words, and the segmentation can be completely aligned (Chinese annotations and endangered language International Phonetic Alphabet symbols Parallel sentence pairs with one-to-one correspondence) and sentences with errors ((Chinese annotations do not have one-to-one correspondence with International Phonetic Alphabet symbols for endangered languages)) are output separately. Re-segment the sentences with errors and manually align them, and merge the manually adjusted sentences with the sentences obtained after automatic word segmentation by the machine.

A2.3 Use English abbreviation symbols to mark function words in Chinese translation, and replace them with corresponding Chinese words according to the context during Chinese translation.

A3 performs part-of-speech tagging on the words segmented in A2 and obtains part-of-speech tagging for endangered languages.

A4 annotates the preprocessed sentences in A2 to obtain dependency syntactic relationship annotation;

The A5 CoNLL-U format endangered language dependency structure tree library is constructed. Each endangered language sentence obtained by A2 is composed of the International Phonetic Alphabet of one or more words. Each endangered language word is represented by 10 field information, which are:

(1)ID: The International Phonetic Alphabet index of endangered language words, each new sentence is marked starting from the integer 1.

(2)FORM: International Phonetic Alphabet for endangered language words.

(3) LEMMA: The root morpheme of endangered languages, replaced by - here.

(4)UPOSTAG: part-of-speech tag for endangered languages, see A3.

(5)XPOSTAG: A specific part-of-speech tag, replaced by - here.

(6)FEATS: lexical or grammatical features of endangered languages, replaced by - here.

(7) HEAD: The dominant word sequence number of the current endangered language words, which is the value of ID or 0 (that is, the root node).

(8)DEPREL: Endangered language dependency defined, see A4.

(9)DEPS: Secondary dependency relationship, replaced by - here.

(10)MISC: Chinese annotations corresponding to the International Phonetic Alphabet of endangered language words.

A6 manual verification includes verification of Chinese annotations, part-of-speech tagging, and dependency tagging.

B performs dependency syntax analysis on endangered languages based on a double affine classifier, and obtains the dependency structure of sentences in endangered languages.

The Chinese dependency syntax analysis framework designed by Zhang et al. proposed a graph-based endangered language dependency syntax analysis model TuParser. The TuParser model consists of five parts: embedding layer, encoding layer, feature dimensionality reduction layer, biaffine parsing layer and decoding layer;

The B1 embedding layer designs the endangered language input vector method, using e _i to represent the constructed input vector. For an endangered language sentence containing n words W = {w ₁ ,..., _wi ,...,w _n }, where _wi represents the i-th endangered language word. This method uses part-of-speech features to enrich the representation information of the input vector e _i

in Represents endangered language word embedding vector, /> Represents part-of-speech feature vector,/> Represents the vector splicing operation.

The B2 coding layer uses three consecutive BiLSTMs to contextually encode the input vector output by B1. For the i-th endangered language word w _i , the output vector e _i after the embedding layer is sent to the BiLSTM, and the output result of the last layer of the model is Context feature vector r _i .

r _i =BiLSTM(e _i ,θ _bilstm )

Among them, θ _bilstm is the BiLSTM model parameter.

In the feature dimensionality reduction layer, B3 performs nonlinear transformation on the output vector r _i of B2 through the MLP network to remove redundant information irrelevant to the current decision-making, thus improving the overall training speed and analysis accuracy of the TuParser model. The four syntactic feature vectors of endangered languages after dimensionality reduction are expressed by the following formulas:

Among them, MLP ^(*) refers to an independent multi-layer perceptron network, and Respectively represent the characteristics of the dependency arc child node, the characteristics of the dependence arc parent node, and the type of dependence relationship of endangered languages (including: subject-predicate relationship, object-verb relationship, preposition-object relationship, predicate-medial relationship, verb-complement relationship, definite center relationship, orientation relationship, Parallel relationship, double object structure, clause structure, concurrent structure, conjunction structure, function word structure, core relationship) child node characteristics and dependency type parent node characteristics.

B4 will reduce the dimensionality of the syntactic feature vector in B3 Input to the double affine parsing layer, and use the double affine attention mechanism in the dependency arc classifier and dependency relationship classifier respectively to obtain the dependency arc representation score S ^arc between nodes and the classification relationship representation score S ^rel between nodes. The scoring function formula Down:

Among them, U ^(*) is the model weight matrix, H ^(*) represents the syntactic feature matrix of endangered languages in dependency arc prediction and relationship type prediction, and I is the identity matrix.

After B5 obtains the inter-node dependency arc representation score S ^arc and the inter-node classification relationship representation score S ^rel from B4, it uses the first-order Eisner algorithm designed for the dependency syntax analysis task at the decoding layer. The essence of the Eisner algorithm is to find through dynamic programming Maximum spanning tree, that is, by continuously merging the analysis results of adjacent substrings, finally obtains the dependency structure of the entire endangered language sentence.

C established an endangered language-Chinese neural machine translation model that integrates syntactic information, selected Transformer as the baseline model, and designed a neural machine translation model TuSynTRM that integrates syntactic information based on it.

C1 uses the dependency syntax analysis model TuParser in B to determine whether there is a dependency relationship between words in the endangered language sentence, and the corresponding dependency relationship type.

C2 integrates the characteristics of syntactic information (including: endangered language parts of speech, dependencies) embedding and position coding (word order index, dominant word index of endangered languages);

C2.1 takes the endangered language part-of-speech tagging and A4 dependency tagging extracted from A3 as part of the input feature embedding;

For an endangered language sentence containing n words W = {w ₁ ,..., _wi ,...,w _n }, where _wi represents the i-th endangered language word, the input of the fused syntactic information extracted in W Feature embedding e _i can be expressed as Equation 5-1:

in Represents pre-trained endangered language word vectors (representing endangered language words),/> Represents a randomly initialized part-of-speech feature vector, /> Represents the endangered language dependency feature vector,/> Represents the vector splicing operation.

C2.2 treats the word order index and dominant word index of endangered languages as a kind of positional information, allowing the TuSynTRM model to additionally learn syntactic information of endangered languages. Use PEO(·) and PEH(·) to represent the position-encoded functions of the endangered language word order index and dominant word index respectively. The calculation method is as follows:

Among them, pos _order represents the word order index of endangered language words in the sentence, pos _head represents the dominant word index of the current endangered language words, d _model represents the dimension of the position vector, and i represents a certain dimension in the position vector.

The C3 TuSynTRM model uses traditional attention mechanism methods to extract information from endangered languages and Chinese, and model the relationship between bilingual languages.

For the input endangered language word vector sequence H is mapped to three matrices. The three matrices are denoted as Q, K, and V. After matrix multiplication, scaling, and masking operations, the correlation matrix is obtained, and then normalized using the Softmax function, and finally the similarity of Q and K is compared. Weighted summation with V, the self-attention value is obtained. The relevant formula is as follows:

Q＝W _q H

K＝W _k H

V=W _v H

Among them, V is the matrix representing the input features, Q and K are the feature matrices used to calculate the attention self-attention weight; K ^T is the transpose matrix of K, d _k is the number of columns of the Q and K matrices, that is, the vector dimension, D _h represents the length of the input endangered language word vector, is the projection matrix;/> Indicates comparing Q and K similarity.

Multi-head attention consists of multiple self-attentions. The calculation process first maps Q, K, and V into T subsets through linear transformation, as shown in the following formula:

where t represents the t-th head, are all parameter matrices.

Then perform self-attention operations on each head separately to obtain the single-head output head _t , as shown in the following formula:

head _t =Attention(Q _t ,K _t ,V _t )

Finally, after splicing and linearly transforming the outputs of T heads, the output of multi-head attention is obtained, which can be expressed as:

MultiHead(Q,K,V)=Concat(head ₁ ,...,head _T )W _c

where W _c is the weight matrix. MultiHead(Q,K,V) represents the output of multi-head attention;

Concat(head ₁ ,…,head _T ) means concatenating and linearly transforming the output of T heads.

The present invention will be further described below according to the steps and examples:

1 Taking the northern dialect of Tujia as the research object, semi-automatically establishes a dependency structure tree bank.

1.1 Export a total of 6438 Tujia sentences, and export these sentences through the ELAN speech annotation software in the form of three lines of text "International Phonetic Alphabet, Chinese Translation, and Chinese Translation".

1.2 Data preprocessing

1.2.1 After removing duplicate corpus, 6023 sentences were obtained. Subsequently, all punctuation marks appearing in the International Phonetic Symbol layer and the Chinese translation layer are replaced with "spaces"

1.2.2 The Tujia-Chinese parallel sentence pairs obtained after removing duplication and punctuation marks are automatically segmented by machine using "space" as the separator, and the Tujia-Chinese parallel sentence pairs that are fully aligned after automatic segmentation are obtained There are 5102 sentence pairs. Sentences that could not be completely aligned after automatic segmentation were re-segmented and manually aligned. The manually adjusted sentences were merged with the sentences obtained after automatic segmentation by the machine. A total of 6023 parallel sentence pairs were obtained in Tujia-Chinese translation.

1.2.3 According to the grammatical meaning of Tujia function words, English abbreviation symbols are used to mark them, and when translating into Chinese, the function word abbreviation symbols are replaced with corresponding Chinese words according to the context.

1.3 In order to more accurately describe the part-of-speech of each word after Tujia sentence segmentation, 36 Tujia part-of-speech tagging symbols were designed with reference to the part-of-speech tagging instructions of jieba word segmenter and the existing Tujia corpus;

1.4, designed the annotation relationship table of the Tujia language dependency structure treebank. There are 14 types of Tujia language dependency relationships, and the corresponding relationship descriptions and annotation examples are given. The dependence relationships between words in the Tujia language annotation examples are determined by Text is shown in bold.

1.5 Each Tujia sentence consists of the International Phonetic Alphabet of one or more words. Each Tujia word is represented by 10 field information, which are:

(1)ID: The International Phonetic Alphabet index of Tujia words, each new sentence is marked starting from the integer 1.

(2)FORM: The International Phonetic Alphabet of Tujia words.

(3) LEMMA: The root morpheme of Tujia language, replaced by - here.

(4)UPOSTAG: Tujia language part-of-speech tag.

(5)XPOSTAG: A specific part-of-speech tag, replaced by - here.

(6)FEATS: The lexical or grammatical features of Tujia language, replaced by - here.

(7) HEAD: The dominant word sequence number of the current Tujia word, which is the value of ID or 0 (that is, the root node).

(8)DEPREL: Tujia language dependency relationship.

(9)DEPS: Secondary dependency relationship, replaced by - here.

(10)MISC: Chinese annotation corresponding to the International Phonetic Alphabet of Tujia words.

The contents that have been automatically tagged with parts of speech and dependency syntactic relationships are organized, and a Tujia dependency structure treebank is automatically generated according to the above-mentioned CoNLL-U data format by writing a Python script, which contains a total of 6023 corpus items.

1.6 Manual verification, the specific verification ideas are divided into three parts: Chinese annotation verification, part-of-speech tagging verification, and dependency verification.

2. Construct TuParser, a Tujia language dependency syntax analysis model based on double affine classifier;

2.1 The embedding layer combines the language representation vector and the feature vector;

2.2 The encoding layer uses three consecutive BiLSTMs to contextually encode the input vector output by the embedding layer.

2.3 In the feature dimensionality reduction layer, the output vector of the coding layer is nonlinearly transformed through the MLP network.

2.4 Input the dimensionally reduced syntactic feature vector into the double affine parsing layer, and use the double affine attention mechanism in the dependency arc classifier and dependency relationship classifier respectively to obtain the inter-node dependency arc representation score and the inter-node classification relationship representation. Score;

The 2.5 decoding layer uses the first-order Eisner algorithm designed for dependency syntax analysis tasks, and finally obtains the dependency structure of the entire Tujia sentence.

3. Construct a Tujia-Chinese neural machine translation model that integrates syntactic information

3.1 Use the dependency syntax analysis model TuParser to clarify whether there is a dependency relationship between words in endangered language sentences, and the corresponding relationship type.

3.2 Integrate feature embedding and position coding of syntactic information, and extract two types of syntactic information, part-of-speech tagging and dependency tagging of endangered languages, as part of the input feature embedding.

3.3 Treat the word order index and dominant word index of endangered languages as a kind of positional information, allowing the TuSynTRM model to additionally learn syntactic information of endangered languages.

3.4 The TuSynTRM model uses the traditional attention mechanism to extract information from endangered languages and Chinese, and model the relationship between bilingual languages.

Based on the above operations, an endangered language translation model that integrates syntactic information is constructed. It also overcomes the problems of manually annotating endangered language corpus, which is time-consuming and labor-intensive, requires a lot of professional knowledge, has a small amount of data, and has poor results using conventional neural machine translation methods.

Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. of. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined by the claims.

Claims (6)

1. An endangered language translation model method that integrates syntactic information, which is characterized by constructing an endangered language translation model that integrates syntactic information, that is, the word order index, part-of-speech tagging, dominant word index, and dependency syntax included in the endangered language dependency structure tree library Relational annotations are added as syntactic features to the encoding end of the machine translation model Transformer model to build an endangered language-Chinese neural machine translation model; including the following steps: 1) Use a semi-automatic method to construct a dependency structure tree bank of endangered languages in the standard format of dependency syntax; 2) Perform dependency syntax analysis on endangered languages based on a double affine classifier, build a dependency syntax analysis model for endangered languages TuParser based on a double affine classifier, and obtain the dependency structure of endangered language sentences; Graph-based methods and dual affine classifier models, integrating multiple language embedding methods and coding models, are used to understand the structure of endangered languages; 3) Based on the machine translation model Transformer, establish an endangered language-Chinese neural machine translation model TuSynTRM that integrates syntactic information; including: C1 uses the dependency syntax analysis model to determine whether there is a dependency relationship between words in the endangered language sentence, and the corresponding dependency relationship type; C2 integrates feature embedding and position coding of syntactic information; C3 uses the attention mechanism to extract information from endangered languages and Chinese, models the relationship between bilinguals, and builds an endangered language translation model that integrates syntactic information; including: C31. Map the input endangered language word vector sequence into matrices Q, K, V, and obtain the correlation matrix after matrix multiplication, scaling, and masking operations; C32. Perform matrix normalization, compare the similarities of Q and K to obtain the weight coefficient, and sum it with V to obtain the self-attention value; C33. Perform self-attention operations on each self-attention head to obtain single-head output; C34. After splicing and linearly transforming the outputs of multiple self-attention heads, the output of multi-head attention is obtained; D. Use the constructed endangered language translation model to realize endangered language translation that integrates syntactic information. 2. The endangered language translation model method integrating syntactic information according to claim 1, characterized in that in step 1), the standard format is specifically the CoNLL-U format. 3. The endangered language translation model method integrating syntactic information as claimed in claim 1, characterized in that, in step C2, the characteristics of the syntactic information include: endangered language parts of speech and dependencies; the position coding includes word order indexes and dominant words of the endangered language. So. 4. The endangered language translation model method integrating syntactic information as claimed in claim 3, characterized in that the specific process of step C2 includes: C2.1 uses the extracted two types of syntactic information, part-of-speech tagging and dependency tagging of endangered languages, as part of the input feature embedding; C2.2 uses the word order index and dominant word index of endangered languages as position information, and performs position coding on the word order index and dominant word index of endangered languages, so that the model can learn the syntactic information of endangered languages. 5. The endangered language translation model method integrating syntactic information as claimed in claim 1, characterized in that, in step C3, the input endangered language word vector sequence H is mapped into three matrices, The three matrices are recorded as Q, K, and V. After matrix multiplication, scaling, and masking operations, the correlation matrix is obtained, and then normalized using the Softmax function. Finally, the similarity of Q and K is compared with the weighted sum of V to obtain the self- Attention value; expressed as: Q＝W _q H K＝W _k H V=W _v H Among them, V is a matrix representing the input features, Q and K are the feature matrices used to calculate the attention self-attention weight; d _k is the number of columns of the Q and K matrices, that is, the vector dimension, D _h represents the length of the input endangered language word vector, is the projection matrix. 6. The endangered language translation model method integrating syntactic information as claimed in claim 5, characterized in that the process of obtaining multi-head attention is expressed as: Map Q, K, and V into T subsets through linear transformation, as shown below: Among them, t represents the t-th head, Both are parameter matrices; Perform the self-attention operation on each head separately to obtain the single-head output head _t , which is expressed as: head _t =Attention(Q _t ,K _t ,V _t ) After splicing and linearly transforming the outputs of T heads, the output of multi-head attention is obtained, which can be expressed as: MultiHead(Q,K,V)=Concat(head ₁ ,...,head _T )W _c Among them, W _c is the weight matrix; MultiHead (Q, K, V) represents the output of multi-head attention; Concat(head ₁ ,…,head _T ) means concatenating and linearly transforming the output of T heads.