JP2021026760A - Machine translation apparatus and method - Google Patents

Abstract

To provide a new machine translation method and apparatus capable of sharing the same decoding network for different translation directions.SOLUTION: A machine translation device 800 includes: a pre-processing unit which generates a plurality of vectors corresponding to each source language word in an input text of a source language with respect to the input text of the source language; an encoding unit which encodes the plurality of vectors to generate a plurality of encoded vectors; and a decoding unit in which a plurality of encoding vectors and information indicating a translation direction are input into a single decoding network, the output text of the target language corresponding to the input text of the source language is output from the single decoding network, and output order of target language words included in the output text is matched with the translation direction. If the translation direction indicated by the information entered into the single decoding network is changed, parameters of each node in the single decoding network are not changed.SELECTED DRAWING: Figure 8

Description

The present invention relates to the field of natural language processing, and more specifically to machine translation devices and methods.

Machine translation is the process of converting one natural language (source language) into another natural language (target language) using a computer. Machine translation is a field of computational linguistics, one of the ultimate goals of artificial intelligence, and has important scientific research value. At the same time, machine translation also has important practical value. With the globalization of the economy and the rapid development of the Internet, machine translation technology has played an increasingly important role in promoting political, economic and cultural exchanges.

With the great progress of research on deep learning, machine translation (Neural Machine Translation, NMT) based on artificial neural networks has gradually emerged. The core of the technology is a deep neural network that has an extremely large number of nodes (neurons) and can automatically learn translation knowledge from the corpus. Sentences in one language are vectorized and then transmitted layer by layer over the network, transformed into a computer-understandable form of expression, and a multi-layered complex transmission operation that translates into another language. Will be generated. We have realized a translation form such as "understand the language and generate a translation". The biggest advantage of this translation method is that the translation flow is good, it is more consistent with grammatical norms, and it is easier to understand. The quality is "dramatically" improved compared to traditional translation techniques.

Google has provided a new architecture (Transformer) to realize NMT. The Transformer architecture includes two parts: an encoder (Encoder) and a decoder (Decoder). The encoder makes a deep semantic representation of the input text, and the decoder produces the output text based on the semantic representation of the input text. Both the encoder and the decoder are formed by stacking a multi-layer network.

A conventional decoding network can only generate and output a word-by-word translation in a set translation direction (for example, left to right or right to left). That is, one decryption network can only be built and trained in a certain translation direction. If the user wants to change the translation direction, he can only rebuild and train other decryption networks corresponding to the translation direction. However, this clearly has a disadvantage in saving time and software / hardware costs.

In view of the above circumstances, it is desired to provide a new machine translation method and apparatus capable of sharing the same decoding network for different translation directions.

According to one aspect of the present disclosure, a step of executing a process on an input text of the source language so as to generate a plurality of vectors corresponding to each source language word in the input text of the source language, and the plurality of steps. Corresponds to the step of encoding the vector of the above to generate a plurality of coded vectors, the step of inputting the plurality of coded vectors and the information indicating the translation direction into a single decoding network, and the input text of the source language. The output text of the target language to be output is output from the single decoding network, and the output order of the target language words included in the output text matches the translation direction. A machine translation method is provided in which the parameters of each node in the single decoding network are not changed when the translation direction indicated by the information input to the network is changed.

In the method according to the embodiment of the present disclosure, processing is executed for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language. Processing to encode the plurality of training vectors to generate a plurality of coding training vectors, and processing to input the plurality of coding training vectors and information indicating a translation direction into a single decoding network. And, a plurality of training prediction vectors are output from the single decoding network in a plurality of time steps, and each word in the target language system is output in the output text of the target language in one time step for each of the training prediction vectors. The processing corresponding to each word in the output text of the target language, which includes the probability of being a word, and the correct answer text of the target language to be output in a plurality of time steps corresponding to the translation direction. A plurality of correct answer vectors are determined based on the words, and each of the correct answer vectors includes the probability that each word in the target language system is a word in the output text of the target language in one time step, and in that time step. Adjust the parameters of each node in the decryption network based on the first loss function, which indicates the difference between the process with the highest probability of corresponding the word to be output and at least the training prediction vector and the corresponding correct answer vector. The decryption network is trained by the processing.

In the method according to the embodiment of the present disclosure, the single decoding network includes a hidden layer, and is further output from the hidden layer of the decoding network at each time step corresponding to the translation direction. The processing of inputting the intermediate vector to the classification network and outputting the classification result indicating the output order of each word in the training output text of the target language output from the single decoding network from the classification network and the above-mentioned The decoding network is trained by a process of further adjusting the parameters of each node in the decoding network based on the comparison between the output order and the translation direction.

In the method according to the embodiment of the present disclosure, the parameters of each node in the classification network are adjusted based on the second loss function indicating the difference between the classification result output from the classification network and the translation direction. The process trains the classification network.

In the method according to the embodiment of the present disclosure, the step of inputting the plurality of coding vectors and the information indicating the translation direction into a single decoding network sets the plurality of coding vectors and the first translation direction. The source language further includes a step of inputting the indicating information into a single decoding network and a step of inputting the plurality of coding vectors and the information indicating the second translation direction into a single decoding network. The step of outputting the output text of the target language corresponding to the input text of the above from the single decoding network outputs the first output text of the target language from the single decoding network, and the first output. The step in which the output order of the target language words included in the text matches the first translation direction and the second output text of the target language are output from the single decoding network and included in the second output text. It further includes a step in which the output order of the target language words matches the second translation direction.

In the method according to the embodiment of the present disclosure, the decryption network includes a hidden layer, and the method outputs from the hidden layer of the decryption network at each time step corresponding to the first translation direction. The intermediate vector to be input is input to the classification network, each time step corresponds to each word in the first output text of the target language, and each in the first output text of the target language output from the single decoding network. The step of outputting the first classification result indicating the output order of words from the classification network and the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the second translation direction are classified. Input to the network, each time step corresponds to each word in the second output text of the target language, and indicates the output order of each word in the second output text of the target language output from the single decoding network. The step of outputting the second classification result to be performed from the classification network, the value based on the appearance probability of each word in the first output text of the target language, and the first classification result are input to the score network and from the score network. A step of outputting the first score, a value based on the appearance probability of each word in the second output text of the target language, and a step of inputting the second classification result into the score network and outputting the second score from the score network. And, and, the step of outputting the output text of the target language corresponding to the input text of the source language from the single decoding network is the output corresponding to one of the first score and the second score. It also includes a step to select the text as the output text for the target language.

In the method according to the embodiment of the present disclosure, processing is executed for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language. Processing, processing to encode the plurality of training vectors to generate a plurality of encoded training vectors, and inputting the plurality of encoded training vectors and information indicating one translation direction into a single decoding network. And the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the one translation direction is input to the classification network, and the target language output from the single decoding network The process of outputting the classification result indicating the output order of each word in the training output text from the classification network, the value based on the appearance probability of each word in the training output text of the target language, and the classification result are input to the score network. Then, the process of outputting the training score from the score network, the process of calculating the similarity between the training output text of the target language output from the decoding network and the correct answer text of the target language, and the training score The score network is trained by a process of adjusting the parameters of each node in the score network based on a third loss function that indicates the difference between the similarity and the score network.

In the method according to the embodiment of the present disclosure, the correct answer text of the target language further includes the output text of the target language output from another decoding network when the input text of the same source language is input. , The network scale of the other decryption network is larger than the network scale of the decryption network.

In the method according to the embodiment of the present disclosure, the output text of the target language output from the decoding network is used as the correct answer text of the target language corresponding to the input text of the source language, and training processing of another decoding network is performed. The network scale of the other decryption network is smaller than the network scale of the decryption network.

According to another aspect of the present disclosure, before performing processing on the source language input text so as to generate a plurality of vectors corresponding to each source language word in the source language input text. The processing unit, the coding unit for encoding the plurality of vectors to generate a plurality of coded vectors, the plurality of coded vectors, and the information indicating the translation direction are input to a single decoding network. , A decoding unit for outputting the output text of the target language corresponding to the input text of the source language from the single decoding network, and the output order of the target language words included in the output text is the translation direction. A machine in which the parameters of each node in the single decoding network are not changed if the translation direction indicated by the information input to the single decoding network is changed, including a decoding unit that matches. A translation device is provided.

The apparatus according to the embodiment of the present disclosure executes processing on the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language. Processing to encode the plurality of training vectors to generate a plurality of coding training vectors, and processing to input the plurality of coding training vectors and information indicating a translation direction into a single decoding network. And, a plurality of training prediction vectors are output from the single decoding network in a plurality of time steps, and each word in the target language system is output in the output text of the target language in one time step for each of the training prediction vectors. The processing corresponding to each word in the output text of the target language, which includes the probability of being a word, and the correct answer text of the target language to be output in a plurality of time steps corresponding to the translation direction. A plurality of correct answer vectors are determined based on the words, and each of the correct answer vectors includes the probability that each word in the target language system is a word in the output text of the target language in one time step, and in that time step. Adjust the parameters of each node in the decryption network based on the first loss function, which indicates the difference between the process with the highest probability of corresponding the word to be output and at least the training prediction vector and the corresponding correct answer vector. A training unit for training the decryption network is further included.

In the apparatus according to the embodiment of the present disclosure, the single decoding network includes a hidden layer, and the training unit further corresponds to the translation direction of the decoding network at each time step. The intermediate vector output from the hidden layer is input to the classification network, and the classification result indicating the output order of each word in the training output text of the target language output from the single decoding network is output from the classification network. The decryption network is configured to be trained by performing the process of further adjusting the parameters of each node in the decryption network based on the comparison between the output order and the translation direction. Will be done.

In addition, in the apparatus according to the embodiment of the present disclosure, the training unit is further in the classification network based on a second loss function indicating the difference between the classification result output from the classification network and the translation direction. The process of adjusting the parameters of each node is configured to train the classification network.

In the apparatus according to the embodiment of the present disclosure, the decoding unit further inputs the plurality of coding vectors and information indicating the first translation direction into a single decoding network, and the target language is the first. One output text is output from the single decoding network, the output order of the target language words included in the first output text matches the first translation direction, and the plurality of coding vectors and the second translation direction Is input to a single decoding network, the second output text of the target language is output from the single decoding network, and the output order of the target language words included in the second output text is It is configured to match the second translation direction.

In the apparatus according to the embodiment of the present disclosure, the decoding network includes a hidden layer, and the apparatus outputs from the hidden layer of the decoding network at each time step corresponding to the first translation direction. The intermediate vector to be input is input to the classification network, each time step corresponds to each word in the first output text of the target language, and each in the first output text of the target language output from the single decoding network. The process of outputting the first classification result indicating the output order of words from the classification network and the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the second translation direction are classified. Input to the network, each time step corresponds to each word in the second output text of the target language, and indicates the output order of each word in the second output text of the target language output from the single decoding network. The process of outputting the second classification result to be performed from the classification network, the value based on the appearance probability of each word in the first output text of the target language, and the first classification result are input to the score network and from the score network. A process of outputting the first score, a value based on the appearance probability of each word in the second output text of the target language, and a process of inputting the second classification result into the score network and outputting the second score from the score network. And the process of selecting the output text corresponding to one of the first score and the second score as the output text of the target language, and the selection unit for executing the above.

The apparatus according to the embodiment of the present disclosure executes processing on the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language. Processing, processing to encode the plurality of training vectors to generate a plurality of encoded training vectors, and inputting the plurality of encoded training vectors and information indicating one translation direction into a single decoding network. And the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the one translation direction is input to the classification network, and the target language output from the single decoding network The process of outputting the classification result indicating the output order of each word in the training output text from the classification network, the value based on the appearance probability of each word in the training output text of the target language, and the classification result are input to the score network. Then, the process of outputting the training score from the score network, the process of calculating the similarity between the training output text of the target language output from the decoding network and the correct answer text of the target language, and the training score A training unit for training the score network is further included by a process of adjusting the parameters of each node in the score network based on a third loss function indicating the difference between the similarity and the score network.

In the apparatus according to the embodiment of the present disclosure, the correct answer text of the target language further includes the output text of the target language output from another decoding network when the input text of the same source language is input. , The network scale of the other decryption network is larger than the network scale of the decryption network.

In the apparatus according to the embodiment of the present disclosure, the output text of the target language output from the decoding network is used as the correct answer text of the target language corresponding to the input text of the source language, and training processing of another decoding network is performed. The network scale of the other decryption network is smaller than the network scale of the decryption network.

In the machine translation method and the machine translation apparatus according to each embodiment of the present disclosure, the text output of the target language in the first translation direction can be realized by using the same decoding network, and the target language in the second translation direction can be realized. Text output can also be realized. As a result, the time cost and software / hardware cost can be significantly reduced as compared with the conventional method of individually constructing and training a decoding network for a certain translation direction.

It is a flowchart which shows the process of the machine translation method which concerns on one Example of this disclosure. An example of a coded network is shown. The time-ordered input / output schematic diagram of the decryption network is shown. A schematic diagram of the process by which the decoding network determines the words to be output in one time step is shown. It is a flowchart which shows the specific training process of a decryption network. It is a flowchart which shows the process of the machine translation method which concerns on another Example of this disclosure. It is a flowchart which shows the specific training process of a scoring network. It is a functional block diagram which shows the structure of the machine translation apparatus which concerns on embodiment of this disclosure. It is a schematic diagram of the architecture of the exemplary computing device according to the embodiment of the present disclosure.

Hereinafter, each preferred embodiment of the present invention will be described with reference to the drawings. The following description with reference to the drawings is provided to aid in understanding the exemplary embodiments of the invention limited by the claims and their equivalents. It contains various concrete details to aid understanding, but they are only exemplary. Therefore, it can be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and gist of the present invention. In order to make the specification clearer and simpler, detailed description of well-known functions and structures in the art is omitted.

First, a machine translation method according to an embodiment of the present disclosure will be described with reference to FIG. As shown in FIG. 1, the machine translation method includes the following steps.

First, in step S101, processing is executed for the input text of the source language so as to generate a plurality of vectors corresponding to each source language word in the input text of the source language. Traditional machine learning methods often cannot process text data directly, so the input text in the target language must first be converted to numeric data before being input to each subsequent network. For example, the input text of the source language may be one sentence. By executing the word division process on the sentence, the sentence is divided into a plurality of words, and then the plurality of words are converted into word vectors of a specific dimension. For example, this conversion can be realized by a method of word embedding.

Then, in step S102, the plurality of vectors are encoded to generate a plurality of encoded vectors. For example, the coding can be performed by a coding network. FIG. 2 shows an example of a coded network. In FIG. 2, x ₁ , x ₂ , x ₃ , ..., X _m indicate a plurality of vectors obtained by vectorizing the input text of the source language in step S101, and h ₁ , h ₂ , h ₃ , ... , h _m represents a coded vector generated by the encoding network. As a matter of course, the scale of the coded network (for example, the number of nodes and the number of network layers) is not limited to the example of FIG.

Next, in step S103, the plurality of coding vectors and information indicating the translation direction are input to a single decoding network. Here, the translation direction is specified by the user. For example, if the user wants the decryption network to output the translated text in a left-to-right order, the decryption network is populated with information indicating the translation direction from left to right. Alternatively, if the user wants the decryption network to output the translated text in a right-to-left order, the decryption network is populated with information indicating the translation direction from right to left. For example, the information indicating the translation direction may be a multidimensional feature vector (for example, 128 dimensions). The value of each element in the multidimensional feature vector is randomly determined and fixed for the same translation direction.

Finally, in step S104, the output text of the target language corresponding to the input text of the source language is output from the single decoding network, and the output order of the target language words included in the output text is in the translation direction. Match.

Here, when the translation direction indicated by the information input to the single decoding network is changed, the parameters of each node in the single decoding network are not changed.

For example, the parameters of each node in the single decoding network when the translation direction indicated by the information input to the single decoding network is the first translation direction, and the single decoding network. The parameters of each node in the single decoding network when the translation direction indicated by the input information is the second translation direction are the same. In other words, if the same decoding network is used, the output of the target language text in the first translation direction can be realized, and the output of the target language text in the second translation direction can also be realized. As a result, the time cost and the software / hardware cost can be significantly reduced as compared with the method of individually constructing and training the decoding network for a certain translation direction by the conventional technique.

The decryption network sequentially outputs each word in the translated text (that is, the output text of the target language) at predetermined processing time intervals. In the first time step, when the translation direction indicated by the information input to the decoding network in step S103 is from left to right, the decoding network outputs the first target language word from the left. .. Then, in the second time step with a predetermined processing time interval from the first time step, the decoding network continuously outputs the second target language word from the left. The same applies until the decryption network outputs the final target language word. Similarly, in the first time step, when the translation direction indicated by the information input to the decoding network in step S103 is from right to left, the decoding network is the first target language word from the right. Is output. Then, in the second time step with a predetermined processing time interval from the first time step, the decoding network continuously outputs the second target language word from the right. The same applies until the decryption network outputs the final target language word.

The output of the current time step is further fed back to the bottom layer of the decoding network as an input when decoding is performed in the next time step. In other words, the decoding network can further generate the current output based on the preceding output of the decoding network, other than inputting the coding vector from the decoding network and the information indicating the translation direction. ..

FIG. 3 shows a schematic diagram of input / output of the decryption network in chronological order. In FIG. 3, each block shows the same decryption network, but the time steps in which the decryption networks are located are different. In the first time step t ₁ , in addition to the coding vectors h ₁ , h ₂ , h ₃ , ..., H _m , information <L2R> indicating the translation direction is input to the decoding network. Here, <L2R> indicates the translation direction from left to right. The decoding network _{outputs the target language word y 1} corresponding to the translation direction in the _{time step t 1} , and y ₁ is the first word from the left in the translated text. In the second time step _{t 2, h 1,} h _2, h 3 is a coding vector, ..., in addition to _{h m,} receives the output _{y 1} of the first time step _{t 1} the decoding network. Further, the decoding network _{outputs the target language word y 2} corresponding to the translation direction in the _{time step t 2} , and y ₂ is the second word from the left of the translated sentence. At each subsequent time step, the decryption network performs the same process until the decryption network outputs the last target language word corresponding to the translation direction (that is, the last target language word from the left).

Although not shown in Figure 3, similarly, in the first time step t _1, enter information <R2L> indicating the translation direction decoding network, indicating the direction of translation from right to left is <R2L> In this case, the decoding network outputs _{the target language word y 1} corresponding to the translation direction in the _{time step t 1 , and it can be understood that y 1} is the first word from the right in the translated text. In the second time step _{t 2, h 1,} h _2, h 3 is a coding vector, ..., in addition to _{h m,} receives the output _{y 1} of the first time step _{t 1} the decoding network. Further, the decoding network _{outputs the target language word y 2} corresponding to the translation direction in the _{time step t 2} , and y ₂ is the second word from the right in the translated sentence. At each subsequent time step, the decryption network performs the same process until the decryption network outputs the last target language word corresponding to the translation direction (that is, the last target language word from the right).

Next, the details of how the decryption network determines the target language word to be output will be described in detail. Specifically, a plurality of prediction vectors are output from the single decoding network in a plurality of time steps, and each word in the target language thesaurus is output text in the target language in one time step for each of the prediction vectors. It includes the probability of being a word in, and each time step corresponds to each word in the output text of the target language.

FIG. 4 shows a schematic diagram of the process by which the decryption network determines the words to be output in one time step. The first time step will be described as an example. For example, the target language thesaurus may be set as a thesaurus containing 10,000 words. When the input text of the source language is given, the word corresponding to the input text of the source language is selected from the words included in the target language thesaurus and output. In this case, the decoding network outputs a prediction vector having a dimension of 10000 in the first time step.

In FIG. 4, an intermediate vector output from the hidden layer of the decryption network in 401, for example, a floating-point type vector is shown. The intermediate vector is provided to the fully connected layer. The fully connected layer projects the intermediate vector onto a higher dimensional vector. If the target language thesaurus is a thesaurus containing 10000 words (ie, the thesaurus size vocab_size = 10000), this higher dimensional vector is a 10000 dimensional vector. Finally, the Softmax layer normalizes each element in this higher dimensional vector to a probability value between 0 and 1. After normalization, this higher dimensional vector is the prediction vector described above and is shown at 402 in FIG. In FIG. 4, diagonal lines indicate the largest of all probabilities. From this, it can be seen that the probability corresponding to the fifth element is the largest. Assuming the target language thesaurus is an English thesaurus, if the fifth element corresponds to the word "I" in the target language thesaurus, the decryption network outputs the word "I" in the first time step.

The probability of a word output by the decryption network at each time step can be expressed by the following equation (1).

On the other hand, the probabilities of words output by the decoding network corresponding to the translation directions L2R and R2L based on the prior art at each time step are shown by the following equations (2) and (3), respectively.

As can be seen from the above equations (1) to (3), in the method according to the present disclosure, the parameter of the decoding network is similarly θ for different translation directions L2R and R2L. On the other hand, in the method of the prior art, the parameters of the decoding network are different for different translation directions and are θ ^L2R and θ ^R2 L, respectively. In other words, individual decoding networks are provided for different translation directions. Use.

Further, although the first time step has been described as an example with reference to FIG. 4, the processing in the other time steps is similar to the processing in the first time step. The difference is that each time step generates one prediction vector corresponding to the time step, and generally, the prediction vectors output in different time steps are different.

The specific process of the machine translation method according to the present disclosure and a single decoding network capable of sharing the same node parameters for different translation directions have been described above. The machine translation method described above is the process of performing translations based on a trained decryption network. Next, a specific training process of the decryption network will be described with reference to FIG.

First, in step S501, a process is executed for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language.

Then, in step S502, the plurality of training vectors are encoded to generate a plurality of encoded training vectors.

Next, in step S503, the plurality of coding training vectors and information indicating the translation direction are input to a single decoding network.

Then, in step S504, a plurality of training prediction vectors are output from the single decoding network in each of the plurality of time steps, and each word in the target language thesaurus is assigned to each of the training prediction vectors in the target language in one time step. Includes the probability of being a word in the output text of, and each time step corresponds to each word in the output text of the target language.

The above steps S501 to S504 are the same as the steps and processes of the machine translation method described above. The difference is that the input text in any source language is entered after the decryption network has been trained. There is no correct answer text in the target language that corresponds to the input text in the arbitrary source language. On the other hand, for decryption networks in the training process, the source language training text from the training sample library is entered. There is a correct text in the target language that corresponds to the source language training text.

A further difference is that the prediction vector output from the trained decryption network is accurate, in other words, the word corresponding to the element with the highest probability in the prediction vector should be output. The exact word. On the other hand, the training prediction vector output from the decoding network in the training process is not accurate, in other words, the word corresponding to the element with the highest probability in the training prediction vector is not necessarily the accurate word to be output.

Next, in step S505, a plurality of correct answer vectors are determined based on the words of the correct answer texts of the target language to be output in the plurality of time steps corresponding to the translation direction, and each of the correct answer vectors is given the target language. The probability that each word in the thesaurus is a word in the output text of the target language in one time step is included, and the probability corresponding to the word to be output in the time step is the largest.

The first time step will still be described as an example. Assuming that the input translation direction is from left to right, the first word from the left in the correct text is "I", and "I" corresponds to the fifth element in the vector, in the first time step, The correct vector is the vector with the largest value of the fifth element. For example, the value of the fifth element is 1, and the values of the other elements are all 0.

Finally, in step S506, the parameters of each node in the decryption network are adjusted based on at least the first loss function indicating the difference between the training prediction vector and the corresponding correct answer vector.

For example, in one possible embodiment, the first loss function may indicate the difference between the respective values of the elements in the training prediction vector and the respective values of the elements in the correct answer vector. Alternatively, in another possible embodiment, the first loss function may indicate the difference between the element with the highest probability value in the correct answer vector and the corresponding element in the training prediction vector. For example, assuming that the value of the 5th element in the correct answer vector is the largest, for example, the value is 1, the first loss function is the probability value of the 5th element in the training prediction vector and the correct answer. The difference from the probability value of the fifth element in the vector may be shown.

Further, as can be seen from the above explanation, the information indicating the translation direction has the greatest influence on the target language word output in the first time step. In each subsequent time step, information indicating the translation direction is further provided to the decoding network as input, but with the passage of time, a larger weight is given to the relatively close leading output, so that the translation direction is given. The weight of the information that indicates the translation direction becomes smaller and smaller, in other words, the information that indicates the translation direction has less influence on the output target language word. That is, in an architecture that shares the same decryption network, the output of subsequent time steps is not sensitive to the translation direction, and information about the translation direction may be lost. Thus, in another possible embodiment, training for the single decryption network can further add a classification network for auxiliary training of the decryption network.

Specifically, as described above, the single decryption network includes a hidden layer, and after step S506, the decryption network is further trained by the following processing.

In step S507, the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the translation direction is input to the classification network, and the training output of the target language output from the single decoding network is output. The classification result indicating the output order of each word in the text is output from the classification network.

The intermediate vector output from the hidden layer of the decoding network is the intermediate vector 401 described above with reference to FIG. 4, and the hidden layer is the uppermost hidden layer in front of the fully connected layer. The classification network can be realized by a convolutional neural network (CNN). For example, the classification network can be represented by the discriminant function of the following equation (4).

For example, the classification result output from the classification network may be a probability value indicating that the output order of each word in the training output text of the target language is a specific order. The probability value _pdir can be expressed by the following equation (5).

Specifically, when the specific order is set as an order from left to right, if the probability value output from the classification network is 1, the output order of each word in the training output text of the target language is from the left. If the order is to the right and the probability value output from the classification network is 0, the output order of each word in the training output text of the target language is the order from right to left, and the probability of being output from the classification network. If the value is between 0 and 1 and the value is close to 1, then the output order of each word in the training output text of the target language is approximately left-to-right, of which some words. If the output order is disturbed, the probability value output from the classification network is between 0 and 1, and the value is close to 0, then the output order of each word in the training output text of the target language is approximately right to left. The order is to, and the output order of some words is disturbed.

Then, in step S508, the parameters of each node in the decoding network are further adjusted based on the comparison between the output order and the translation direction. Here, the translation direction is the direction indicated by the information input to the decoding network.

That is, the decoding network that is not sensitive to the translation direction is further corrected by the classification network. Specifically, the classification result output via the classification network is input based on the training input text of the source language and the translation direction instructed by the user, and the output order of each word in the training output text of the target language is input. When instructing to match the translation direction to be performed, for example, the translation direction specified by the user is from left to right, and the output order of each word in the training output text of the target language output from the classification network is from left to right. If the probability value in the right order is 1, it is considered that the decryption network does not need to be further adjusted. On the other hand, based on the training input text of the source language and the translation direction instructed by the user, the classification result output via the classification network is the translation in which the output order of each word in the training output text of the target language is input. When indicating that the direction is opposite, for example, the translation direction indicated by the user is left to right, and the output order of each word in the training output text of the target language output from the classification network is from left to right. When the probability value in the order of is 0, it is considered that the decoding network needs to be adjusted significantly. Alternatively, a translation in which the classification result output via the classification network is input based on the training input text of the source language and the translation direction instructed by the user, and the output order of each word in the training output text of the target language is input. When instructing that the direction is approximately the same, for example, the translation direction indicated by the user is left to right, and the output order of each word in the training output text of the target language output from the classification network is from left to right. If the probability value in the order of is 0.8, it is considered necessary to fine-tune the decryption network.

Therefore, in this embodiment, additional classification networks can be added to gradually train until the actual output order of the target language words of the decryption network matches the input, user-instructed translation direction. As such, translation direction information can be learned or used.

Steps S507 and S508 are not necessary steps for training a single decryption network according to the present disclosure and are therefore shown by dashed blocks in FIG.

As with decryption networks, classification networks need to be trained. For example, training the classification network by adjusting the parameters of each node in the classification network based on a second loss function indicating the difference between the classification result output from the classification network and the translation direction. Can be done. Here, the translation direction is the direction indicated by the information input to the decoding network. In this case, the direction instructed by the information input to the decoding network instructed by the user is set as the approximate correct answer of the classification network. As described above, the classification result output from the classification network is a probability value indicating that the word output order is from left to right. For example, when the direction instructed by the information input to the decryption network by the user is from left to right and the probability value output from the classification network is 0.8 with respect to the training data, 1 Adjust the parameters of each node in the classification network as the approximate correct answer for the classification network. Or, for example, when the direction indicated by the information input to the decoding network indicated by the user is from right to left, and the probability value output from the classification network is 0.8 with respect to the training data. , 0 is set as the approximate correct answer of the classification network, and the parameters of each node in the classification network are adjusted.

Alternatively, another possible embodiment is to train the classification network by adjusting the parameters of each node in the classification network based on the difference between the classification result and the actual word output order. it can. In this embodiment, the direction indicated by the information input to the decoding network, which is instructed by the user, is not used as the approximate correct answer of the classification network. In the training process, it is necessary to calculate in advance the probability value that the output order is in the specific translation direction as the correct answer probability value of the classification result of the classification network for the actual word output order. For example, the direction instructed by the information input to the decoding network, which is instructed by the user, is from left to right. For example, y ₁ , y ₂ , y ₃ , and y ₄ are sequentially output and decoded in each time step. When the word order actually output by the network is y ₁ , y ₃ , y ₂ , y ₄ , the probability value that the output order is from left to right is 0.5. In this case, if the probability value output from the classification network is 0.8, 0.5 is set as the correct answer for the classification network, and the parameters of each node in the classification network are adjusted.

The training of the classification network may be performed individually prior to the training of the decryption network. Alternatively, the classification network may be trained at the same time as the decryption network. When trained at the same time, the two networks constrain and coordinate, and training does not end until both loss functions converge.

The embodiment in which the translation direction specified by the user is input to the decoding network and the decoding network outputs the target language word according to the translation direction has been described above. However, the present invention is not limited to this. FIG. 6 shows a flowchart of the process of the machine translation method according to another embodiment of the present disclosure. In another embodiment, the single decoding network may output output texts in two target languages for each of the two translation directions. Then, based on a predetermined rule, a preferable one of the two output texts is selected and output as the final translation.

Specifically, as shown in FIG. 6, the machine translation method includes the following steps.

First, in step S601, a process is executed so as to generate a plurality of vectors corresponding to each source language word in the input text of the source language with respect to the input text of the source language.

Then, in step S602, the plurality of vectors are encoded to generate a plurality of encoded vectors.

The above steps S601 and S602 are the same as steps S101 and S102 in the figure.

Next, in step S603, the plurality of coding vectors and information indicating the first translation direction are input to a single decoding network.

Then, in step S604, the first output text of the target language is output from the single decoding network, and the output order of the target language words included in the first output text matches the first translation direction.

Next, in step S605, the plurality of coding vectors and information indicating the second translation direction are input to a single decoding network.

Then, in step S606, the second output text of the target language is output from the single decoding network, and the output order of the target language words included in the second output text matches the second translation direction.

For example, the first translation direction may be from left to right, and the second translation direction may be from right to left. That is, the translation results corresponding to the two translation directions are acquired in each of the steps S603 to S606.

Then, in step S607, the first output text and the second output text are graded, respectively, and the first score and the second score are obtained.

For example, by constructing a scoring network, scoring for the output text can be realized.

Specifically, the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the first translation direction is input to the classification network, and each time step is in the first output text of the target language. The first classification result corresponding to each word and indicating the output order of each word in the first output text of the target language output from the single decoding network is output from the classification network. Here, the classification network may be the classification network described above, and as described above, the classification result output from the classification network can be shown as a _dir. Then, the value based on the appearance probability of each word in the first output text of the target language and the first classification result are input to the score network, and the first score is output from the score network. Here, for example, the value based on the appearance probability of each word in the first output text of the target language may be the average value of the probabilities of all the output target language words. As described above, when the target language words output in each time step are indicated by y ₁ , y ₂ , ..., Y _{n , respectively, the probability of the target language words y i} output in the current time step is Sc (y). _i ), which can be calculated by the following equation (6).

The scoring network can be represented by the following equation (7).

Similarly, an intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the second translation direction is input to the classification network, and each time step is each in the second output text of the target language. The second classification result corresponding to the words and indicating the output order of each word in the second output text of the target language output from the single decoding network is output from the classification network. Then, the value based on the appearance probability of each word in the second output text of the target language and the second classification result are input to the score network, and the second score is output from the score network.

Finally, in step S608, the output text corresponding to the larger one of the first score and the second score is selected as the output text of the target language.

Therefore, in the other embodiment, when the user does not need to participate, a more preferable one is dynamically selected from the output texts corresponding to the two translation directions and output as the final translation. Can be done.

In the other embodiment, the decoding network training method is completely the same as the decoding network training method in the above-described one embodiment, and the contents described above are decoded in the other embodiment. Fully applicable to training networks. Further, in order to avoid redundancy, detailed description is omitted here.

Similar to the decryption and classification networks described above, the scoring network also needs to be trained before use. Next, a specific training process of the score network will be described with reference to FIG. 7. As shown in FIG. 7, the score network can be trained by the following steps.

First, in step S701, a process is executed for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language.

Then, in step S702, the plurality of training vectors are encoded to generate a plurality of encoded training vectors.

Next, in step S703, the plurality of coding training vectors and information indicating one translation direction are input to a single decoding network.

Then, in step S704, the intermediate vector output from the hidden layer of the decoding network in each time step corresponding to the one translation direction is input to the classification network, and the target output from the single decoding network is input. The classification result indicating the output order of each word in the training output text of the language is output from the classification network.

Then, in step S705, the value based on the appearance probability of each word in the training output text of the target language and the classification result are input to the score network, and the training score is output from the score network. When starting training, the training scores output from the score network are not accurate.

Next, in step S706, the degree of similarity between the training output text of the target language output from the decryption network and the correct answer text of the target language is calculated. For example, the similarity can be calculated by using a BLEU (bilingual evaluation evaluation) algorithm as an automatic evaluation method for machine translation. The closer the translation of the machine translation is to the result of the manual translation (that is, the correct text of the target language), the higher the translation quality and the higher the BLEU score obtained.

Finally, in step S707, the parameters of each node in the rating network are adjusted based on a third loss function that indicates the difference between the training score and the similarity. That is, a predetermined number of times by using the similarity (for example, BLEU score), which is an accurate criterion for evaluating the translation quality of the training output text of the target language, as a calibration reference of the score output from the score network. After training, the scoring network can output a score that accurately evaluates the translation quality of the translation output from the decoding network.

The machine translation method according to the two examples of the present disclosure has been described above. In one embodiment, the decoding network outputs the output text (translation) of the corresponding target language based on the input translation direction, that is, the translation finally output by the decoding network is first. Associated with the translation direction indicated by the user. In another embodiment, two different translation directions are input to the decoding network, and one of the two different translations corresponding to the two different translation directions is selected to have a more favorable translation quality, and the final translation is completed. Output as. That is, in the other embodiment, it is considered unnecessary to input the translation direction into the decoding network.

In the decryption network training, the manually translated text for the training input text of the source language can be used as the correct answer text of the target language, and the correct answer text of the target language is when the input text of the same source language is input. , The output text of the target language output from another decryption network may be further included, and the network scale of the other decryption network is larger than the network scale of the decryption network. Here, the network scale is a network parameter such as the number of nodes and the number of network layers. Unsurprisingly, the other decryption network is a trained network.

However, T indicates the number of training samples.

In some cases, the correct manual translation text is difficult to achieve, while the output text of the target language from another decryption network can be more easily achieved, so the output of a larger alternative decryption network. By adding as the correct answer, it is possible to realize knowledge transfer from another large-scale decoding network to a small-scale decoding network, and to realize a variety of knowledge for learning. For example, another large decoding network may be a decoding network performed on a large device, while the small decoding network may be a decoding network performed on a small portable device. Thereby, by transferring the knowledge learned by another large decoding network directly to the small decoding network, the time required to complete the training of the small decoding network can be effectively shortened, and the decoding can be performed. It is possible to avoid the situation where the loss function of the network cannot converge for a long period of time.

As described above, the decoding network according to the present disclosure can perform training as a small-scale decoding network by using the output text of the target language of another large-scale decoding network as the correct answer. On the other hand, the output text of the target language output by the decoding network according to the present disclosure after the training is completed may be applied to the training process of the decoding network having a small scale as the correct answer text as well.

Specifically, the output text of the target language output from the decoding network is applied to the training process of another decoding network as the correct answer text of the target language corresponding to the input text of the source language, and the other decoding is performed. The network scale of the decryption network is smaller than the network scale of the decryption network.

As will be proved in the experiment, when the network is trained with the same training data, the machine translation method according to each embodiment of the present disclosure ensures the same decoding speed as compared with the method of the prior art. Further, the BLEU score can be improved by about 1 to 2.5 points.

The specific process of the machine translation method according to each embodiment of the present disclosure has been described in detail with reference to the drawings. Next, the machine translation apparatus according to the embodiment of the present disclosure will be described with reference to FIG.

As shown in FIG. 8, the machine translation apparatus 800 includes a preprocessing unit 801, a coding unit 802, and a decoding unit 803.

The preprocessing unit 801 is used to perform processing on the input text of the source language so as to generate a plurality of vectors corresponding to each source language word in the input text of the source language. Traditional machine learning methods often cannot process text data directly, so the input text in the target language must first be converted to numeric data before being input to each subsequent network. For example, the input text of the source language may be one sentence. By executing the word division process on the sentence, the sentence is divided into a plurality of words, and then the plurality of words are converted into word vectors of a specific dimension. For example, this conversion can be realized by a method of word embedding.

The coding unit 802 is used to encode the plurality of vectors to generate a plurality of coded vectors. For example, the coding can be performed by the coding network described above with reference to FIG.

The decoding unit 803 inputs the plurality of coding vectors and information indicating the translation direction into a single decoding network, and decodes the output text of the target language corresponding to the input text of the source language into the single decoding network. Used to output from the network, the output order of the target language words contained in the output text matches the translation direction, and the translation direction indicated by the information input to the single decoding network is changed. If so, the parameters of each node in the single decryption network are unchanged.

Here, as described above, the translation direction is specified by the user. For example, if the user wants the decryption network to output the translated text in a left-to-right order, the decryption network is populated with information indicating the translation direction from left to right. Alternatively, if the user wants the decryption network to output the translated text in a right-to-left order, the decryption network is populated with information indicating the translation direction from right to left. For example, the information indicating the translation direction may be a multidimensional feature vector (for example, 128 dimensions). The value of each element in the multidimensional feature vector is randomly determined and fixed for the same translation direction.

Here, when the translation direction indicated by the information input to the single decoding network is changed, the parameters of each node in the single decoding network are not changed.

For example, the parameters of each node in the single decoding network when the translation direction indicated by the information input to the single decoding network is the first translation direction, and the single decoding network. The parameters of each node in the single decoding network when the translation direction indicated by the input information is the second translation direction are the same. In other words, if the same decoding network is used, the output of the target language text in the first translation direction can be realized, and the output of the target language text in the second translation direction can also be realized. As a result, the time cost and the software / hardware cost can be significantly reduced as compared with the method of individually constructing and training the decoding network for a certain translation direction by the conventional technique.

The decryption network sequentially outputs each word in the translated text (that is, the output text of the target language) at predetermined processing time intervals. In the first time step, when the translation direction indicated by the information input to the decoding network is from left to right, the decoding network outputs the first target language word from the left. Then, in the second time step with a predetermined processing time interval from the first time step, the decoding network continuously outputs the second target language word from the left. The same applies until the decryption network outputs the final target language word. Similarly, in the first time step, when the translation direction indicated by the information input to the decoding network is from right to left, the decoding network outputs the first target language word from the right. Then, in the second time step with a predetermined processing time interval from the first time step, the decoding network continues to output the second target language word from the right. The same applies until the decryption network outputs the final target language word.

The output of the current time step is further fed back to the bottom layer of the decoding network as an input when decoding is performed in the next time step. In other words, the decryption network can still generate the current output based on the predecessor output of the decryption network, except to input the coding vector from the decoding network and the information indicating the translation direction. ..

Further, as described above, the decoding network outputs a plurality of prediction vectors at each time step, and for each of the prediction vectors, each word in the target language thesaurus is a word in the output text of the target language in one time step. Each time step corresponds to each word in the output text of the target language. In each time step, the word in the thesaurus corresponding to the element having the highest probability in the prediction vector corresponding to the time step is selected and output as a translation.

The specific configuration of the machine translation apparatus according to the present disclosure and a single decoding network capable of sharing the same node parameters for different translation directions have been described above. The machine translation device described above performs translations based on a trained decryption network. Therefore, the apparatus includes a process of executing a process for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language, and the plurality of processes. The process of encoding the training vector of the above to generate a plurality of coded training vectors, the process of inputting the plurality of coded training vectors and the information indicating the translation direction into a single decoding network, and a plurality of training predictions. Each vector is output from the single decoding network in multiple time steps, and each training prediction vector has a probability that each word in the target language system is a word in the output text of the target language in one time step. Multiple, based on the processing corresponding to each word in the output text of the target language, each time step being included, and the correct text word of the target language to be output in the plurality of time steps corresponding to the translation direction. The correct answer vector is determined, and each of the correct answer vectors includes the probability that each word in the target language system is a word in the output text of the target language in one time step, and the word to be output in the time step. Executes the process of adjusting the parameters of each node in the decryption network based on the first loss function indicating the difference between the process having the highest corresponding probability and at least the training prediction vector and the corresponding correct answer vector. By doing so, a training unit 804 for training the decryption network may be further included.

Further, as can be seen from the above explanation, the information indicating the translation direction has the greatest influence on the target language word output in the first time step. In each subsequent time step, information indicating the translation direction is further provided to the decoding network as input, but with the passage of time, a larger weight is given to the relatively close leading output, so that the translation direction is given. The weight of the information that indicates the translation direction becomes smaller and smaller, in other words, the information that indicates the translation direction has less influence on the output target language word. That is, in an architecture that shares the same decryption network, the output of subsequent time steps is not sensitive to the translation direction, and information about the translation direction may be lost. Thus, in another possible embodiment, training for the single decryption network can further add a classification network for auxiliary training of the decryption network.

Specifically, the single decryption network includes a hidden layer, and the training unit 804 is further output from the hidden layer of the decryption network at each time step corresponding to the translation direction. A process of inputting an intermediate vector to the classification network and outputting a classification result indicating the output order of each word in the training output text of the target language output from the single decoding network from the classification network and the output. It is configured to train the decryption network by performing a process of further adjusting the parameters of each node in the decryption network based on the comparison of the sequence with the translation direction. Here, the translation direction is the direction indicated by the information input to the decoding network.

The intermediate vector output from the hidden layer of the decoding network is the intermediate vector 401 described above with reference to FIG. 4, and the hidden layer is the uppermost hidden layer in front of the fully connected layer. The classification network can be realized by a convolutional neural network (CNN).

Specifically, when the specific order is set as an order from left to right, if the probability value output from the classification network is 1, the output order of each word in the training output text of the target language is from the left. If the order is to the right and the probability value output from the classification network is 0, the output order of each word in the training output text of the target language is from right to left, and the probability of being output from the classification network. If the value is between 0 and 1 and the value is close to 1, then the output order of each word in the training output text of the target language is approximately left-to-right, of which some words. If the output order is disturbed and the probability value output from the classification network is between 0 and 1, and the value is close to 0, the output order of each word in the training output text of the target language is approximately from right to left. The output order of some words is disturbed.

That is, the decoding network that is not sensitive to the translation direction is further corrected by the classification network. Specifically, the classification result output via the classification network is input based on the training input text of the source language and the translation direction instructed by the user, and the output order of each word in the training output text of the target language is input. When instructing to match the translation direction to be performed, for example, the translation direction specified by the user is from left to right, and the output order of each word in the training output text of the target language output from the classification network is from left to right. If the probability value in the right order is 1, it is considered that the decryption network does not need to be further adjusted. On the other hand, based on the training input text of the source language and the translation direction instructed by the user, the classification result output via the classification network is the translation in which the output order of each word in the training output text of the target language is input. When indicating that the direction is opposite, for example, the translation direction indicated by the user is left to right, and the output order of each word in the training output text of the target language output from the classification network is from left to right. When the probability value in the order of is 0, it is considered that the decoding network needs to be adjusted significantly. Alternatively, a translation in which the classification result output via the classification network is input based on the training input text of the source language and the translation direction instructed by the user, and the output order of each word in the training output text of the target language is input. When instructing that the direction is approximately the same, for example, the translation direction indicated by the user is left to right, and the output order of each word in the training output text of the target language output from the classification network is from left to right. If the probability value in the order of is 0.8, it is considered necessary to fine-tune the decryption network.

Therefore, in this embodiment, additional classification networks can be added to gradually train until the actual output order of the target language words of the decryption network matches the input, user-instructed translation direction. As such, translation direction information can be learned or used.

As with decryption networks, classification networks need to be trained. For example, the training unit 804 may perform the process of adjusting the parameters of each node in the classification network based on the second loss function indicating the difference between the classification result output from the classification network and the translation direction. Can train classification networks. Here, the translation direction is the direction indicated by the information input to the decoding network. In this case, the direction instructed by the information input to the decoding network instructed by the user is set as the approximate correct answer of the classification network. As described above, the classification result output from the classification network is a probability value indicating that the word output order is from left to right. For example, when the direction instructed by the information input to the decryption network by the user is from left to right and the probability value output from the classification network is 0.8 with respect to the training data, 1 Adjust the parameters of each node in the classification network as the approximate correct answer for the classification network. Or, for example, when the direction indicated by the information input to the decoding network indicated by the user is from right to left, and the probability value output from the classification network is 0.8 with respect to the training data. , 0 is set as the approximate correct answer of the classification network, and the parameters of each node in the classification network are adjusted.

Alternatively, another possible embodiment is to train the classification network by adjusting the parameters of each node in the classification network based on the difference between the classification result and the actual word output order. it can. In this embodiment, the direction indicated by the information input to the decoding network, which is instructed by the user, is not used as the approximate correct answer of the classification network. In the training process, it is necessary to calculate in advance the probability value that the output order is in the specific translation direction for the actual word output order as the correct answer probability value of the classification result of the classification network. For example, the direction instructed by the information input to the decoding network, which is instructed by the user, is from left to right. For example, y ₁ , y ₂ , y ₃ , and y ₄ are sequentially output and decoded in each time step. When the word order actually output by the network is y ₁ , y ₃ , y ₂ , y ₄ , the probability value that the output order is from left to right is 0.5. In this case, if the probability value output from the classification network is 0.8, 0.5 is set as the correct answer for the classification network, and the parameters of each node in the classification network are adjusted.

The training of the classification network may be performed individually prior to the training of the decryption network. Alternatively, the classification network may be trained at the same time as the decryption network. When trained at the same time, the two networks constrain and coordinate, and training does not end until both loss functions converge.

The embodiment in which the translation direction specified by the user is input to the decoding network and the decoding network outputs the target language word according to the translation direction has been described above. However, the present invention is not limited to this.

According to another embodiment of the present disclosure, the decoding unit 803 further inputs the plurality of coding vectors and information indicating the first translation direction into a single decoding network, and the target language. The first output text is output from the single decoding network, the output order of the target language words included in the first output text matches the first translation direction, and the plurality of coded vectors and the second translation The information indicating the direction is input to a single decoding network, the second output text of the target language is output from the single decoding network, and the output order of the target language words included in the second output text. May be configured to match the second translation direction.

It can now be seen that in the other embodiment, the single decoding network may output output text in two target languages for each of the two translation directions.

Then, the decoding unit 803 further selects and outputs a selection unit (not shown) for selecting and outputting a preferable one of the two output texts as a final translation based on a predetermined rule. It may be included.

Specifically, the selection unit inputs an intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the first translation direction into the classification network, and each time step is the first of the target language. A process of outputting the first classification result corresponding to each word in one output text and instructing the output order of each word in the first output text of the target language output from the single decoding network from the classification network. And, the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the second translation direction is input to the classification network, and each time step is each word in the second output text of the target language. The process of outputting the second classification result indicating the output order of each word in the second output text of the target language output from the single decoding network from the classification network and the target language. A process of inputting a value based on the appearance probability of each word in the first output text and the first classification result into the score network and outputting the first score from the score network, and each word in the second output text of the target language. The process of inputting the value based on the appearance probability of and the second classification result into the score network and outputting the second score from the score network, and the output text corresponding to the larger one of the first score and the second score. It is configured to perform the process of selecting as the output text of the target language.

Therefore, in the other embodiment, when the user does not need to participate, a more preferable one is dynamically selected from the output texts corresponding to the two translation directions and output as the final translation. Can be done.

Similar to the decryption and classification networks described above, the scoring network also needs to be trained before use.

Specifically, the training unit 804 performs processing on the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language. And the process of encoding the plurality of training vectors to generate a plurality of coded training vectors, and the process of inputting the plurality of coded training vectors and information indicating one translation direction into a single decoding network. And, the intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the one translation direction is input to the classification network, and the training output of the target language output from the single decoding network is input. The process of outputting the classification result indicating the output order of each word in the text from the classification network, the value based on the appearance probability of each word in the training output text of the target language, and the classification result are input to the score network. The process of outputting the training score from the score network, the process of calculating the similarity between the training output text of the target language output from the decoding network and the correct answer text of the target language, and the process of calculating the training score and the similarity. The score network can be trained by the process of adjusting the parameters of each node in the score network based on a third loss function that indicates the difference between degrees.

The machine translation apparatus according to the two embodiments of the present disclosure has been described above. In one embodiment, the decoding network outputs the output text (translation) of the corresponding target language based on the input translation direction, that is, the translation finally output by the decoding network is first. Associated with the translation direction indicated by the user. In another embodiment, two different translation directions are input to the decoding network, and one of the two different translations corresponding to the two different translation directions is selected to have a more favorable translation quality, and the final translation is completed. Output as. That is, in the other embodiment, it is considered that the decoding network does not need to input the translation direction.

In the decryption network training, the manually translated text for the training input text of the source language can be used as the correct answer text of the target language, and the correct answer text of the target language is when the input text of the same source language is input. , The output text of the target language output from another decryption network may be further included, and the network scale of the other decryption network is larger than the network scale of the decryption network. Here, the network scale is a network parameter such as the number of nodes and the number of network layers. Unsurprisingly, the other decryption network is a trained network.

On the other hand, the output text of the target language output by the decoding network according to the present disclosure after the training is completed may be applied to the training process of the decoding network having a small scale as the correct answer text as well.

Specifically, the output text of the target language output from the decoding network is applied to the training process of another decoding network as the correct answer text of the target language corresponding to the input text of the source language, and the other decoding is performed. The network scale of the decryption network is smaller than the network scale of the decryption network.

Since the machine translation apparatus according to the embodiment of the present disclosure fully corresponds to the machine translation method described above, many details are not explained in the description of the machine translation apparatus. It can be appreciated by those skilled in the art that all the details of the machine translation method described above are similarly applicable to machine translation equipment.

Further, the method or apparatus according to the embodiment of the present disclosure may be realized by the architecture of the calculation apparatus 900 shown in FIG. As shown in FIG. 9, the calculation device 900 includes a bus 910, one or more CPUs 920, a read-only memory (ROM) 930, a random access memory (RAM) 940, and a communication port 950 connected to the network. , The input / output unit 960, the hard disk 970, and the like may be included. The storage device in the calculator 900, such as the ROM 930 or the hard disk 970, can store various data or files used in the processing and / or communication of the machine translation method according to the present disclosure, and program instructions executed by the CPU. .. Of course, the architecture shown in FIG. 9 is merely exemplary, and when implementing different devices, one or more units of the calculator shown in FIG. 9 may be used, depending on the actual needs. It may be omitted.

The embodiments of the present disclosure may be implemented as a computer-readable storage medium. Computer-readable instructions are stored in the computer-readable storage medium according to the embodiment of the present disclosure. When the computer-readable instruction is executed by the processor, the machine translation method according to the embodiment of the present disclosure described above can be executed with reference to the above drawings. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache (cache). The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.

As described above, the machine translation method and the machine translation apparatus according to each embodiment of the present disclosure have been described in detail with reference to FIGS. 1 to 9. In the machine translation method and the machine translation apparatus according to each embodiment of the present disclosure, the text output of the target language in the first translation direction can be realized by using the same decoding network, and the target language in the second translation direction can be realized. Text output can also be realized. As a result, the time cost and software / hardware cost can be significantly reduced as compared with the method of individually constructing and training the decoding network for a certain translation direction in the prior art, and the decoding network can be reduced. In the training, by adding more classification networks, the translation can be gradually trained until the actual output order of the target language words of the decryption network matches the input, user-instructed translation direction. If the direction information can be learned or used and the user does not need to participate, the final translation is made by dynamically selecting the more preferred one from the two translation results corresponding to different translation directions. Finally, by transferring the knowledge learned by another large decoding network directly to the small decoding network, a variety of knowledge for learning is realized, and training of the small decoding network is realized. The time to complete can be effectively reduced, and the situation where the loss function of the decryption network cannot converge for a long period of time can be avoided.

It should be noted that herein, the terms "included", "included" or any other variation thereof are defined not only by a process, method, article or device containing a set of elements, but also containing those elements. It is intended to cover non-exclusive inclusion to further include other elements not listed in, or to further include elements specific to such processes, methods, articles or devices. Unless otherwise restricted, the elements limited by the statement "contains ..." do not preclude the presence of other identical elements in the process, method, article or device containing the element.

Finally, the series of processes described above includes not only processes executed in chronological order in the order described here, but also processes executed in parallel or individually rather than in chronological order. It should be noted.

Through the description of the above embodiments, those skilled in the art will understand that the present invention may be implemented by a combination of software and a required hardware platform, and of course, all may be implemented by software. it can. Based on this understanding, all or part of the technical proposal of the present invention that contributes to the background art may be a computer device (which may be a personal computer, a server, a network device, etc.) according to each embodiment of the present invention. Or in the form of a computer software product that includes several instructions and can be stored on a storage medium such as a ROM / RAM, floppy disk, disk, etc., used to perform the methods described in some of the examples. It can be embodied.

The present invention has been described in detail above. In the present specification, the principles and embodiments of the present invention have been interpreted using specific examples, but the above description of the examples is merely for assisting in understanding the method and its gist of the present invention. Absent. Further, it is obvious to those skilled in the art that there are changes in the specific embodiments and scope of application based on the gist of the present invention. Therefore, the content of this specification should not be understood as a limitation to the present invention.

Claims (18)

A preprocessing unit for performing processing on the source language input text so as to generate a plurality of vectors corresponding to each source language word in the source language input text.
A coding unit for encoding the plurality of vectors to generate a plurality of coded vectors, and
To input the plurality of coding vectors and information indicating the translation direction into a single decoding network, and to output the output text of the target language corresponding to the input text of the source language from the single decoding network. The decoding unit includes a decoding unit in which the output order of the target language words included in the output text matches the translation direction.
A machine translation device in which the parameters of each node in the single decoding network are not changed when the translation direction indicated by the information input to the single decoding network is changed. A process of executing a process for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language.
The process of encoding the plurality of training vectors to generate a plurality of coded training vectors, and
The process of inputting the plurality of coding training vectors and information indicating the translation direction into a single decoding network, and
A plurality of training prediction vectors are output from the single decoding network in a plurality of time steps, and each word in the target language thesaurus is used as a word in the output text of the target language in one time step for each of the training prediction vectors. Each time step corresponds to each word in the output text of the target language, including the probability that it will be
A plurality of correct answer vectors are determined based on the words of the correct answer text of the target language to be output in the plurality of time steps corresponding to the translation directions, and each of the correct answer vectors has one word in the target language thesaurus. The processing that includes the probability of being a word in the output text of the target language in the time step and has the highest probability of corresponding to the word to be output in the time step.
By performing at least the process of adjusting the parameters of each node in the decryption network based on the first loss function indicating the difference between the training prediction vector and the corresponding correct answer vector, The device of claim 1, further comprising a training unit for training. The single decryption network includes a hidden layer, the training unit further
The intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the translation direction is input to the classification network, and each word in the training output text of the target language output from the single decoding network. The process of outputting the classification result indicating the output order of the above from the classification network and
A claim configured to train the decryption network by performing a process of further adjusting the parameters of each node in the decryption network based on a comparison of the output sequence with the translation direction. Item 2. The device according to item 2. The training unit further
The classification network is configured to be trained by the process of adjusting the parameters of each node in the classification network based on the second loss function indicating the difference between the classification result output from the classification network and the translation direction. The device according to claim 3. The decoding unit further
The plurality of coding vectors and the information indicating the first translation direction are input to a single decoding network, the first output text of the target language is output from the single decoding network, and the first output text is output. The output order of the target language words included in the above matches the first translation direction.
The plurality of coding vectors and the information indicating the second translation direction are input to a single decoding network, the second output text of the target language is output from the single decoding network, and the second output. The device according to claim 1, wherein the output order of the target language words included in the text is configured to match the second translation direction. The decryption network includes a hidden layer, and the device is
The intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the first translation direction is input to the classification network, and each time step corresponds to each word in the first output text of the target language. , A process of outputting the first classification result indicating the output order of each word in the first output text of the target language output from the single decoding network from the classification network.
An intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the second translation direction is input to the classification network, and each time step corresponds to each word in the second output text of the target language. Then, a process of outputting the second classification result indicating the output order of each word in the second output text of the target language output from the single decoding network from the classification network, and
A process of inputting a value based on the appearance probability of each word in the first output text of the target language and a first classification result into the score network and outputting the first score from the score network.
A process of inputting a value based on the appearance probability of each word in the second output text of the target language and a second classification result into the score network and outputting the second score from the score network.
The apparatus according to claim 5, further comprising a process of selecting an output text corresponding to one of the larger one of the first score and the second score as the output text of the target language, and a selection unit for executing the process. A process of executing a process for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language.
The process of encoding the plurality of training vectors to generate a plurality of coded training vectors, and
The process of inputting the plurality of coding training vectors and the information indicating one translation direction into a single decoding network, and
The intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the one translation direction is input to the classification network, and in the training output text of the target language output from the single decoding network. A process of outputting the classification result indicating the output order of each word from the classification network and
A process of inputting a value based on the appearance probability of each word in the training output text of the target language and a classification result into the score network and outputting the training score from the score network.
The process of calculating the similarity between the training output text of the target language output from the decryption network and the correct text of the target language, and
A training unit for training the score network is further included by a process of adjusting the parameters of each node in the score network based on a third loss function indicating the difference between the training score and the similarity. , The apparatus according to claim 6. The correct answer text of the target language further includes the output text of the target language output from another decoding network when the input text of the same source language is input.
The device according to claim 1, wherein the network scale of the other decryption network is larger than the network scale of the decryption network. The output text of the target language output from the decoding network is applied to the training process of another decoding network as the correct answer text of the target language corresponding to the input text of the source language.
The device according to claim 1, wherein the network scale of the other decryption network is smaller than the network scale of the decryption network. A step of executing processing on the input text of the source language so as to generate a plurality of vectors corresponding to each source language word in the input text of the source language.
A step of encoding the plurality of vectors to generate a plurality of coded vectors, and
A step of inputting the plurality of coding vectors and information indicating the translation direction into a single decoding network, and
The output text of the target language corresponding to the input text of the source language is output from the single decoding network, and the output order of the target language words included in the output text matches the translation direction. ,
A machine translation method in which the parameters of each node in the single decoding network are not changed when the translation direction indicated by the information input to the single decoding network is changed. A process of executing a process for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language.
The process of encoding the plurality of training vectors to generate a plurality of coded training vectors, and
The process of inputting the plurality of coding training vectors and information indicating the translation direction into a single decoding network, and
A plurality of training prediction vectors are output from the single decoding network in a plurality of time steps, and each word in the target language thesaurus is used as a word in the output text of the target language in one time step for each of the training prediction vectors. Each time step corresponds to each word in the output text of the target language, including the probability that it will be
A plurality of correct answer vectors are determined based on the words of the correct answer text of the target language to be output in the plurality of time steps corresponding to the translation directions, and each of the correct answer vectors has one word in the target language thesaurus. The processing that includes the probability of being a word in the output text of the target language in the time step and has the highest probability of corresponding to the word to be output in the time step.
Claim that the decrypted network is trained by a process of adjusting the parameters of each node in the decrypted network based on at least a first loss function indicating the difference between the training prediction vector and the corresponding correct answer vector. 10. The method according to 10. The single decryption network contains a hidden layer, and further
The intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the translation direction is input to the classification network, and each word in the training output text of the target language output from the single decoding network. The process of outputting the classification result indicating the output order of the above from the classification network and
11. The method of claim 11, wherein the decoding network is trained by a process of further adjusting the parameters of each node in the decoding network based on a comparison of the output order with the translation direction. Claim that the classification network is trained by the process of adjusting the parameters of each node in the classification network based on the second loss function indicating the difference between the classification result output from the classification network and the translation direction. 12. The method according to 12. The step of inputting the plurality of coding vectors and the information indicating the translation direction into a single decoding network is
A step of inputting the plurality of coding vectors and information indicating the first translation direction into a single decoding network, and
Further including the step of inputting the plurality of coding vectors and the information indicating the second translation direction into a single decoding network.
The step of outputting the output text of the target language corresponding to the input text of the source language from the single decoding network is
A step in which the first output text of the target language is output from the single decoding network and the output order of the target language words included in the first output text matches the first translation direction.
A step of outputting the second output text of the target language from the single decoding network and the output order of the target language words included in the second output text matches the second translation direction is further included. The method according to claim 10. The decryption network includes a hidden layer, the method of which
The intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the first translation direction is input to the classification network, and each time step corresponds to each word in the first output text of the target language. , A step of outputting the first classification result indicating the output order of each word in the first output text of the target language output from the single decoding network from the classification network.
An intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the second translation direction is input to the classification network, and each time step corresponds to each word in the second output text of the target language. Then, a step of outputting the second classification result indicating the output order of each word in the second output text of the target language output from the single decoding network from the classification network, and
A step of inputting a value based on the appearance probability of each word in the first output text of the target language and the first classification result into the score network and outputting the first score from the score network.
Further including a value based on the appearance probability of each word in the second output text of the target language, and a step of inputting the second classification result into the score network and outputting the second score from the score network.
The step of outputting the output text of the target language corresponding to the input text of the source language from the single decoding network is
14. The method of claim 14, further comprising selecting the output text corresponding to one of the first and second scores as the output text of the target language. A process of executing a process for the training input text of the source language so as to generate a plurality of training vectors corresponding to each source language word in the training input text of the source language.
The process of encoding the plurality of training vectors to generate a plurality of coded training vectors, and
The process of inputting the plurality of coding training vectors and the information indicating one translation direction into a single decoding network, and
The intermediate vector output from the hidden layer of the decoding network at each time step corresponding to the one translation direction is input to the classification network, and in the training output text of the target language output from the single decoding network. A process of outputting the classification result indicating the output order of each word from the classification network and
A process of inputting a value based on the appearance probability of each word in the training output text of the target language and a classification result into the score network and outputting the training score from the score network.
The process of calculating the similarity between the training output text of the target language output from the decryption network and the correct text of the target language, and
15. The 15th claim, wherein the score network is trained by a process of adjusting the parameters of each node in the score network based on a third loss function indicating the difference between the training score and the similarity. Method. The correct answer text of the target language further includes the output text of the target language output from another decoding network when the input text of the same source language is input.
The method according to claim 10, wherein the network scale of the other decryption network is larger than the network scale of the decryption network. The output text of the target language output from the decoding network is applied to the training process of another decoding network as the correct answer text of the target language corresponding to the input text of the source language.
The method according to claim 10, wherein the network scale of the other decryption network is smaller than the network scale of the decryption network.

Images